The book will keep changing as I find mistakes, improve figures, and add material. The changes are
listed here, newest first.
More code in the later chapters.
New code samples, each run as printed: the agent loop with two tools (Chapter 13), the tiny assistant's manager
(Chapter 14), retrieval in about forty lines (Chapter 15), an attention head with a key-value cache and the
quantizer the fewer-bits figure runs (Chapter 24), and the answerer's confidence check (Chapter 25).
Chapter 3 shows the whole program that invents Harbor Town and writes its text, instead of a few templates.
Chapter 12's mixed model gets a trace's printed result wrong, and the figure now says so and why.
A review, and a real model.
A chapter-by-chapter accuracy review fixed several dozen errors: example sentences that contradicted Harbor Town,
trainer code excerpts that had drifted, history and credits, and overstatements about Adam, scaling laws,
position encodings and speculative decoding.
New chapter 27, From the Tiny Model to a Real One: modern block parts, cleaning a dataset, stable training,
checking a run, and a costed recipe for a 124-million and a 1-billion-parameter model, with a recipe calculator and
a near-duplicate detector.
The assistant and the afterword are now chapters 28 and 29.
New material throughout: special tokens, perplexity, teacher forcing, pretraining data, the reinforcement step,
reward hacking, tool calls as tokens, MCP, computer use, benchmarks, speech, image editing, video provenance,
pricing, fine-tuning an open model, memory-bound generation, memorization, interpretability, and a route to
building your own.
Every figure checked at phone and desktop widths, and fixed where labels collided or were too small.
A new front page: a cover illustration, an opening that says it is not magic but software, and the training
replay moved below the introduction.
Without WebGPU, pages now explain why it might be missing and fall back: recorded GPU results in the Chapter 9
race, recordings of the trainer and the GPU playground, and the real-model demos running on the CPU.
A real codebase, and a restyle.
New chapter 17, Training on a Real Codebase: the three code-training stages the synthetic model skipped,
done on the TypeScript of earth_explorer, an MIT-licensed WebGPU climate map.
A model trained on its files in import order, with fill-in-the-middle documents, completes gaps in files it never
saw; a figure measures what repository context buys.
stage7_agent.mjs runs an agent loop against the project's real tests, and a small model trained on
330 of those runs drives the read, search, edit, test loop in the page.
New title, A Look Behind the AI Curtain, and a new look distinct from the graphics book.
Chapter 9: WebGPU's coming subgroup matrices. Chapter 3: where Harbor Town comes from. A clearer cover figure.
A Look Behind the AI Curtain.
How We Got Here: eighty years of history and people, an interactive timeline and model growth.
One Stream of Tokens: how a model tells answering, reading a picture and writing code apart, with a model
trained on prose and programs together.
Many Agents and a Manager: orchestrators, subagents, pipelines and critics, with a live manager.
Moving Pictures: video generation, with a clip made from the sprite model.
Open Models and Hugging Face: open weights, the hub, file formats and runtimes, and a real
135-million-parameter model in the page.
A Tiny Assistant, End to End: every model in the book in one chat box, and the same design for the Claude API.
The code model was retrained on four kinds of program and now reads and fixes functions too.
The drawing chapter gained a color sprite world, a 600-thousand-parameter generator, and a measured section on
where new pictures come from.
The book edition.
The single long article became a book of twenty-one chapters, sharing the runtime of
Computer Graphics: Past to Present: a chapter drawer, search,
source buttons and editable playgrounds.
A second tiny model, trained on synthetic programs, with tests run in the page.
A live CLIP-style trainer and a patch-token figure for reading pictures.
A live diffusion model that learns to draw shapes from a word, with classifier-free guidance.
Chapters on how the major models differ, on serving a model (cache and quantization, measured), and on where
models go wrong, with a live calibration figure.
A kernel-by-kernel walk through the WebGPU trainer.
First published.
The original single-page explainer, with the WebGPU trainer, the Harbor Town model, and fifteen interactive figures.