A Look Behind the AI Curtain: How AI Works and Language Models Learn

Few technologies have arrived wrapped in as much mystique as AI. It comes with a flood of terminology (tokens, transformers, embeddings, parameters, fine-tuning, reasoning, agents, context windows) and with dozens of companies and hundreds of products, programs and models whose names change faster than anyone can learn them. Under that cloud, most people are left with the impression that whatever happens inside is beyond understanding: that it is a kind of magic, and that the only choices are to trust it or to fear it.

It's not magic. It's software. Claude, and every other system of its kind, is at heart a very large program for one small task: given some text, guess what comes next. Everything else, the conversation, the code, the explanations, the pictures, the teams of agents, is built on that one idea, trained into it by an enormous amount of practice. This book looks through the cloud and takes the machine apart, one piece and one term at a time, with tiny working copies you can poke at along the way, and at the end puts the pieces back together into one small assistant you can talk to. Once you can see what is actually happening, it stops looking like magic, and you can take control of it: judge what it produces, know when to trust it and when not to, and have a say in how it is used, rather than leaving all of that to the people who build it.

FIRST

Pretraining

Read trillions of words, guessing each next piece and being corrected. This is where the knowledge and the language come from.

THEN

Post-training

Teach the guesser to behave like an assistant: follow instructions, prefer helpful and honest answers, think before answering, use tools.

EVERY TIME YOU TYPE

Inference

Your words go in as numbers, the model produces one piece at a time, and each piece is fed back in to produce the next.

We start with the eighty-year story of how the idea arrived and who carried it, then with text becoming numbers and a model that only counts. From there we build up the transformer one part at a time, train it, and watch what scale buys. The second half of the book is about the second half of building a model: turning a document continuer into an assistant that answers, that knows what kind of answer is wanted, that thinks, uses tools and works in a team of agents; then the same trick for code, for pictures, and for video, both reading and making them. The third part sorts out which model is which, where the open models live and how to run one yourself, why newer models are better, what happens when you press Enter, and where it all goes wrong. The last part walks through the trainer, shows how to grow the book's tiny model into a real one you could train yourself, and wires everything from the book into one working assistant.

YOU NEED

Curiosity

And a little patience with sliders. Some chapters show short JavaScript, which you can read as plain instructions or skip.

YOU DON'T NEED

A math background

Every idea is explained in words and pictures first. Where a formula appears, it comes with a "plain words" panel.

TOOLS

A browser

The figures run in the page. The trainer and one chapter use WebGPU, which current Chrome, Edge, Safari and Firefox support.

If you would like a preview of where all this is going, here is the whole thing happening at once. Don't worry about the chart or the numbers yet; each piece gets its own chapter, and by the end of the book every part of this picture will make sense.

step 0 error —

Watch it learn. A real training run, recorded from the trainer that comes with this book. On the left, how wrong the model's guesses are as it trains: it starts as 145,343 random numbers and the only thing it is ever told is "guess the next piece of text, and here is how wrong you were." On the right, what it writes when handed the single word "Mara" and asked to keep going: letter salad at first, then words, then sentences about an invented town, and by the end it can answer a question about that town. The last text is written live, by the finished model running in your browser. A model this small still slips now and then, so the page checks each try against the town's facts and shows the first one with no mistakes. Every idea in this figure gets its own chapter; this is just the whole thing happening at once.

Everything was built from scratch in plain JavaScript and WebGPU, with no libraries. The models were trained by a single-file trainer that runs in your browser, and every number in the book came out of it.

a note on how this book was made

AI was used to help author this book: Claude helped write the text, the interactive figures, the code samples, and the synthetic training examples, under the author's direction and review, and the cover illustration was made with Google Gemini. That seemed fitting for a book whose purpose is to help you understand how AI works, and it is also a reason to read with the usual care. The models in the figures were trained only on text and pictures generated by the book's own code and on public-domain novels; the claims about real systems are the author's understanding as of this writing, and the sources are listed in the final chapter.

Contents