Nothing you type leaves this page. The tokenizer and the word model learn from four paragraphs of Ada Lovelace’s 1843 notes on the Analytical Engine (public domain), live in your tab. The word vectors are 165 words from Stanford’s GloVe (public domain under the PDDL). The attention head and the networks are toys, small enough to see through.
Model workshop
The observatory tells you how a language model works. The workshop hands you the parts: five small working models in this tab, from the tokenizer to the training that tunes every dial.
Workshop exploredThat’s every part of the loop. If you want to see how they fit back together, the observatory is next door.
Chop text into tokens
A model never reads letters or words. It reads numbers, one per token, from a fixed vocabulary. Most vocabularies come from byte-pair encoding: start with the 256 possible bytes, glue the most common neighbouring pair into a new token, and repeat. Common words end up as one token. Rare ones shatter into pieces.
This tokenizer just trained itself in your tab on four paragraphs of Lovelace. Type something and watch it split your text. Drag the slider to rebuild the vocabulary from bare bytes, one merge at a time.
Training…
␣ marks a space and ↵ a line break. A tile tinted red, like <D0>, is a single raw byte that no merge has absorbed.
In real modelsGPT-style tokenizers run the same algorithm but learn 50,000 to 200,000 merges from far more text. Tokens are what you get billed for and what a context window counts, and many languages need more tokens per sentence than English (Petrov and colleagues, 2023).
DefenceA look-alike letter or an invisible character changes the tokens while the text looks the same. Normalise text before you check it (Unicode NFKC), flag mixed scripts and confusables (UTS 39), and strip or reveal invisible characters.
Words as points in space
Each token id picks a row from a table of vectors: long lists of numbers, learned in training. Words used in similar ways end up pointing in similar directions, so likeness becomes an angle you can measure. The cosine of that angle is 1 when two vectors point the same way and near 0 when they have little in common.
These are real vectors: 165 words from GloVe, 50 numbers each, learned by Stanford researchers from six billion tokens of Wikipedia and newswire. Pick a word and it moves to the centre of the orrery. Its nearest neighbours orbit at a distance set by their cosine, so the tighter the orbit, the more alike the words.
Vectors add. king − man + woman starts at king, takes away what man contributes, adds what woman does, and lands nearest king itself. Leave out the three words you typed and queen is the closest. That’s the usual answer: the highest-cosine word that isn’t one of the inputs.
Run a sum and the orrery recentres on where it lands.
Analogy code normally refuses to answer with any word you typed. Allow it and king − man + woman returns king, and doctor − man + woman returns doctor. Word vectors do absorb biases from their text (Caliskan and colleagues measured them in GloVe, 2017), but the famous analogies overstate them (Nissim and colleagues, 2020).
In real modelsA language model learns its own table with thousands of numbers per token, and attention then reshapes each vector for its context. GloVe has one vector per word, all its senses mixed together, which is why virus sits nearer cow than malware here.
DefenceEmbeddings leak what they encode. Researchers have rebuilt much of the original text from sentence embeddings alone (Morris and colleagues, 2023), so protect a vector database like the documents it came from.
Every word looks at every word
Reading one word at a time loses the thread. Attention lets every word look at every other word, all at once, and borrow meaning from the ones that matter.
Each word turns its vector into three smaller ones: a query (what am I looking for?), a key (what am I about?) and a value (what do I pass on if chosen?).
One word’s score for another is its query dotted with the other’s key, divided by the square root of how many numbers each holds. Softmax turns a row of scores into positive weights that add up to 1. It exponentiates first, so the biggest score wins by a lot. A word’s output is everyone’s values, blended by those weights.
This head is a toy, wired by hand. We chose its weights so a pronoun looks for the noun it stands for. Pick the word that’s looking.
Show the vectors and weights
In real modelsA large model runs dozens of heads in every layer, over dozens of layers, each with learned weights. Nobody wires them by hand. Some heads still end up attending to the previous token (Olsson and colleagues, 2022), and some link a pronoun to what it refers to (Clark and colleagues, 2019).
DefenceAttention can’t tell who wrote a token. Instructions hidden in a web page, an email or a file get attended to just like yours, which is why prompt injection works. The defences you can rely on live outside the model. Keep untrusted content apart from instructions, grant only the tools the task needs, and put a human in front of anything irreversible.
Roll the weighted die
A model never writes a word. It scores every token in its vocabulary, softmax turns the scores into probabilities, and a sampler rolls a weighted die. You hold the sampler’s three dials.
Temperature divides every score before the softmax. Below 1 it sharpens the odds toward the favourite, above 1 it flattens them, and at 0 the favourite always wins. Top-k keeps only the k likeliest words. Top-p keeps the smallest set of top words whose probabilities add up to at least p. Whatever survives is rescaled to add up to 1, and rolled.
The model here only counts which word follows which pair of words in four Lovelace paragraphs, blended so every word keeps a chance. At temperature 0 it recites her.
Dashed outline: what the model said. Solid bar: what the die will use after your dials.
In real modelsThe vocabulary holds 50,000 to 200,000 tokens and the scores come from the network, but the dials are the same. Libraries apply them in different orders, and this one applies temperature, then top-k, then top-p.
DefenceA probability isn’t a fact. At temperature 0 a model states its favourite continuation with total confidence, right or wrong, and even then the output can change between runs. Treat generated text as untrusted input and validate it before it runs code, moves money or sends a message.
Watch a network learn
Nobody sets a network’s dials by hand. Training does it: guess, measure the miss, nudge every dial a little in the direction that shrinks the miss, repeat. The miss is the loss. Backpropagation works out which way each dial should turn, and the learning rate sets how far.
This network is tiny. It takes a point’s two coordinates, passes them through one or two layers of tanh neurons, and outputs the probability that the point is gold. Press play and watch the boundary bend to fit. Then turn attacker: click the field to plant gold points where blue belongs. it’ll believe you lol.
- Step
- 0
- Loss
- -
- Accuracy
- -
- Poisoned
- none
- Dials
- -
In real modelsThe same loop with billions of dials, a fresh batch of examples at every step, and optimisers such as Adam that adapt each dial’s step size. A frontier model trains for weeks to months on thousands of accelerators.
DefenceThat was data poisoning: a few mislabelled points teach an exception while clean accuracy barely moves. A backdoor adds a trigger, as when Gu and colleagues (2017) made a small sticker flip an image classifier. Know where every training example came from, look for outliers and duplicates, and test on inputs chosen to trigger hidden behaviour.