Model workshop
The observatory tells you how a language model works. The workshop hands you the parts. Five small working models, each running in this tab: the tokenizer that chops text into pieces, the space where words become places, the attention that lets words look at each other, the weighted die that picks the next word, and the training that tunes every dial.
Nothing you type leaves this page. The tokenizer and the word model learn, live, from four paragraphs of Ada Lovelace’s 1843 notes on the Analytical Engine (public domain). The word vectors are 165 words from Stanford’s GloVe (public domain under the PDDL). The attention head and the networks are toys, built small enough to see through.
Workshop exploredYou have taken apart every stage of the loop: text to tokens, tokens to vectors, vectors through attention, scores to a roll of the die, and the training that set it all. The observatory is where the parts fit back together.
Chop text into tokens
A model never reads letters or words. It reads numbers, one per token, from a fixed vocabulary. Most modern models build that vocabulary with byte-pair encoding or a close variant: start from the 256 possible bytes, find the pair of neighbours that turns up most often in a pile of text, glue it into a new token, and repeat. Common words end up as one token. Rare ones shatter into pieces.
This tokenizer has just trained itself, in your tab, on four paragraphs of Lovelace. Type below and watch your text split. Drag the slider to rebuild the vocabulary from bare bytes, one merge at a time.
Training…
␣ marks a space and ↵ a line break. A tile tinted red, like <D0>, is a single raw byte that no merge has absorbed.
In real modelsGPT-style tokenizers run this same algorithm on bytes, with the same kind of splitting pattern, but learn 50,000 to 200,000 merges from far more text. Tokens are what you are billed for and what a context window counts. Text unlike the training sample costs more tokens: this one compresses Lovelace well and modern English badly, and real tokenizers make many languages pay more per sentence than English (Petrov and colleagues, 2023).
DefenceA look-alike letter or an invisible character changes the tokens while the text looks the same, so a filter that compares strings can miss what the model reads. Normalise text before you check it (Unicode NFKC folds full-width and other compatibility forms), flag mixed scripts and confusable characters (Unicode Technical Standard 39), and strip or reveal invisible ones.
Meaning becomes a place
Each token id picks a row from a table of vectors: long lists of numbers, learned in training. Words used in similar ways end up pointing in similar directions, so likeness becomes an angle you can measure. Its cosine is 1 when two vectors point the same way and near 0 when they have little in common.
These are real vectors: 165 words from GloVe, 50 numbers each, which Stanford researchers learned from six billion tokens of Wikipedia and newswire. Pick a word to put it at the centre of the orrery. Its nearest neighbours orbit at a distance set by their cosine: the tighter the orbit, the more alike the words.
Vectors add. king − man + woman starts at king, takes away what man contributes, adds what woman does, and lands nearest king itself. Leave out the three words you typed and queen is the closest: the usual answer is the highest-cosine word that is not one of the inputs.
Run a sum: the orrery recentres on the landing point.
Analogy code normally refuses to answer with any word you typed. Allow it and king − man + woman returns king, and doctor − man + woman returns doctor, not nurse. Word vectors do absorb biases from their text (Caliskan and colleagues measured them in GloVe, 2017), but the famous analogies overstate them (Nissim and colleagues, 2020).
In real modelsA language model learns its own table, with thousands of numbers per token, and the table is only the starting point: attention then reshapes each vector by its context. GloVe has one vector per word for every sense at once, which is why virus sits nearest to cow here, and only then malware.
DefenceEmbeddings leak what they encode. Researchers have rebuilt much of the original text from sentence embeddings alone (Morris and colleagues, 2023), so a vector database deserves the same protection as the documents it was built from.
Every word looks at every word
Reading one word at a time loses the thread. Attention lets every word look at every other word, all at once, and borrow meaning from the ones that matter.
Each word turns its vector into three smaller ones: a query (what am I looking for?), a key (what am I about?) and a value (what do I pass on if chosen?). One word’s score for another is its query dotted with the other’s key, divided by the square root of how many numbers each holds. Softmax turns a row of scores into weights that are positive and add up to 1; it exponentiates first, so the biggest score wins by a lot. The word’s output is everyone’s values, blended by those weights.
This head is a toy, wired by hand: we chose its weights so that a pronoun looks for the noun it stands for. Pick the word that is looking.
Show the vectors and weights
In real modelsA large model runs dozens of heads side by side in every layer, over dozens of layers, each with learned weights and vectors of 64 to 128 numbers. Nobody wires them by hand, yet researchers find heads doing jobs like these: some attend to the previous token (Olsson and colleagues, 2022), some link a pronoun to what it refers to (Clark and colleagues, 2019). Real models mark position with position vectors or rotations of the queries and keys, not this toy’s one slot per position.
DefenceAttention cannot tell who wrote a token. Instructions hidden in a web page, an email or a file sit in the same sequence as yours and get attended to just the same, which is why prompt injection works. The dependable defences live outside the model: keep untrusted content apart from instructions, give the model only the tools and permissions the task needs, and put a human confirmation in front of anything irreversible.
Roll the weighted die
A model never writes a word. It scores every token in its vocabulary, softmax turns the scores into probabilities, and a sampler rolls a weighted die. You hold the sampler’s three dials.
Temperature divides every score before the softmax: below 1 it sharpens the odds toward the favourite, above 1 it flattens them toward the long tail, and at 0 the favourite always wins. Top-k keeps only the k likeliest words. Top-p keeps the smallest set of top words whose probabilities add up to at least p. Whatever survives is rescaled to add up to 1, and rolled.
The model here is small and honest: it counted which word follows which pair of words in the same four Lovelace paragraphs, blended with pair and single-word counts so every word keeps a chance. At temperature 0 it recites her.
Dashed outline: what the model said. Solid bar: what the die will use after your dials.
In real modelsThe vocabulary holds 50,000 to 200,000 tokens and the scores come from the network, not from counts, but the dials are the same: temperature and top-p in most model APIs, top-k in some. Libraries apply them in different orders; this one applies temperature, then top-k, then top-p.
DefenceA probability is not a fact. At temperature 0 a model states its favourite continuation with total confidence, right or wrong, and even then the output can change between runs. Treat generated text as untrusted input and validate it before it runs code, moves money or sends a message.
Watch a network learn
Nobody sets a network’s dials by hand. Training does it: guess, measure the miss, nudge every dial a little in the direction that shrinks the miss, repeat. The miss is the loss. Backpropagation works out which way each dial should turn; the learning rate sets how far.
This network is tiny. It takes a point’s two coordinates, passes them through one or two layers of tanh neurons, and outputs the probability that the point is gold. Press play and watch the boundary bend to fit. Then turn attacker: click the field to plant gold points where blue belongs.
- Step
- 0
- Loss
- -
- Accuracy
- -
- Poisoned
- none
- Dials
- -
In real modelsThe same loop with bigger numbers: billions of dials instead of dozens, a fresh batch of examples at every step instead of every point at once, and optimisers such as Adam that adapt each dial’s step size. A frontier model trains for weeks to months on thousands of accelerators.
DefenceThat was data poisoning. A few mislabelled points teach the network an exception while its accuracy on clean data barely moves, which is what makes poisoning hard to spot. A backdoor adds a trigger: Gu and colleagues (2017) made a small sticker flip an image classifier while its clean accuracy held. Guard the training data: know where every example came from, look for outliers and duplicates, and test the model on inputs chosen to trigger hidden behaviour.