Understanding models · Chapter 3 of 8
Choosing the next token
A model scores every possible next token, turns the scores into chances and picks one at random; temperature, top-k and top-p change the pick, and base, instruct and reasoning models are trained to write differently.
Goals
- See the 20 likeliest next tokens for a prompt.To do
- Move temperature and top-p, and pick again.To do
- Ask a base model and an instruct model the same question.To do
- Pass the check.To do
Step 1 of 4: The read
One token at a time
In the last chapter, text became tokens. A model writes the same way: it reads all the text so far, picks one next token, adds it to the text and starts again. A reply of 100 tokens takes 100 of these steps.
Scores become chances
At each step the model gives every token it knows a score, called a logit. SmolLM2-360M, the small open model in this chapter, knows 49,152 tokens, so it gives 49,152 scores. The scores are then turned into chances, also called probabilities, that add up to 100%. A higher score means a larger chance: one point more means about 2.7 times the chance.
A random pick, with settings
Then one token is picked at random, by those chances, like a draw from a bag where likely tokens have more tickets. So the same prompt can get different replies. Three settings change the draw. Temperature divides every score before it becomes a chance: below 1 the likely tokens get even likelier, above 1 the chances even out. Top-k keeps only the k likeliest tokens. Top-p keeps the likeliest tokens until their chances add up to p, so a top-p of 0.9 drops the long tail of unlikely ones. Whatever is kept is scaled back up to 100%.
Base and instruct models
A base model has only learned to continue text, so a question may get more questions. An instruct model was trained further on conversations, written in a chat format that marks who is speaking. It answers, then writes an end token, a special token that means it is done.
Reasoning models
A reasoning model writes its working first, in the same token-by-token way, and then the answer. As of 2026-11-03, closed models such as Claude Opus 5.5 and GPT-6.1 Sol make reasoning a setting, and open ones such as DeepSeek-R1 and Olmo 3 7B Think ship as separate reasoning models.
In the visual
The numbers in the visual were recorded from SmolLM2-360M and its instruct version, both from Hugging Face. If your browser has WebGPU, a way for web pages to use the graphics card, you can also download the instruct model and run it live.
Step 2 of 4: Try it: Next-token sampler
Next-token sampler
Choose a prompt and read the chances, then shape them and pick, one token at a time. Last, ask the base and the instruct model the same question.
Mode:RealReal, recorded: every score, chance and answer comes from running SmolLM2-360M and SmolLM2-360M-Instruct on a computer on 2026-11-03, with Hugging Face's Transformers.js. The picks, temperature, top-k and top-p are computed in this page from those numbers, and the text you step through is the base model's own recorded run. The reasoning models come from the snapshot of 2026-11-03. The live run is real too, after an opt-in 204 MB download.
Step 3 of 4: Check
Check
Prompt: The capital of France is
Model: SmolLM2-360M (base)
Next token Chance
Paris 20.3%
the 19.6%
located 10.5%
situated 4.7%
called 3.7%
… 15 more, then all the rest
Temperature: 1
Top-k: off
Top-p: 0.5Step 4 of 4: What you can do now
What you can do now
- Explain how a model turns scores into chances and picks the next token.
- Say what temperature, top-k and top-p do to the chances.
- Tell a base model's output from an instruct model's, and say what a reasoning model adds.