You ask Claude to write a date parser. It produces clean code using a library function that does not exist, never has, and is named exactly what that function should be named. Yesterday the same prompt produced a different, working implementation. Nothing about this is a malfunction. Both behaviors fall straight out of what a language model is, and once you hold the right model of the machine, you can predict them before they happen.
The machine, minus the math
A language model does one thing: given a sequence of tokens, it produces a probability distribution over what token comes next. Then it samples one, appends it, and repeats. That is generation, all of it. Everything an agent does, every file edit, every "I'll run the tests now", is a stream of next-token choices.
Three parts of that sentence carry all the operational weight.
Tokens, not words. The model reads and writes in tokens: chunks that correspond to words, subwords, or characters. For Claude, one token is roughly 3.5 English characters as of July 2026, per Anthropic's glossary. You mostly ignore tokens at the text level, but they become load-bearing the moment you care about context budgets, which is the entire subject of Lesson 1.3.
A distribution, not an answer. The model does not know the next token; it scores every possible next token and the decoding process picks from the high-probability region. The temperature parameter controls how adventurous that pick is: low temperature sticks close to the most probable token, high temperature explores. This is why generation is called sampling.
Sampling means non-determinism. Because the pick is probabilistic, the same prompt legitimately produces different outputs on different runs. And here is the detail that surprises even experienced engineers: Anthropic's docs are explicit that even with temperature set to 0, identical inputs may produce different outputs across API calls. Determinism is not on the menu. If your workflow assumes "same prompt, same code", your workflow is wrong, and the fix is never prompt-tweaking; it is verification that does not care which of the plausible outputs you got.
The model's weights come from pretraining on a huge text corpus, then refinement into an assistant (Claude is trained with reinforcement learning from human feedback, per the glossary). Two operational facts fall out of the training story. First, the training cutoff: the weights encode the world up to a date, and nothing after. Second: the model has no lookup step. It is not consulting a reference when it names an API; it is emitting the statistically plausible next token given everything it has absorbed.
Confident when wrong is the default, not a bug
Put next-token prediction and no-lookup together and confident wrongness stops being mysterious. When you ask for code against a library the model knows well, plausible-next-token and correct-next-token coincide, and the output is good. When you ask about a library version released after cutoff, or an internal API it has never seen, the machinery does not halt. It keeps emitting the most plausible tokens, and the most plausible name for a function that should exist is a well-formed, idiomatic, wrong one. The fluency of the output carries zero information about its truth. Same generator, same confidence, either way.
The tone problem compounds it. Assistant training rewards helpful, decisive answers, so uncertainty does not read in the prose. You will not reliably get "I am not sure this method exists." You will get the method, used correctly, in code that compiles. The tell is never in the text; it is in the check you run afterward. This single fact justifies about a third of this course.
Attention is the third piece. The transformer architecture lets every token attend to every other token in context, which is n squared pairwise relationships for n tokens. Anthropic's engineering guidance draws the practical conclusion: models have a finite attention budget, and every token added depletes it. Where you put information matters, and how much you put in matters more. That preview becomes the whole of Lesson 1.3.
In practice: predictions you can now make
The model of the machine converts directly into operational predictions:
"It wrote broken code, so the model is bad at this."
"It worked yesterday, so something is broken today."
"It sounded very confident, so it is probably right."
"It knows the current version of my framework. It knows everything."
"It sampled a plausible implementation. Plausible and correct overlap less in unfamiliar territory. What check catches the gap?"
"Two draws from a distribution differed. Expected. My verification, not my prompt, absorbs that variance."
"Confidence is a register of the prose, produced the same way for right and wrong answers. It is not evidence."
"Its knowledge stops at training cutoff. For anything that ships releases, I make it read current docs instead of remembering."
That last row is why this course's own build rules require fetching live documentation before stating tool facts, and why Claude Code gives the model tools for reading files and fetching URLs at all: retrieved text in context beats remembered text in weights, every time recency matters.
Lablab-1-1Watch the distribution
Goal: Observe non-determinism directly by running the identical prompt in two fresh sessions and diffing the results.
Prereqs: the agentic-practice repo from Lesson 0.2.
- In the practice repo, start a fresh session with
claudeand paste exactly this prompt:
Write a single file, parse-duration.ts, exporting one function parseDuration(input: string): number that parses strings like "1h30m", "45s", "2d" into milliseconds. No dependencies. Include your reasoning for the design as comments.
- Approve the write, then run
git add -A && git commit -m "run 1". - Exit the session. Remove the file and commit:
git rm parse-duration.ts && git commit -m "reset". Start a completely fresh session and paste the identical prompt. - Commit run 2, then diff the two implementations:
git diff HEAD~2 HEAD -- parse-duration.ts. - Note what varied: parsing strategy, error handling, edge cases covered, API shape. Then note what a test suite for this function would have to look like to not care about any of that variance.
Verify
- You have two committed implementations produced from a byte-identical prompt.
- You can name at least two substantive differences between them (strategy, edge-case handling, or API shape, not formatting).
- You can state which differences a behavioral test would tolerate and which it would catch.
>Troubleshooting
- The two runs came out nearly identical: not a refutation, just two nearby draws. Run a third, or widen the task ("add support for compound strings like 1h30m45s") and watch strategies diverge.
- Claude asked clarifying questions in one run and wrote code immediately in the other: that is the non-determinism, demonstrated at the level of behavior instead of code. Count it and move on.
Where it breaks
The mental model has its own failure mode: treating the model as dumb autocomplete and over-steering it. Operators who internalize "it just predicts tokens" sometimes conclude they must specify every line, which throws away the machine's actual strength: it has absorbed more code than any human has read, and given good context it generalizes startlingly well. The model is not a junior you must micromanage and not an oracle you must trust. It is an extremely well-read generator with no ground truth of its own. Give it context and constraints; take from it drafts and hypotheses; never take from it facts you have not checked.
The other break: assuming the cutoff only matters for obscure things. It bites hardest on the tools you use most, because popular tools ship fast. A model whose weights predate a framework's latest major version will confidently scaffold the old API in your new project. The countermeasure is always retrieval: current docs in context outrank stale weights.
Knowledge check
Knowledge check
Sources
- Claude Platform glossary (tokens, temperature, non-determinism, RLHF, context window): https://docs.claude.com/en/docs/about-claude/glossary (fetched July 2026)
- Effective context engineering for AI agents (attention budget, pairwise attention): https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents (fetched July 2026)
- How Claude Code works: https://code.claude.com/docs/en/how-claude-code-works (fetched July 2026)