How LLMs Work
Gen AI · Foundations of Generative AI
Theory
A Large Language Model (LLM) produces text one token at a time. It reads everything so far, computes a probability for every possible next token, picks one, appends it and repeats until it emits a stop token or reaches a length limit. Most LLMs use the Transformer architecture, whose self-attention step lets each token weigh how relevant every other token in the input is, however far apart they are.
Training happens in stages. Pretraining teaches general language and knowledge by predicting the next token across a huge text collection. Instruction tuning and alignment then teach the model to follow requests and act as a helpful assistant. Because each token is sampled from probabilities, the same prompt can give slightly different answers.
Why we use it
Knowing that an LLM predicts the next token explains both its strengths (fluent writing, following patterns) and its weaknesses (confident mistakes, sensitivity to wording). It helps you write better prompts and set realistic expectations.
When to use it
Keep this mental model whenever you design a prompt, choose settings like temperature or debug a strange answer. For exact maths or live lookups, pair the LLM with tools or retrieval instead of trusting what it memorised.
How it works
- Prompt text arrives
- Split the text into tokens
- Transformer layers apply self-attention across all tokens
- Score every possible next token with a probability
- Pick one token and append it to the text
- Stop token or length limit reached? Yes: Finish the response. No: Loop back and predict the next token.
- Response is shown to the user
Example
Input so far: "The capital of France is"
Model predicts next token: " Paris" (highest probability)
Appends it, then predicts the next token again: "."
Final output: "The capital of France is Paris."Key points
- Generates one token at a time, feeding each new token back in as input
- Self-attention relates every token to every other token in the context
- Pretraining learns language; instruction tuning and alignment make it an assistant
- Sampling from probabilities means the same prompt can vary between runs
- No live lookup by default: knowledge comes from training data or your prompt