How LLMs Work

Gen AI · Foundations of Generative AI

Theory

A Large Language Model (LLM) produces text one token at a time. It reads everything so far, computes a probability for every possible next token, picks one, appends it and repeats until it emits a stop token or reaches a length limit. Most LLMs use the Transformer architecture, whose self-attention step lets each token weigh how relevant every other token in the input is, however far apart they are.

Training happens in stages. Pretraining teaches general language and knowledge by predicting the next token across a huge text collection. Instruction tuning and alignment then teach the model to follow requests and act as a helpful assistant. Because each token is sampled from probabilities, the same prompt can give slightly different answers.

Why we use it

Knowing that an LLM predicts the next token explains both its strengths (fluent writing, following patterns) and its weaknesses (confident mistakes, sensitivity to wording). It helps you write better prompts and set realistic expectations.

When to use it

Keep this mental model whenever you design a prompt, choose settings like temperature or debug a strange answer. For exact maths or live lookups, pair the LLM with tools or retrieval instead of trusting what it memorised.

How it works

  1. Prompt text arrives
  2. Split the text into tokens
  3. Transformer layers apply self-attention across all tokens
  4. Score every possible next token with a probability
  5. Pick one token and append it to the text
  6. Stop token or length limit reached? Yes: Finish the response. No: Loop back and predict the next token.
  7. Response is shown to the user

Example

Input so far: "The capital of France is"
Model predicts next token: " Paris" (highest probability)
Appends it, then predicts the next token again: "."
Final output: "The capital of France is Paris."

Key points

Gen AI topics

Foundations of Generative AI

Prompt Engineering

Retrieval & Embeddings

Customizing Model Behavior

AI Agents & Tool Use

Multimodal & Applied AI

Ethics, Cost & Careers

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Generative AI for the Data Stack

LLMs, prompt engineering, RAG, embeddings, agents and the tools shaping modern AI products — explained practically. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Section {{ sec.num }}

{{ sec.label }}

Gen AI  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

💡

{{ current.tldr }}

Explained

{{ para }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
See It In Action
{{ current.example }}
Key Takeaways — click to check off
{{ pt.mark }} {{ pt.text }}

Share this topic