Context Window

Gen AI · Foundations of Generative AI

Theory

The context window is the maximum number of tokens a model can consider in one request, counting everything: the system prompt, conversation history, pasted documents, tool results and the answer it generates. It works like short-term working memory. Anything outside the window is invisible, and a model does not remember earlier chats unless the app sends that history again.

A bigger window lets you include whole reports or long code files, but more is not automatically better. Models can miss details buried in the middle of a very long prompt, and every extra token adds cost and latency. If a request exceeds the window it is rejected or older content is cut, so apps trim history, summarise, or retrieve only the relevant parts.

Why we use it

It is a hard budget for every request. Knowing it tells you how much history, how many documents and how long an answer you can fit, and explains why a chatbot suddenly 'forgets' early messages.

When to use it

Plan around it when building chat history, RAG prompts or document summarisation. If your data is far larger than the window, retrieve only the relevant chunks or summarise in stages instead of pasting everything in.

How it works

  1. Gather system prompt, history, documents and question
  2. Count the tokens in every input piece
  3. Reserve room for the expected answer length
  4. Does input + answer reserve fit? Yes: Send the request. No: Trim history or retrieve fewer chunks.
  5. Model reads only what is inside the window
  6. Answer is generated within the remaining budget

Example

Model context window: 128,000 tokens

Your request:
  - System prompt: 200 tokens
  - Uploaded PDF: 40,000 tokens
  - Conversation history: 2,000 tokens
  - Your question: 50 tokens
  Total input: 42,250 tokens — well within the 128k limit, leaving room for a long generated answer.

Key points

Gen AI topics

Foundations of Generative AI

Prompt Engineering

Retrieval & Embeddings

Customizing Model Behavior

AI Agents & Tool Use

Multimodal & Applied AI

Ethics, Cost & Careers

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Generative AI for the Data Stack

LLMs, prompt engineering, RAG, embeddings, agents and the tools shaping modern AI products — explained practically. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Section {{ sec.num }}

{{ sec.label }}

Gen AI  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

💡

{{ current.tldr }}

Explained

{{ para }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
See It In Action
{{ current.example }}
Key Takeaways — click to check off
{{ pt.mark }} {{ pt.text }}

Share this topic