Parameters & Model Size
Gen AI · Foundations of Generative AI
Theory
Parameters are the numeric weights inside a neural network that are adjusted during training. Together they encode what the model has learned: grammar, facts, coding patterns and style. More parameters give a model more capacity, so size is a rough guide to capability, but it is far from the whole story.
Small models can run on a laptop or phone and answer quickly at low cost; very large models need data-centre hardware. Training data quality, training method and fine-tuning matter as much as size, and a small model tuned for one narrow task can beat a much larger general model on that task. In practice, teams pick the smallest model that passes their own quality tests.
Why we use it
Model size drives speed, cost, hardware needs and quality. Understanding the trade-off stops you paying for a huge model when a small one does the job, or under-sizing for a hard reasoning task.
When to use it
Weigh size when choosing a model for a product: prove the task with a capable model first, then test smaller ones. Prefer small or local models for privacy, offline use or high volume, and larger ones for complex reasoning.
How it works
- Define the task and the quality bar
- Build a small test set of real examples
- Try a large, capable model first
- Does a smaller model also pass? Yes: Use it: faster and cheaper. No: Keep the larger one or fine-tune.
- Monitor quality and cost in production
- Re-test when requirements change
Example
GPT-2 (2019): ~1.5 billion parameters — could barely hold a coherent conversation.
Modern flagship LLMs: 100B+ parameters — write code, reason step-by-step, pass professional exams.
A small 7B model fine-tuned only on medical text can beat a 175B general model on medical Q&A.Key points
- Parameters are learned weights; size is a rough, imperfect proxy for capability
- Bigger models cost more and respond more slowly
- Training data quality matters as much as parameter count
- Smaller fine-tuned models can beat larger general models on narrow tasks
- Pick the smallest model that passes your own tests