Evaluating LLM Apps
Gen AI · Customizing Model Behavior
Theory
Evaluation means measuring how well your LLM feature works on a fixed set of test cases, so you can tell whether a prompt, model or retrieval change made things better or worse. Start with a test set of realistic inputs and expected outputs, including edge cases and past failures, often 20 to 100 cases to begin with. Then score each output.
Scoring methods depend on the task. Exact match or rules work for labels, numbers and JSON fields. For free text, use a rubric scored by people or by a second model acting as a judge ("Is the answer supported by the context? yes or no"), and spot-check the judge against human ratings. For RAG, evaluate retrieval separately: did the right chunk appear in the top-k? Track quality alongside latency and cost, rerun the suite on every change, and add real production failures to the test set over time.
Why we use it
LLM outputs vary and a change that fixes one case can break others. A repeatable test set turns 'it looks better' into a measured pass rate, which is how you ship changes with confidence.
When to use it
Build the test set before you tune prompts, and rerun it whenever you change the prompt, model, chunking or retrieval settings. Keep monitoring in production, because real user inputs drift from your tests.
How it works
- Collect realistic test inputs with expected outputs
- Run the current version of the app on every case
- Score each output: exact match, rules or rubric judge
- Compute pass rate, plus latency and cost
- Is the new version better? Yes: Ship it; save scores as the baseline. No: Keep the old one; study failures.
- Add new production failures to the test set
Example
test_cases = [
{"q": "Which city hosts the Jaipur office?", "expected": "Jaipur"},
{"q": "Is the refund window 30 days?", "expected": "Yes"},
# ... 50 more realistic cases
]
passed = 0
for case in test_cases:
answer = my_llm_app(case["q"]) # your app, pseudo-call
if answer.strip() == case["expected"]:
passed += 1
print(f"pass rate: {passed / len(test_cases):.0%}")
# Compare this number before and after every prompt or model change.Key points
- Build a fixed test set with edge cases and past failures
- Use exact match or rules where possible; rubrics for free text
- Check a model-as-judge against human ratings before trusting it
- Evaluate retrieval separately from generation in RAG apps
- Rerun the suite on every change and track cost and latency too