Structured Output (JSON)
Gen AI · Prompt Engineering
Theory
Structured output means asking a model to reply in a fixed, machine-readable shape, usually JSON with named fields, instead of free text, so your code can parse the answer and load it into a table. A typical prompt describes the exact fields, their types and allowed values, for example sentiment as one of "positive", "negative" or "neutral" and issue as a short string or null, and gives one example of valid output.
Many model APIs also offer a JSON or schema mode that constrains the reply to match a schema you supply, which is more reliable than wording alone. Either way, always validate in code: parse the JSON, check required fields and allowed values, and retry or flag the row when validation fails. This pattern turns messy text such as reviews, emails or tickets into clean columns you can query with SQL.
Why we use it
Free-text answers are hard to use in pipelines: formats drift and parsing breaks. A fixed schema lets you treat the model like a function that returns typed fields you can store, join and aggregate.
When to use it
Use it whenever model output feeds code, a database or a dashboard: extraction, classification, tagging and tool arguments. For answers that a person reads directly, natural prose is usually fine.
How it works
- Define the fields, types and allowed values
- Put the schema and one valid example in the prompt
- Send each text record with low temperature
- Parse the reply as JSON in code
- Does it parse and validate? Yes: Load the fields into the table. No: Retry once, then flag for review.
- Query the new columns with SQL
Example
import json
ALLOWED = {"positive", "negative", "neutral"}
prompt = (
"Return only JSON with keys sentiment (positive|negative|neutral) "
"and issue (short string or null).\n"
"Review: Late, box damaged"
)
reply = call_llm(prompt, temperature=0) # pseudo-call
# reply -> '{"sentiment": "negative", "issue": "delivery"}'
data = json.loads(reply) # fails loudly if not JSON
assert data["sentiment"] in ALLOWED # validate before loading
print(data["issue"]) # deliveryKey points
- Describe every field, its type and allowed values in the prompt
- Use the API's JSON or schema mode when available
- Always parse and validate in code; never assume the reply is valid
- Use low temperature for extraction and classification
- Use null for missing values instead of letting the model invent them