Structured Output (JSON/Schema)

Latest update: 26/05/03

Definition

Structured output is when you instruct an AI to return its response in a specific, predictable format – like JSON, XML, or a defined schema – rather than natural prose, so the result can be reliably read and processed by other software.

What Is Structured Output?

AI models naturally produce text. Flowing, conversational, human-readable text. That’s great for reading – and useless for any downstream system that needs to parse a specific field, slot data into a database, or trigger logic based on a particular value.

Structured output solves that. Instead of asking a question and getting a paragraph, you tell the AI exactly what shape to put the answer in: a JSON object, a CSV row, a list in a defined format, an XML block. The output becomes machine-readable – predictable enough for software to consume without ambiguity.

It’s the difference between asking a research assistant “what do we know about this company?” and asking them to fill in a form with specific fields. Same knowledge, completely different usability.

💡 How Does It Work?

You specify the structure you want as part of the prompt – either by describing it clearly in natural language, providing a JSON schema, or both. The model generates a response shaped to match that specification.

Modern APIs from Anthropic and OpenAI also support structured output at the API level – you define the exact schema you want returned, and the model is constrained to produce output that conforms to it. This is more reliable than instruction alone because it removes the generation ambiguity entirely.

Think of it like a form versus a blank page. A blank page can hold anything. A form has defined fields. Structured output gives the AI a form to fill in – which means whatever it writes fits the expected shape, and you don’t have to parse free text to get the values out.

Why It Matters for Your Prompts

The moment you want to use AI output programmatically – store it in a database, feed it into another system, trigger logic based on its values – you need structured output. Parsing natural language reliably is hard. Parsing JSON is trivial.

This also affects how you write prompts. Specifying the exact structure you want upfront – the field names, the data types, what to put when a value isn’t available – removes a lot of ambiguity from the generation process. A prompt that says “return a JSON object with fields: name (string), score (integer 1–10), reason (string, max 50 words)” leaves the model much less room to produce something unparseable than “give me a score and explain why.”

Common failure modes when people get this wrong: the model wraps JSON in markdown code fences you didn’t ask for, it invents field names that weren’t in the schema, it returns a mix of JSON and explanatory prose, or it uses inconsistent formatting across runs. Specifying output format precisely and using API-level structured output when available are the fastest fixes.

🌐 Real-World Example

A hiring platform uses AI to screen resumes. They need the AI to evaluate each resume and return: a fit score, a list of matched skills, the top strength, and a flag for any disqualifying factors.

Without structured output: The AI returns a thoughtful two-paragraph assessment per candidate. Useful for reading – impossible to sort, filter, or feed into their ranking system automatically.

With structured output: They prompt the AI to return a JSON object with defined fields for each evaluation dimension. Every resume produces the same structure. Their backend reads the score, populates the dashboard, ranks candidates, and routes the top 20% automatically – no manual parsing, no ambiguity, no failed extraction.

Same AI, same evaluation quality. The structure is what made it deployable.

Related Terms

Prompt Engineering – Specifying output format is one of the most practical prompt engineering skills; structured output is its most rigorous application.
Agentic Workflows – Agentic workflows almost always depend on structured output to pass data reliably between steps.
Prompt Template – Structured output prompts are almost always templates – the schema is fixed, and only the input content changes.
Inference – API-level structured output works by constraining the model’s token sampling during inference to only produce tokens that conform to the schema.
Few-Shot Prompting – Including one or two examples of correctly formatted output in the prompt is one of the most effective ways to improve structured output consistency without using API-level schema enforcement

Encyclopedia Fundamental Prompting Techniques Architecture & Technical

Frequently Asked Questions

Do I need to use JSON specifically, or can I use other formats?

JSON is the most common choice because it’s widely supported in software and easy for models to generate. But structured output can be any consistent format: XML, CSV, YAML, a numbered list with defined fields, even a simple delimiter-separated format. The key is that the format is predictable and parseable. JSON is a sensible default; use whatever your downstream system expects.

Why does the AI sometimes add extra text around the JSON I asked for?

Models are trained to be conversational by default, and that training can push them toward adding preamble (“Here is the JSON you requested:”) or closing text even when you’ve asked for raw output. The fixes: explicitly say “return only the JSON, no other text,” use system prompts to set the behavior persistently, or use API-level structured output which hard-constrains the format. For production systems, API-level enforcement is more reliable than instruction alone.

What’s the difference between asking for JSON in a prompt vs. using API-level structured output?

Prompt-level instruction asks the model to try to format correctly – it usually works but can fail on edge cases. API-level structured output (available through the Anthropic and OpenAI APIs) enforces the schema at the generation level – the model physically cannot produce output that doesn’t conform to your schema. For production use cases where a malformed response breaks downstream systems, API-level enforcement is worth the extra setup.

Can structured output handle nested data and arrays?

Yes. JSON supports nested objects, arrays, optional fields, and most data structures you’d need. More complex schemas do increase the chance of generation errors, so it’s worth testing thoroughly with examples that cover edge cases – particularly empty arrays, null values, and long nested structures. For very complex schemas, breaking the task into multiple simpler structured calls can improve reliability.

References

Anthropic – Structured Outputs – Documentation on using JSON schemas with Claude’s API for reliable structured generation.
OpenAI – Structured Outputs – OpenAI’s guide to schema-constrained generation with worked examples and best practices.