You parse the response. It is malformed JSON. There is a trailing comma after the last array element. An unclosed brace three levels deep. Your extraction pipeline just threw an exception at 2 AM, and the on-call engineer is staring at a Slack alert that says SyntaxError: Unexpected token ',' in JSON at position 4,217. Sound familiar?
Getting AI models to produce structured, machine-parseable output is one of the most practically important problems in applied AI engineering. Every data pipeline, every API integration, every automated workflow that touches a language model needs structured output to function. And yet, getting it reliably remains harder than it should be.
This guide covers every technique available in 2026 for getting clean structured output from ChatGPT, Claude, and Gemini -- from native model features to prompt-based strategies to production-grade fallback patterns. If you have been fighting malformed JSON, read on.
Info
Who this guide is for: Developers building applications that consume AI output programmatically. If you are prompting through a chat interface for personal use, most of these techniques still apply, but the production patterns in the second half are specifically for API-level integrations.
Why Structured Output Is Hard
Language models do not generate data structures. They generate text, one token at a time, left to right. A model producing JSON is not constructing an object and serializing it. It is predicting the next character based on everything that came before.
This fundamental mismatch between token-by-token generation and the rigid syntax requirements of formats like JSON creates several predictable failure modes.
Unclosed Brackets and Braces
The model opens a nested object, generates several fields, then loses track of the nesting depth. The result: a response that ends abruptly or closes with the wrong number of braces. This happens more frequently with deeply nested schemas and long outputs that push the model's effective attention span.
Extra Commentary
You ask for JSON. The model gives you JSON -- preceded by "Here is the JSON you requested:" and followed by "Let me know if you need any changes!" Your parser chokes on the English sentences wrapping the actual data.
Markdown Code Fences
A close cousin of extra commentary. The model wraps the JSON in triple backticks with a json language tag. Useful for human readers. Breaks every JSON.parse() call.
Inconsistent Schemas
You define a schema with firstName and lastName. The model returns first_name and last_name on one request, name (as a single combined field) on the next, and FirstName with Pascal case on the third. The structure is technically valid JSON each time. It is also useless to your application.
Type Mismatches
Your schema expects "age": 34 as a number. The model returns "age": "34" as a string. Or "active": "true" instead of "active": true. These mismatches are silent killers -- the JSON parses fine but your downstream code breaks when it tries to do arithmetic on a string.
Array vs Object Confusion
You ask for a list of items. Sometimes you get an array [{...}, {...}]. Sometimes you get a wrapper object {"items": [{...}, {...}]}. Sometimes, when there is only one result, you get a bare object {...} instead of a single-element array [{...}].
Understanding these failure modes is the first step. The rest of this guide is about eliminating them.
Model-Native Structured Output Features
Every major model provider now offers built-in features for structured output. These should be your first choice whenever available, because they enforce structure at the decoding level rather than relying on the model to voluntarily comply.
ChatGPT / GPT-4: JSON Mode and Structured Outputs
OpenAI provides three progressively stricter mechanisms.
JSON Mode is the simplest. Set response_format: { type: "json_object" } in your API call, and the model is constrained to produce valid JSON. You still need to describe the desired schema in your prompt -- JSON mode only guarantees syntactic validity, not schema compliance.
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": "Extract contact info. Return JSON with fields: name (string), email (string), phone (string or null)."
},
{
"role": "user",
"content": "Reach out to Jane Park at jane.park@acme.io or call 555-0142."
}
]
)Structured Outputs with json_schema go further. You provide a full JSON Schema definition, and the model's output is guaranteed to conform to it. Field names, types, required fields, enums -- all enforced.
response = client.chat.completions.create(
model="gpt-4o",
response_format={
"type": "json_schema",
"json_schema": {
"name": "contact_info",
"strict": True,
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"phone": {"type": ["string", "null"]}
},
"required": ["name", "email", "phone"],
"additionalProperties": False
}
}
},
messages=[
{"role": "user", "content": "Reach out to Jane Park at jane.park@acme.io or call 555-0142."}
]
)Function calling is the third mechanism. You define functions with parameter schemas, and the model generates arguments that conform to those schemas. This is technically designed for tool use, but many developers use it purely as a structured output mechanism.
Tip
When using OpenAI's Structured Outputs with strict: true, every field in your schema must be listed in required, and additionalProperties must be false. The model cannot produce fields you did not define.
Claude: Tool Use and XML-Style Structure
Anthropic's Claude does not have a dedicated JSON mode flag in the same way as OpenAI. Instead, Claude provides two strong mechanisms for structured output.
Tool use is the primary approach. You define tools with input schemas, and Claude generates structured arguments that conform to those schemas. Even when you are not actually calling tools, you can define a "dummy" tool whose sole purpose is to shape the output.
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=[
{
"name": "extract_contact",
"description": "Extract contact information from text",
"input_schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"phone": {"type": ["string", "null"]}
},
"required": ["name", "email", "phone"]
}
}
],
messages=[
{"role": "user", "content": "Reach out to Jane Park at jane.park@acme.io or call 555-0142."}
]
)XML-style prompting is Claude's other strength. Claude follows XML tag instructions with high fidelity. Wrapping output instructions in XML tags produces remarkably consistent structure, even without tool use.
Extract the contact information from the following text.
<text>Reach out to Jane Park at jane.park@acme.io or call 555-0142.</text>
Return the result inside <result> tags as valid JSON with this exact schema:
{"name": string, "email": string, "phone": string | null}
<result>By opening the <result> tag yourself, you prime Claude to continue directly with the JSON content and close the tag when done. No preamble, no commentary.
Gemini: JSON Response Mode
Google's Gemini API provides a response_mime_type parameter. Setting it to application/json constrains the model to produce JSON output. Combined with a response_schema parameter, you get schema-level enforcement similar to OpenAI's Structured Outputs.
import google.generativeai as genai
model = genai.GenerativeModel("gemini-2.0-flash")
response = model.generate_content(
"Reach out to Jane Park at jane.park@acme.io or call 555-0142.",
generation_config=genai.GenerationConfig(
response_mime_type="application/json",
response_schema={
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"phone": {"type": "string"}
},
"required": ["name", "email", "phone"]
}
)
)When to Use Native Features vs Prompt-Based Approaches
Native features should be your default choice for API integrations. They provide guarantees that no prompt can match. But there are cases where prompt-based approaches are necessary or preferable:
- Chat interfaces where you do not control API parameters
- Multi-format outputs where you need JSON embedded within prose
- Legacy or fine-tuned models that do not support structured output features
- Cost optimization when tool use adds token overhead you want to avoid
- Complex conditional schemas that are difficult to express in JSON Schema
For everything else, use the native features. They exist because prompt-based approaches are unreliable at scale, and the providers know it.
Prompt-Based Structured Output Techniques
When native features are not available, you need to get structured output through prompt engineering alone. These techniques work across all models and interfaces, including the ChatGPT web UI, Claude.ai, and Gemini.
Schema-First Prompting
Show the model the exact schema you want, then ask it to fill in the data. This is the single most effective prompt-based technique.
Extract product information from the following review and return it as JSON matching this exact schema:
{
"product_name": string,
"rating": number (1-5),
"pros": string[],
"cons": string[],
"would_recommend": boolean,
"price_mentioned": number | null
}
Review: "The Keychron Q1 Pro is fantastic. Build quality is exceptional, the gasket mount typing feel is superb, and wireless via Bluetooth works flawlessly. Battery lasts about 3 weeks. My only complaints: the stock keycaps feel a bit cheap for the $200 price point, and the software is clunky. 4 out of 5 stars. Absolutely recommend."The schema acts as a contract. The model sees the exact field names, types, and constraints before it starts generating. This dramatically reduces field naming inconsistencies and type mismatches.
Few-Shot with Examples
Provide two or three complete input-output examples before your actual request. This is more verbose but more reliable, because the model can pattern-match against concrete examples rather than abstract schemas.
Extract product data as JSON. Here are examples:
Input: "Love my AirPods Max. Noise cancellation is incredible, but $549 is steep. 4 stars."
Output: {"product_name": "AirPods Max", "rating": 4, "pros": ["noise cancellation"], "cons": ["expensive"], "would_recommend": true, "price_mentioned": 549}
Input: "The Bose QC Ultra are decent. Sound is good, comfort is average, ANC is below AirPods. 3 stars for $429."
Output: {"product_name": "Bose QC Ultra", "rating": 3, "pros": ["good sound"], "cons": ["average comfort", "ANC below competitors"], "would_recommend": false, "price_mentioned": 429}
Now extract from this review:
Input: "The Keychron Q1 Pro is fantastic. Build quality is exceptional, gasket mount typing feel is superb, wireless works flawlessly. Stock keycaps feel cheap for $200. Software is clunky. 4 out of 5 stars. Absolutely recommend."
Output:Notice how the prompt ends with Output: and nothing else. This primes the model to continue the pattern immediately with JSON, no preamble.
For more on this technique, see our guide on few-shot prompting.
The Nuclear Option: "Output ONLY Valid JSON"
Sometimes brute force works. If the model keeps adding commentary around your JSON, be explicit.
Output ONLY valid JSON. No markdown code fences. No explanation before or after. No commentary. Just the raw JSON object, starting with { and ending with }.This instruction is surprisingly effective when combined with a schema. It is less effective on its own, because the model still needs to know what JSON to produce.
Delimiter-Based Extraction
If you need the model to produce both prose and structured data, use explicit delimiters so your code can extract the structured portion.
Analyze this text and provide:
1. A brief summary (2-3 sentences)
2. Structured data extraction
Format your response exactly like this:
---SUMMARY---
[your summary here]
---JSON---
[your JSON here]
---END---Your parsing code splits on the delimiters and extracts the JSON block. This is robust because even if the summary contains JSON-like text, your parser knows to look between ---JSON--- and ---END---.
XML Tag Wrapping (Claude-Specific)
Claude has a strong affinity for XML-style tags. Use them to create unambiguous boundaries around structured output.
Analyze the customer feedback below.
<feedback>
The onboarding was confusing. I couldn't find the settings page for 10 minutes. Once I found it, customization options were great. Support response was fast but the first answer didn't solve my issue.
</feedback>
Provide your analysis inside these XML tags:
<analysis>
<sentiment>[positive/negative/mixed]</sentiment>
<topics>
<topic name="..." sentiment="..." />
</topics>
<json_summary>
[valid JSON with fields: overall_sentiment, nps_estimate (1-10), top_issue, top_praise]
</json_summary>
</analysis>This technique combines the reliability of XML tag compliance with the flexibility of embedding JSON within a larger structured response.
Handling Common Failure Modes
Even with the best techniques, failures happen. Here is how to handle each one programmatically.
Stripping Markdown Code Fences
The most common failure. Easy to fix.
function extractJSON(response: string): unknown {
// Strip markdown code fences
let cleaned = response.trim();
// Handle wrapping
const fenceMatch = cleaned.match(/`(?:json)?\s*\n?([\s\S]*?)\n?\s*`/);
if (fenceMatch) {
cleaned = fenceMatch[1].trim();
}
return JSON.parse(cleaned);
}
__CODE_BLOCK_11__typescript
function extractJSONFromProse(response: string): unknown {
// Find the first { or [ and the last } or ]
const firstBrace = response.indexOf('{');
const firstBracket = response.indexOf('[');
const start = firstBrace === -1 ? firstBracket
: firstBracket === -1 ? firstBrace
: Math.min(firstBrace, firstBracket);
if (start === -1) {
throw new Error('No JSON structure found in response');
}
const isArray = response[start] === '[';
const closeChar = isArray ? ']' : '}';
const lastClose = response.lastIndexOf(closeChar);
if (lastClose === -1) {
throw new Error(No closing ${closeChar} found in response);
}
const jsonStr = response.slice(start, lastClose + 1);
return JSON.parse(jsonStr);
}
__CODE_BLOCK_12__typescript
import { z } from 'zod';
const ContactSchema = z.object({
name: z.string().min(1),
email: z.string().email(),
phone: z.string().nullable(),
});
type Contact = z.infer<typeof ContactSchema>;
function parseContact(response: string): Contact {
const raw = extractJSON(response);
return ContactSchema.parse(raw);
}
__CODE_BLOCK_13__typescript
const ProductSchema = z.object({
name: z.string(),
price: z.coerce.number(),
in_stock: z.coerce.boolean(),
rating: z.coerce.number().min(1).max(5),
});
__CODE_BLOCK_14__
Given the following TypeScript type, extract data from the text below and return valid JSON conforming to this type:
type CompanyProfile = {
name: string;
founded: number;
headquarters: {
city: string;
country: string;
};
revenue_millions: number | null;
public: boolean;
competitors: string[];
};
Text: "Stripe, founded in 2010 by the Collison brothers, is headquartered in San Francisco, USA. The privately held company processes hundreds of billions in payments annually. Key competitors include Adyen, Square, and PayPal."
__CODE_BLOCK_15__python
def extract_records_chunked(document: str, chunk_size: int = 2000) -> list[dict]:
chunks = split_into_chunks(document, chunk_size)
all_records: list[dict] = []
for chunk in chunks:
response = extract_from_chunk(chunk)
records = parse_and_validate(response)
all_records.extend(records)
return deduplicate(all_records)
__CODE_BLOCK_16__typescript
import { OpenAI } from 'openai';
async function streamStructuredOutput(prompt: string): Promise<unknown> {
const client = new OpenAI();
const stream = await client.chat.completions.create({
model: 'gpt-4o',
response_format: { type: 'json_object' },
messages: [{ role: 'user', content: prompt }],
stream: true,
});
let accumulated = '';
for await (const chunk of stream) {
const delta = chunk.choices[0]?.delta?.content ?? '';
accumulated += delta;
// Optionally parse partial JSON for progress indicators
// Full parse happens after stream completes
}
return JSON.parse(accumulated);
}
__CODE_BLOCK_17__typescript
async function extractWithRetry(
prompt: string,
schema: z.ZodSchema,
maxRetries: number = 2
): Promise<z.infer<typeof schema>> {
let lastError: string | null = null;
let lastResponse: string | null = null;
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const messages = [{ role: 'user' as const, content: prompt }];
if (lastError && lastResponse) {
messages.push({
role: 'user' as const,
content: Your previous response failed validation:\n\nResponse: ${lastResponse}\n\nError: ${lastError}\n\nPlease fix the JSON and return only the corrected version.,
});
}
const response = await callModel(messages);
lastResponse = response;
try {
const parsed = extractJSON(response);
return schema.parse(parsed);
} catch (err) {
lastError = err instanceof Error ? err.message : String(err);
}
}
throw new Error(Failed after ${maxRetries + 1} attempts. Last error: ${lastError});
}
__CODE_BLOCK_18__
Generate a CSV with these exact headers in this exact order:
name,email,role,department,start_date
Rules:
- Use comma delimiters, no spaces after commas
- Quote any field that contains commas
- Use ISO 8601 dates (YYYY-MM-DD)
- First row must be the header row
- No blank lines between rows
- No trailing newline
Data source: [your text here]
__CODE_BLOCK_19__
Format the comparison as a markdown table with these columns:
| Feature | Tool A | Tool B | Winner |
Rules:
- Align the separator row with dashes (---)
- Keep cell content concise (under 30 characters)
- Use "Tie" when features are equivalent
- Sort rows by importance (most important first)
__CODE_BLOCK_20__python
from openai import OpenAI
from pydantic import BaseModel
class ProductListing(BaseModel):
title: str
price_usd: float
currency: str
availability: str # "in_stock", "out_of_stock", "pre_order"
features: list[str]
rating: float | None
review_count: int | None
EXTRACTION_PROMPT = """Extract product listing data from the following text.
Return valid JSON matching this schema exactly:
{
"title": string,
"price_usd": number,
"currency": string (ISO 4217),
"availability": "in_stock" | "out_of_stock" | "pre_order",
"features": string[],
"rating": number | null (1-5 scale),
"review_count": integer | null
}
Output ONLY the JSON object. No markdown, no explanation.
Text:
{text}"""
def extract_product(text: str) -> ProductListing:
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
response_format={"type": "json_object"},
messages=[
{"role": "user", "content": EXTRACTION_PROMPT.format(text=text)}
]
)
raw = response.choices[0].message.content
return ProductListing.model_validate_json(raw)
__CODE_BLOCK_21__typescript
import Anthropic from '@anthropic-ai/sdk';
import { z } from 'zod';
const CategoryResult = z.object({
primary_category: z.enum([
'technical', 'business', 'opinion', 'tutorial', 'news', 'review'
]),
confidence: z.number().min(0).max(1),
secondary_categories: z.array(z.string()),
topics: z.array(z.string()).max(5),
reading_level: z.enum(['beginner', 'intermediate', 'advanced']),
estimated_read_minutes: z.number().int().positive(),
});
async function categorizeContent(text: string) {
const client = new Anthropic();
const response = await client.messages.create({
model: 'claude-sonnet-4-20250514',
max_tokens: 512,
tools: [
{
name: 'categorize',
description: 'Categorize a piece of content',
input_schema: {
type: 'object',
properties: {
primary_category: {
type: 'string',
enum: ['technical', 'business', 'opinion', 'tutorial', 'news', 'review'],
},
confidence: { type: 'number', minimum: 0, maximum: 1 },
secondary_categories: {
type: 'array',
items: { type: 'string' },
},
topics: {
type: 'array',
items: { type: 'string' },
maxItems: 5,
},
reading_level: {
type: 'string',
enum: ['beginner', 'intermediate', 'advanced'],
},
estimated_read_minutes: { type: 'integer', minimum: 1 },
},
required: [
'primary_category', 'confidence', 'secondary_categories',
'topics', 'reading_level', 'estimated_read_minutes',
],
},
},
],
tool_choice: { type: 'tool', name: 'categorize' },
messages: [{ role: 'user', content: Categorize this content:\n\n${text} }],
});
const toolBlock = response.content.find((b) => b.type === 'tool_use');
if (!toolBlock || toolBlock.type !== 'tool_use') {
throw new Error('No tool use block in response');
}
return CategoryResult.parse(toolBlock.input);
}
__CODE_BLOCK_22__python
MULTI_ENTITY_PROMPT = """Extract all people, organizations, and relationships from the text below.
Return valid JSON with this schema:
{
"people": [
{
"name": string,
"title": string | null,
"organization": string | null
}
],
"organizations": [
{
"name": string,
"type": "company" | "government" | "nonprofit" | "other",
"industry": string | null
}
],
"relationships": [
{
"entity_1": string (name),
"entity_2": string (name),
"relationship": string (e.g., "CEO of", "acquired", "partner of")
}
]
}
Rules:
- Include every person and organization mentioned, even if only once
- Use the most complete version of each name (e.g., "Jane Park" not "Park")
- Relationships must reference names exactly as they appear in the people or organizations arrays
- If a field is unknown, use null rather than guessing
- Output ONLY the JSON. No commentary.
Text:
{text}"""
__CODE_BLOCK_23__
Generate a quarterly performance report. Structure the output as JSON with this format:
{
"report_title": string,
"period": string,
"generated_at": string (ISO 8601),
"executive_summary": string (2-3 sentences),
"sections": [
{
"title": string,
"body": string (markdown-formatted prose),
"metrics": [
{
"name": string,
"value": number,
"unit": string,
"trend": "up" | "down" | "flat",
"change_percent": number
}
],
"action_items": string[]
}
],
"overall_status": "on_track" | "at_risk" | "behind"
}
Include sections for: Revenue, User Growth, Product Development, Customer Satisfaction.
Use this data: [your data here]
`
This pattern produces output that your application can render as a formatted report (using the body fields) while also extracting machine-readable metrics and action items for dashboards and alerts.
## Putting It All Together
Structured output is the bridge between AI and software systems. Without it, AI is a tool for humans typing in chat windows. With it, AI becomes a component in automated pipelines, data processing workflows, and production applications.
Here is the decision tree for choosing your approach:
<div class="bg-slate-50 dark:bg-slate-950/30 rounded-xl p-6 my-6"><div class="space-y-2"><div class="flex items-start gap-4 relative"><div class="flex flex-col items-center"><div class="w-8 h-8 rounded-full bg-slate-700 dark:bg-slate-300 text-white dark:text-slate-900 flex items-center justify-center text-sm font-bold flex-shrink-0">1</div><div class="w-0.5 h-6 bg-slate-300 dark:bg-slate-600 mt-2"></div></div><div class="pb-6"><p class="text-sm text-gray-700 dark:text-gray-300 font-medium">**Can you use the API?** Use model-native structured output features (JSON mode, Structured Outputs, tool use). This is always the most reliable option.</p></div></div><div class="flex items-start gap-4 relative"><div class="flex flex-col items-center"><div class="w-8 h-8 rounded-full bg-slate-700 dark:bg-slate-300 text-white dark:text-slate-900 flex items-center justify-center text-sm font-bold flex-shrink-0">2</div><div class="w-0.5 h-6 bg-slate-300 dark:bg-slate-600 mt-2"></div></div><div class="pb-6"><p class="text-sm text-gray-700 dark:text-gray-300 font-medium">**Do you need guaranteed schema compliance?** Use OpenAI's json_schema with strict: true, Claude's tool use with tool_choice, or Gemini's response_schema. These enforce structure at the decoding level.</p></div></div><div class="flex items-start gap-4 relative"><div class="flex flex-col items-center"><div class="w-8 h-8 rounded-full bg-slate-700 dark:bg-slate-300 text-white dark:text-slate-900 flex items-center justify-center text-sm font-bold flex-shrink-0">3</div><div class="w-0.5 h-6 bg-slate-300 dark:bg-slate-600 mt-2"></div></div><div class="pb-6"><p class="text-sm text-gray-700 dark:text-gray-300 font-medium">**Are you prompting through a chat interface?** Use schema-first prompting with few-shot examples. End your prompt with Output:` or an opening brace to prime the model.Is the output large or complex? Use two-pass generation or chunked extraction. Break the problem down before asking for structure.
Are you in production? Add validation (Zod, Pydantic), extraction utilities (strip code fences, find JSON boundaries), and retry logic with error feedback.
The techniques compound. A production pipeline might use native Structured Outputs as the primary path, prompt-based techniques as a fallback for edge cases, extraction utilities to handle occasional formatting issues, schema validation to catch semantic errors, and retry logic as the final safety net.
Every layer reduces your failure rate. Native features get you to around 99% structural compliance. Validation catches the remaining issues. Retry with error feedback resolves almost all of those. The result is a pipeline that produces clean, parseable, schema-compliant structured data from AI models with the reliability your production systems demand.
Build your structured output prompts faster with our prompt generator -- it handles schema definition, few-shot examples, and output constraints automatically. And if you want to deepen your prompt engineering skills beyond structured output, start with the fundamentals and work your way up to techniques like function calling and tool use.
Structured output is not a nice-to-have. It is the baseline requirement for every serious AI integration. Now you have the tools to get it right.