Prompt Engineering for Data Extraction and Structured Outputs
Most teams treat extraction as a formatting problem. It is actually a control problem. This guide covers JSON mode, schema constraints, validation loops, and the failure modes that make extraction brittle in production.
Why extraction fails even when the output looks right
LLMs are fluent, not precise. When you ask for structured data, the model can produce the right shape while inventing fields, dropping optional keys, or returning nested objects you never asked for. These errors pass visual inspection and break downstream parsers.
The core issue is ambiguity. Free-form prompts leave the model to guess the schema, the nullability of each field, and unit conventions. In production, that guess changes across models, temperatures, and even small wording differences.
The fix is not a smarter model. It is a tighter contract between your prompt and the expected output.
Use JSON mode with a strict schema
If your provider supports JSON mode or constrained decoding, use it. Do not ask the model to "return JSON" in natural language. Ask for a specific schema, with required fields, types, and nullability spelled out.
A good extraction prompt has three parts: the task, the schema, and the constraints. The schema should be small enough that the model can hold it in working memory. If you need many fields, split the extraction into two passes rather than one giant schema.
Notice the prompt does not say "make sure the JSON is valid." It says what valid means: specific enum values, required arrays, and null semantics. That removes interpretation.
Constrain fields before you validate them
Validation catches errors. Constraint prevents them. The most reliable extraction pipelines constrain first, then validate. Constraining means telling the model exactly what each field can and cannot contain.
Useful constraints to encode directly
- Enumeration: "status must be one of: pending, paid, shipped."
- Format: "date must be ISO 8601 (YYYY-MM-DD)."
- Range: "quantity must be an integer greater than 0."
- Nullability: "if the address is missing, set address to null; do not omit the key."
When you embed constraints directly into the prompt, the model spends its budget on following rules rather than guessing format. The result is fewer invalid outputs and less post-processing.
Build a validation loop, not a one-shot prompt
Even with strict prompting, edge cases appear. A receipt may have no items. A date field may be written as "Aug 5" instead of "2026-08-05." A currency symbol may be missing. Production extraction needs a loop, not a single prompt.
The loop validates structure, checks business rules, and gives the model a chance to correct itself. Keep the correction prompt specific—"field 'total' must be a number, not a string"—rather than generic feedback.
For a broader automation strategy around this kind of data flow, see building AI data analysis workflows.
Handle nested and array fields carefully
Nested objects and arrays are the most common failure points. The model may collapse arrays into comma-separated strings, invent sub-fields, or return inconsistent item counts.
The safest approach is to keep nested structures shallow and describe array items with the same rigor you use for top-level fields. If an array item has five properties, specify all five in the schema.
| Pattern | When to use | Risk |
|---|---|---|
| Flat extraction | Simple key-value data | Loses relationships between items |
| Nested object | One-to-one relationships | Model invents sub-fields |
| Array of objects | Line items, events, records | Inconsistent item shapes |
| Two-pass split | Complex records with many fields | Extra latency, more prompts |
For provider-level mechanics around JSON outputs, see structured output and JSON mode.
Use a judge prompt to catch silent failures
Parsing errors are loud—you get a stack trace. Silent failures are worse: the JSON parses, but a field is wrong, an array is empty when it should not be, or a number was extracted as a string. These pass through pipelines and corrupt dashboards.
A lightweight judge prompt can catch silent failures without manual review. Feed the extracted JSON and the original source text to a second model call with a checklist of invariants: required fields present, types correct, counts match, and no invented values.
The judge does not need to be perfect. It only needs to catch failures that would otherwise reach production. Even a 70% catch rate reduces broken data significantly.
Limits and notes
Structured output works best when the source text is clean and the schema is tight. Handwritten notes, OCR output, and noisy transcripts need preprocessing before extraction. If the source is messy, fix the input first.
JSON mode and constrained decoding are not universal. Check your provider's current support before building around them. Test with your specific model and version.
Extraction is one half of the pipeline. The other half is storage. Design your downstream schema to match the extraction schema so you do not spend saved time on reconciliation.