Pulling the payer, the due date and the line items out of an invoice used to mean coaxing a model into valid JSON and then cleaning up whatever came back. LLM structured outputs change the first half of that problem: you hand the API a JSON Schema, and the model’s output is constrained to match it. They do not change the second half. A response can fit the schema perfectly and still carry the wrong amount, a made-up date or a field filled in from nothing.

This explainer walks through how structured outputs work at OpenAI and Anthropic, what each provider says the feature guarantees, and the checks a financial data extraction pipeline still needs.

What structured outputs do

Both providers describe the same core idea. OpenAI’s Structured Outputs guide says the feature “ensures the model will always generate responses that adhere to your supplied JSON Schema,” so a required key is not omitted and an enum value is not invented. Anthropic’s structured outputs documentation says Claude’s responses are constrained to the schema “through constrained decoding,” with the schema compiled into a grammar that limits what the model can produce.

In practice you send a schema with the request. At OpenAI it goes in a json_schema response format with strict: true. At Anthropic it goes in output_config.format with type: "json_schema"; the older output_format parameter is deprecated. Both providers’ SDKs can build the schema from a Pydantic model or a similar type definition and parse the reply back into that type.

Structured outputs are a sibling of function calling, not a replacement. Anthropic offers strict tool use (strict: true on a tool) to constrain tool inputs, and JSON outputs to constrain the final answer; the two can be combined. We covered the tool side in our explainer on LLM function calling.

OpenAI also notes the difference from its older JSON mode: both produce valid JSON, but only Structured Outputs enforce the schema, and it recommends Structured Outputs whenever possible.

The schema rules differ by provider

Neither provider accepts all of JSON Schema, and the gaps matter for financial fields.

Optional fields. OpenAI requires every field to be listed as required. To model a value that may be absent, it suggests a union type with null. Anthropic allows optional properties but caps them at 24 across all strict schemas in a request, and caps parameters with union types at 16, because each one makes the compiled grammar larger.

Numeric and string limits. Anthropic does not support minimum, maximum, multipleOf, minLength or maxLength. Its SDK helpers move those constraints into the field description and, where the helper validates responses, still check them in your code. OpenAI lists allOf, not and conditional keywords among those not yet supported, and adds more exclusions for fine-tuned models.

Size and shape. OpenAI allows up to 5,000 object properties and 10 levels of nesting per schema, and up to 1,000 enum values across all enum properties. It supports recursive schemas. Anthropic does not support recursive schemas and returns a “Schema is too complex for compilation” error when the combined grammar grows too large.

Order and case. OpenAI produces keys in schema order. Anthropic keeps schema order except that required properties come before optional ones. Anthropic also warns that structured outputs do not guarantee the capitalization of string enum and const values, so a response can differ from your enum only in case, with no error. It advises comparing enum values case-insensitively.

Latency. Both say the first request with a new schema is slower while it is processed. Anthropic caches compiled grammars for 24 hours from last use.

When the output still is not valid

Both providers list cases where a response may not match the schema at all:

  • Refusals. OpenAI returns a separate refusal field when the model declines for safety reasons. Anthropic says a refusal takes precedence over the schema.
  • Truncation. If the response hits the token limit, it can be incomplete. Anthropic flags this with stop_reason: "max_tokens", and OpenAI marks the response incomplete with the reason max_output_tokens. Both suggest checking for this before parsing.

Code that parses the result should handle both paths explicitly rather than assume the JSON is always there.

Schema-valid is not the same as correct

The bigger risk for financial extraction is a response that parses cleanly and is wrong. OpenAI says plainly that “Structured Outputs can still contain mistakes.” It also warns that the model “will always try to adhere to the provided schema, which can result in hallucinations if the input is completely unrelated to the schema.” A schema that demands an interest_rate number will get a number, whether or not the document states one.

A few design choices reduce that risk:

  • Make absence representable. Give fields a null option, or an explicit “not stated” value, so the model is not forced to invent a figure. OpenAI suggests telling the model in the prompt what to return when the input cannot produce a valid answer.
  • Validate in code what the schema cannot. Check that line items sum to the total, that dates fall in a plausible range and that currency codes are real. With Anthropic, numeric bounds are not enforced by the API at all.
  • Keep the evidence. Ask for the page or quoted text each value came from, then confirm that text exists in the source. This matters most for long filings, where our analysis of RAG for financial documents found retrieval often breaks on tables and look-alike documents.
  • Treat the document as untrusted. A supplier invoice can carry instructions aimed at the model. Our analysis of prompt injection defenses covers why output checks should assume an injection can get through.

Anthropic adds one data-handling note for regulated teams: its compiled schemas are cached separately from prompts and responses, so sensitive data such as protected health information should not appear in property names, enum values or patterns.

Sources