azyware
Technology

Structured outputs and function calling: making LLMs reliable for software

EZ
Eazyware
· 7 min read
Quick answer

How do structured outputs and function calling make LLMs reliable for software?

Structured outputs and function calling force LLM responses into validated schemas so downstream code never parses prose. The model returns typed fields, the application validates them, and a failure is a countable error, not silent corruption. Here is how each mechanism works and the validation layer you still own.

Structured outputs are the single practice that turns an LLM from a text generator into a component software can depend on. Instead of asking the model for a paragraph and hoping a regular expression finds the order number in it, the application declares a schema, the provider constrains the model to produce output matching it, and the code that consumes the result reads typed fields. Function calling applies the same idea to actions: the model chooses a tool and fills its arguments against a declared signature, and the application decides whether to run it. Both are standard in every build under the LLM applications service; this article explains what each does, where they differ, and the validation you must still own.

Why prose parsing fails in production

Early LLM integrations asked the model to "respond in JSON" and parsed whatever came back. It worked in the demo. In production the model occasionally wrapped the JSON in a sentence, used a single quote, added a trailing comma, renamed a field, or returned a number as a string. Each variation broke a parser somewhere downstream, and because the failure was intermittent and input-dependent, it was hard to reproduce and easy to blame on the user. Worse were the silent failures: a field present but wrong, a date in the wrong format that a lenient parser accepted, an enum value the model invented. These reached the database and were found weeks later.

Reliable LLM outputs require two things: the provider constraining generation to the schema, and the application validating the result regardless. The first removes most malformed responses; the second catches the rest and turns them into a visible error with a retry policy.

JSON mode, structured outputs and function calling compared

MechanismWhat it guaranteesBest forWhat it does not guarantee
JSON modeOutput is syntactically valid JSONQuick prototypes; loosely defined dataField names, types, required fields, enum values
Structured outputs (schema-constrained)Output conforms to a supplied JSON SchemaExtraction, classification, decisions with reasons, any data the code consumesSemantic correctness: the model can fill a valid schema with wrong values
Function calling / tool useModel selects a declared tool and returns arguments matching its signatureActions: look up an order, create a ticket, query a database, hand off to a personThat the action should run; the application must gate it
Grammar-constrained decoding (self-hosted)Token-level enforcement of any grammarOpen-weight models in private deploymentsSame semantic caveats; needs the serving stack to support it

Structured outputs: declare the schema, validate anyway

The application defines a schema for each output type: an extracted invoice with typed amounts and dates, a classification with an enum and a confidence, a decision with a reason and a list of cited passages. The schema is passed to the provider, which constrains generation so that the response conforms. OpenAI's structured outputs and Google's Gemini structured output both accept JSON Schema; Anthropic reaches the same result through tool definitions. Open-weight models served with vLLM support grammar-constrained decoding for the same purpose in private deployments.

Then the application validates the response against the same schema with its own library, because the provider guarantee covers structure, not meaning. Validation checks that required fields are present, that enums are within range, that a date is a real date, that a total equals the sum of its lines where that rule applies, and that any cited passage actually appears in the retrieved context. A validation failure triggers a retry with the error message included, up to a bounded count, after which the request is rejected and logged. Each rejection is a data point for the eval set.

Designing the schema

Keep schemas flat and explicit. Use enums for anything with a fixed set of values, and include an "other" or "unknown" option so the model is not forced to guess. Make optional fields nullable rather than omitting them, so absence is a value the code can check. Add a reason or evidence field next to any judgement, which improves both accuracy and auditability. Version the schema alongside the prompt, because a schema change is a behaviour change and belongs in the same review and eval run described in prompt versioning and evaluation.

Function calling: the model proposes, the application disposes

Function calling, or tool use, gives the model a list of tools with names, descriptions and argument schemas. Given a user request, the model returns the tool it wants to call and the arguments, structured against that schema. The application then decides what to do. This last point is the whole design: the model never executes anything. The application validates the arguments, checks the caller's permissions, applies policy (a refund above a threshold needs a person; a lookup is free to run), executes the tool, and returns the result to the model for the next step.

Tool descriptions are prompts and deserve the same care: a precise description of what the tool does, when to use it and what each argument means reduces wrong tool choices more than any other change. Keep the tool list short per step; a model offered forty tools chooses worse than one offered six. Where an agent needs many tools, route by intent first and expose the relevant subset. The Anthropic tool-use documentation is a good reference for the request and response shapes. Policy gating for consequential actions is covered in policy-gated actions.

Where structured outputs still go wrong

Valid structure with wrong content is the residual risk. A model can return a well-formed invoice record with the wrong total, a classification with an enum that is legal but incorrect, or a tool call with a customer ID that belongs to someone else. The defences are the same ones that apply to any LLM output: evals per output type with per-field accuracy, groundedness checks that tie extracted values back to source text, confidence fields that route low-confidence results to review, and business-rule checks in the validation layer. Structured outputs make errors visible and countable; they do not make them impossible. The broader approach to this is in how to handle hallucinations in production.

A worked example

A lending company was extracting fields from KYC documents with a prompt that asked for JSON and a parser that accepted whatever came back. Most documents worked. A small share produced records with a date of birth in the wrong format, an address field that had absorbed the next line of the document, or an ID type the model had described in words rather than picked from the list. These reached the review queue as apparently valid records, and reviewers caught some but not all.

The rebuild declared a schema per document type with enums for ID type, ISO dates, nullable optional fields, and a confidence and source-region field for each value. The provider was asked for schema-constrained output; the application validated every record, applied rules such as the ID number format for each ID type, and sent any record with a low-confidence field or a rule failure to a human queue with the field highlighted. Malformed records stopped reaching the database, and reviewers spent their time on the genuinely ambiguous pages. The same pattern is described in the KYC document-intelligence case study, and the routing of easy pages to a smaller model, described in multi-model routing, depended on the schema being enforced regardless of which model produced it.

Team and timeline

Moving an existing feature to structured outputs and function calling is usually one AI engineer for one to three weeks: define the schemas, switch the provider calls, write the validation and retry layer, build a per-field eval from traces, and add policy gates to any tool that changes state. On a new build it is part of the first fortnight's skeleton. The work is included in every LLM applications engagement, from $21,000 / ₹13.6L, and in AI agent builds where tool use is the core of the system. A three-week ProofRun at $6,250–10,500 is the right first step when the question is whether extraction accuracy on your documents will be good enough. Current figures are on the pricing page.

Before you start: a checklist

  • List every place downstream code consumes model output and define a schema for each
  • Use enums with an explicit unknown option for any fixed set of values
  • Add a reason, confidence or evidence field beside every judgement
  • Validate every response in the application, not only at the provider
  • Set a bounded retry that feeds the validation error back to the model
  • Gate every state-changing tool with permission and policy checks before execution
  • Keep tool lists short per step and write tool descriptions as carefully as prompts
  • Build a per-field eval from real inputs and version schemas with prompts

Glossary

  • JSON Schema: a standard for describing the shape of JSON data, used to declare output structure to the provider and to validate the response.
  • Structured outputs: a provider mode that constrains generation so the response conforms to a supplied schema.
  • Function calling / tool use: the model returning a chosen tool name and arguments against a declared signature, for the application to execute or refuse.
  • Grammar-constrained decoding: token-level enforcement of a grammar in a self-hosted serving stack.
  • Policy gate: an application-side check that decides whether a proposed action runs, waits for approval or is refused.

See what makes an LLM application production-ready for where structured outputs sit among the six requirements, MCP explained for a standard way to expose tools to agents, and document intelligence for extraction from scans and forms.

Declare the shape, let the provider constrain it, validate it yourself, and gate every action; the model becomes a component your software can trust because its failures are visible.

Frequently asked questions

What is the difference between JSON mode and structured outputs?

▾

JSON mode guarantees only that the response is valid JSON. Structured outputs constrain the response to a supplied schema, so field names, types, required fields and enums are enforced. Structured outputs are the right default for any data the code consumes.

Do we still need to validate if the provider enforces the schema?

▾

Yes. The provider guarantees structure, not meaning. Application-side validation catches wrong enum choices, impossible dates, totals that do not add up and cited passages that are not in the context, and turns them into a retry or a review-queue item.

Is function calling safe for actions like refunds?

▾

Only when the application gates execution. The model proposes a tool and arguments; the application checks permissions and policy, requires approval above thresholds and logs the decision. We design this into every agent build.