Articles
Structured Output Interview Questions: Schemas, Refusals, and Validation Failures
Practice structured output interview questions on schemas, refusals, incomplete responses, and business validation with examples and clear recovery paths.
A model returns valid JSON, every required field is present, and the application still routes a support ticket to the wrong team. Nothing failed at the syntax layer. The error is in the meaning of a value that the schema allowed.
For structured output interview questions, separate three checks: did the response finish successfully, does its payload match the contract, and is the proposed result supported by evidence and application rules? A schema answers only part of that sequence.
Evidence boundary: Provider-specific facts are attributed to official OpenAI documentation. JSON Schema and Pydantic documentation support the validation distinctions. Examples, recovery policies and evaluation figures are original teaching material. No candidate reports are used, and the practice records do not predict your employer's assessment. Sources were checked on September 15, 2026.
What is the difference between JSON mode and schema enforcement?
Official OpenAI distinction: JSON mode targets valid JSON; Structured Outputs adds adherence to a supplied supported schema. Strict output uses a supported subset of JSON Schema, rather than every feature accepted by every validator. Refusals and incomplete responses need separate handling. OpenAI: Structured model outputs
In an interview, distinguish a prompt instruction such as “return JSON” from an interface-level output constraint. The former expresses a request. The latter establishes a more specific generation contract, subject to that provider's documented conditions.
Then distinguish both from factual correctness. A schema can require a field called team and restrict it to two strings. It cannot establish that the model selected the team justified by a particular customer's message unless that relationship is checked elsewhere.
Do not describe constrained output as removing validation from the application. It changes the expected failure distribution. The application still owns response-state checks, schema-version compatibility, domain rules, authorization and the decision to perform any side effect.
A concise answer is: “I use the provider's structure guarantee where supported, then validate the response envelope and the application contract before accepting the result.” Be ready to name the failure paths hidden inside that sentence.
How would you design a useful extraction schema?
Original example: A support-routing service extracts a ticket ID, a proposed team and a short evidence quotation. The application already knows the request's ticket ID. The model should repeat it so mismatches can be detected; it does not become authoritative merely by returning an ID.
Here is a deliberately small payload schema:
{
"type": "object",
"properties": {
"ticket_id": {"type": "string"},
"team": {
"type": ["string", "null"],
"enum": ["billing", "technical", null]
},
"evidence": {
"type": ["string", "null"]
}
},
"required": ["ticket_id", "team", "evidence"],
"additionalProperties": false
}
In standard JSON Schema, listing a property does not automatically require its presence. The required array specifies presence, while additionalProperties: false rejects unlisted keys at that object level. Nested objects need their own appropriate constraints. JSON Schema: Objects
For OpenAI's strict subset, fields must be required; a nullable value can represent an unavailable answer. Confirm the exact schema subset and model support before sending the request. An unsupported schema is a configuration problem to fix, not a reason to repeat the same request indefinitely. OpenAI: Supported schemas
Keep the contract small enough to explain. More fields create more ways to disagree with the source and more migration work for consumers. Add a field because a downstream decision needs it, not because a longer JSON object looks more sophisticated.
Are missing, null and an empty string interchangeable?
No. JSON null is a value, and a property containing null is different from an absent property. Whether either is allowed depends on the schema. JSON Schema: Null
In this example, an omitted evidence field fails the presence requirement. An explicit "evidence": null passes that part of the contract. An empty string is also a string, but the application can reject it as unusable evidence when a routing decision has been proposed.
Suggested business rule: If the model cannot justify a team, it returns null for both team and evidence. If it proposes a team, it must supply a nonempty quotation present in the original ticket. A null value means “no supported routing decision,” not “send to the default team.”
The schema alone does not express that whole policy. The consumer must enforce the relationship between fields and the relationship between the output and its source.
This is a useful follow-up to an apparently correct answer about optional fields. Ask what an empty string means operationally. If it silently becomes a default route, the system may be hiding uncertainty instead of representing it.
Show a schema-valid response that the application should reject
Suppose ticket T17 says, “I was charged twice this month.” The model returns:
{
"ticket_id": "T17",
"team": "technical",
"evidence": "The app crashes on login."
}
Every value fits the schema, but the evidence quotation does not occur in the source. That is a domain-validation failure, not a JSON parsing error. Retrying a parser cannot fix invented evidence.
An application check might look like this after schema validation:
def accept_route(data, ticket_id, source):
if data["ticket_id"] != ticket_id:
return False
team = data["team"]
quote = data["evidence"]
if team is None:
return quote is None
if quote is None or not quote.strip():
return False
return quote in source
This is an original minimal check, not a complete classifier. It rejects the invented quotation and inconsistent null pair. It still cannot prove that a quotation supports the selected team. For example, a real quotation about a duplicate charge can be attached to the wrong team and pass this function.
Explain that limitation rather than adding a token keyword rule and calling the problem solved. Routing correctness needs a separately defined decision policy and labeled evaluation cases; an ambiguous ticket may require abstention or review.
Local validator settings matter too. Pydantic normally supports coercion in many cases; strict mode reduces coercion, with behavior depending on the type and input path. Decide whether converting a numeric string into a number is acceptable for your contract. Pydantic: Strict mode
How should the consumer handle refusal and interruption?
Do not force every provider response through the success-payload parser. OpenAI documents refusal content separately from the requested structured payload, and an incomplete response can indicate that generation stopped before completion. Inspect the response state before consuming the business object. OpenAI: Refusals and incomplete output
Recommended consumer states: Represent success, refusal, incomplete generation, request failure and validation failure as distinct outcomes. Keep their metadata outside the model's business schema. A refusal is not a successfully extracted ticket with empty fields.
An incomplete response also differs from an abstention. The model may not have finished deciding anything. Appending closing braces to a truncated string may make it parse, but it cannot recover an omitted qualification or missing field value faithfully.
Streaming adds another boundary. OpenAI's streaming interface emits typed events as output is produced. Your consumer should distinguish incremental content from completion and error events. OpenAI: Streaming responses
Design recommendation: Show provisional content only where the product makes that status clear. Commit a routing decision after the complete result passes validation. A chunk that happens to contain a team name is not a durable decision, and a partially parsed object is not equivalent to a completed response.
Which failures should you retry?
A blanket “retry three times” answer ignores why the request failed. Use a bounded policy tied to the failure class and the caller's time budget.
| Failure class | Defensible next action |
|---|---|
| Invalid or unsupported request schema | Fix configuration; do not repeat unchanged input. |
| Transient transport failure | Retry within a deadline, with backoff and duplicate-work protection. |
| Refusal | Preserve the refusal outcome and follow the product's handling policy. |
| Incomplete generation | Inspect the cause; adjust the budget or task scope when appropriate. |
| Schema or domain validation failure | Record the violated rule; repair only through a controlled, bounded path. |
These are design recommendations, not universal provider retry rules. Authentication problems, invalid parameters and a genuinely unavailable field each require a different response from a temporary connection error.
If you send validation feedback to the model, provide the specific failed condition and retain the original evidence. Do not silently replace the original response, lose its error category or keep sampling until one output happens to pass.
A second model-generated answer may satisfy the same superficial validator while still being wrong. Count retries in latency, cost and end-to-end acceptance metrics. For repeated tasks that can write data, use a stable operation identity so uncertain network outcomes do not cause duplicate actions.
How does this differ from function calling?
Official OpenAI documentation describes function calling as a flow where the model proposes a tool call, application code executes the function, and the result can be returned to the model. The model's arguments are not themselves execution. OpenAI: Function calling
Choose the interface according to the job. Extracting a ticket classification for your application can use a structured response. Asking the assistant to invoke a routing tool creates an additional execution boundary.
In either case, the application must check permissions using trusted request context. A perfectly typed tenant_id or ticket_id supplied by a model does not prove access. Look up the actual object and authorize the operation independently before committing it.
For the support example, keep “proposed team,” “validated route” and “route saved” as separate states. This makes failures diagnosable: a valid proposal may encounter a concurrent ticket update or a failed write after the model has completed successfully.
How would you evaluate and evolve the contract?
Build fixtures for malformed payloads, missing keys, extra keys, null combinations, valid-but-wrong decisions, unsupported quotations, refusal and interruption. Include ambiguous source material that should lead to abstention. A suite containing only successful JSON misses the refusal, interruption and semantic-error paths.
Original denominator example: Across 100 attempts, suppose there are 4 refusals, 3 incomplete outputs, 2 schema failures and 6 domain failures; the remaining 85 are accepted. Structural validity among the 93 completed non-refusal payloads is 91/93, about 97.8%. End-to-end acceptance is 85/100, or 85%. Neither number alone establishes decision accuracy.
Report correctness on labeled cases and separate justified abstention from avoidable failure. Slice results by source ambiguity, document length, language and schema version where those dimensions matter to the product.
Version the prompt, model configuration, output schema and consumer together. Adding an enum value can break exhaustive handling even when the new payload is valid under the new schema. Adding a required field can break an older producer. Replay representative saved fixtures and shadow the new consumer before switching traffic.
Logs should capture the response state, schema version, failed rule and attempt count without unnecessarily retaining sensitive source text. The goal is enough evidence to reproduce the failure, not an unrestricted archive of every customer message.
Practice five connected questions
These records exercise adjacent implementation and design skills. They are practice material, not a forecast of your exact interview.
| Question | Follow-up to practise |
|---|---|
| Design an LLM API pipeline | Draw separate refusal, interruption, validation and success paths. |
| Design prompts for JSON-only LLM responses | Explain what a prompt instruction cannot establish about the returned object. |
| Reason About Nested JSON Schema Validation | Distinguish required keys, null values and nested extra fields. |
| Design an LLM Log Parsing Workflow | Preserve source evidence and represent fields that cannot be extracted. |
| Build a CSV-to-JSON Classification Service | Validate the classification and schema independently before saving a result. |
Start with the LLM API pipeline exercise. Give your consumer a schema-valid but unsupported answer, then explain exactly which check rejects it and what the caller receives next.