When an LLM returns invalid JSON: truncation, refusals, repair

Your parser throws, the model was told to return JSON, and now the request fails somewhere between generation and your database. The first diagnostic question is whether you received a complete answer, a refusal, or only a streaming fragment. The second is whether the model and the API path you selected actually enforce your schema, because JSON mode alone does not. In AIVAX, response_schema enables gateway validation and bounded healing; response_format can use native provider enforcement or healing, depending on the request and account settings.

The useful boundary is who checks the final answer before your code uses it. Native structured output constrains generation at the provider. AIVAX JSON Healing checks generated output and can ask the model to correct it. Neither mechanism proves that an identifier exists, a claim is true, or a database write is authorized.

Three failures hide behind "invalid JSON"

"Invalid JSON" can describe three different failures. An unfinished object does not parse. A complete object with a missing required field parses but fails its schema. An object containing the wrong customer ID may pass both checks and still be unusable. Diagnose them separately before changing the prompt or retry budget, because each failure has a different fix: truncation needs room or a smaller schema, a schema miss needs validation feedback, and a wrong value needs application-level checks that no schema keyword can express.

Why do I get invalid JSON even with strict: true?

This is the most common report, and it rarely has a single cause. Work through four checks before touching the prompt:

1. Handle a refusal separately from the requested object. A token limit or content filter can also prevent a usable final answer; OpenAI's official parsing helpers raise distinct errors for finish_reason: "length" and "content_filter". Do not repair a refusal into a successful business result.
2. Assemble the answer before parsing it. A streaming delta is a fragment, not necessarily a complete JSON value. Use the SDK's stream accumulator or collect the final answer before calling an ordinary JSON parser, and keep reasoning and tool events separate from answer content; our reasoning protocol guide explains why those channels should not be flattened into one string.
3. Check the mode and schema dialect. OpenAI's json_object mode targets valid JSON, not your required fields or enums. Its strict json_schema mode requires a supported schema, including additionalProperties: false on objects and every field listed in required. Follow the Structured Outputs guide, not just a prompt saying "return JSON."
4. Check the actual API path. A setting in your client does not establish what a gateway sends upstream. For OpenAI directly, Chat Completions uses response_format; Responses uses text.format. In AIVAX, check whether healing is active before assuming the provider receives a native schema parameter.

OpenAI documents refusals separately from schema-conforming answers, and a client-supplied strict: true does not prove what the upstream request contained. In AIVAX, active healing uses schema instructions and validation instead of forwarding the native response format, so the client flag alone does not identify the enforcement path. If a complete, non-refusal response still fails validation under the intended native mode, preserve a redacted minimal request, the response, the model identifier, the completion state, and the parser error for investigation. Reports of "strict mode failed" do not share one cause.

AIVAX has two enforcement paths

Use response_schema when AIVAX should check the final JSON and retry recoverable failures. AIVAX places the schema in the model's instructions, extracts JSON from the generated answer, and validates the candidate. This path does not require native structured-output support, but it does not guarantee that every model can satisfy every schema.

With response_format.type: "json_schema", healing changes the path. Explicit response_format.json_schema.healing_options or the account's automatic JSON Healing setting enables gateway validation and correction. Automatic healing is enabled by default, so omitting healing_options does not by itself select native enforcement.

For the native path, use a compatible model, omit response_schema and explicit healing options, and ensure automatic JSON Healing is disabled for the account. Without healing, AIVAX builds a provider-native schema request rather than performing the healing validation loop. For integrated models, the outgoing strict setting follows the model's configured capability, not the client's flag.

Other response formats, such as json_object, do not activate schema healing by themselves. The AIVAX Structured Responses reference documents the parameters and supported validation features.

Healing extracts, validates, and retries with feedback

Before asking the model again, AIVAX tries to extract JSON from the complete generated text, common delimiter-repaired variants, and JSON code blocks. Extraction can recover some formatting mistakes without another generation; it is not evidence that the recovered values are correct.

If the candidate fails schema validation, AIVAX adds validation feedback to the model's conversation and requests a corrected answer. Unparseable output receives a format-correction instruction. The retry budget is bounded, and repeated failure can end the request with an error rather than a usable result. Those correction messages are part of the healing process, not a promised public diagnostics field.

Healing targets the final schema-constrained answer, including an answer produced after tool results. It does not replace validation or authorization of tool-call arguments. Tool use still requires a compatible model or tool handler; removing the need for native JSON support does not remove other capability requirements.

Each retry is another model generation, so the example's max_attempts: 5 allows the first generation plus five repairs for one request. Fix an impossible schema, missing context, or ambiguous instruction before raising the budget. Recovery is useful for formatting and shape errors, not for turning missing evidence into a trustworthy answer.

Enabling bounded healing in a request

This request explicitly enables gateway healing through response_format. Send this JSON body to POST https://inference.aivax.net/v1/chat/completions with Content-Type: application/json and Authorization: Bearer YOUR_AIVAX_API_KEY. See the inference API guide for the endpoint contract.

Replace <AVAILABLE_MODEL_ID> with a text-generation model available to your account. This example uses gateway healing, not native OpenAI strict mode.

{
    "model": "<AVAILABLE_MODEL_ID>",
    "messages": [{ "role": "user", "content": "Return a short status object." }],
    "stream": false,
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {
                    "status": { "type": "string" },
                    "message": { "type": "string" }
                },
                "required": ["status", "message"]
            },
            "healing_options": { "max_attempts": 5 }
        }
    }
}

The documented range for max_attempts is 1 to 10. This bounds healing retries; it should not be read as a cap on every model call in a tool-using workflow. On success, read the final answer from the chat completion envelope and parse its JSON content. The schema requires string-valued status and message fields; it does not define which status values your application should accept.

To use response_schema instead, move the object currently inside response_format.json_schema.schema to the top-level response_schema field and remove response_format. That selects the gateway-owned healing path rather than native provider enforcement.

Streaming and json_only change the envelope, not the guarantees

json_only: true changes the response envelope and checks only that the final content parses as JSON. With stream: false, a successful response contains the final JSON as application/json, without the normal choices, usage, and generation metadata. With stream: true, AIVAX sends the complete JSON as one SSE data event followed by [DONE].

Without json_only, do not assume that every event in a healing stream is JSON answer content. Reasoning, tool activity, and usage are separate concerns. Select and assemble the final answer channel, and handle error events or interrupted delivery before passing the result downstream.

Valid JSON still needs application-level checks

Define required fields explicitly, give arrays an items schema, and use supported constraints such as enum when the accepted values are known. For extraction, state which fields may be null when evidence is absent and which values must never be invented. On AIVAX's extract endpoint, a field declared as a number can still arrive as a string when the model copies 1.234,56 as the document writes it, as the extraction post found by running the server's parse options. Use the schema subset documented for the mode you selected rather than assuming every JSON Schema keyword works everywhere. In particular, AIVAX's healing validator does not enforce additionalProperties: false; validate unexpected keys in your application if rejecting them matters.

Then apply the checks no schema keyword expresses for your case: whether the status value is one your application acts on, whether the identifiers in it exist, and whether the caller is allowed to trigger the action. Google's Gemini documentation makes the same distinction: syntactically correct JSON still needs application-level value validation. A repaired object can be structurally acceptable and factually wrong.

For a workflow whose output is a routing decision rather than a generated document, see support triage with typed decisions. In either design, the application owns the policy that turns an accepted value into an action. When you swap the model behind a structured call, replay your accepted outputs first; the pin-or-alias guide lists what a replacement run should check.

The request above was run against the live AIVAX API on November 10, 2026 with @openai/gpt-5.4-nano: it returned 200 with the normal chat completion envelope and a message.content of {"status":"ok","message":"Short status returned successfully."}. The response_schema variant returned the same envelope, and with json_only: true the body was the bare JSON for stream: false and a single data: event followed by [DONE] for stream: true. The retry path, the extraction rules and the additionalProperties behavior were read on 2026-11-02 from the AIVAX server implementation and the Structured Responses documentation.