Fix Claude API Truncating JSON Outputs Mid-Response

I had a pipeline pulling structured data out of documents, parsing the response as JSON, and it kept failing on json.loads() with an unexpected end of input. Claude API truncating JSON outputs mid-response was my first assumption, and it turned out to be right, but not for the reason I initially guessed. I was blaming the model for “getting lazy” on long outputs when the actual cause was sitting right there in the response object the whole time.

If you’re currently wrapping your JSON parsing in a try/except and just retrying blindly when it fails, you can stop doing that. There’s a field in the response that tells you exactly why it happened, and most of the fixes take less time than the retry loop you probably already built.

Quick Answer

• Check response.stop_reason before you assume the JSON is broken — this is the single most useful thing you’re probably not doing yet.
• stop_reason == "max_tokens" means your max_tokens value was too low for the full JSON payload; raise it.
• stop_reason == "model_context_window_exceeded" means the response filled the model’s context window rather than hitting your token limit — that needs a different fix than just raising max_tokens.
• If you’re prompting for JSON with plain instructions instead of using Structured Outputs, that’s a separate and very common source of malformed output that looks like truncation but isn’t.
• For streaming implementations, truncation often comes from not correctly accumulating text deltas, not from the model itself.

Why It Fails

max_tokens is set lower than what the JSON actually needs. This is the most common cause by a wide margin. If your schema includes a list that can grow — search results, extracted rows, nested objects — the token count needed to represent it isn’t fixed, and a max_tokens value that worked fine during testing can quietly fall short once real data hits it.

The model’s context window gets filled before max_tokens is reached. This is a genuinely different failure mode from a low max_tokens value, and it’s easy to conflate the two. stop_reason distinguishes between them: max_tokens means your own limit was hit, while model_context_window_exceeded means the combined input and output filled the model’s window first. Treating both the same way in your error handling means you might be raising max_tokens when the real fix is reducing your input size instead.

Relying on prompt instructions alone instead of a schema-enforced output. Asking Claude to “respond only in valid JSON” works most of the time, but “most of the time” isn’t good enough for a production pipeline. Without a mechanism actually constraining generation to match a schema, you can get subtly malformed JSON — a missing closing brace, a stray trailing comma — that looks exactly like truncation when your parser chokes on it, even when the response technically finished with end_turn.

Streaming responses not being reassembled correctly. If you’re streaming and parsing content_block_delta events, a truncated-looking result is often a client-side bug rather than anything the API did. Missing an event type, dropping partial UTF-8 sequences at chunk boundaries, or checking stop_reason mid-stream instead of from the final message_delta event will all produce output that looks cut off.

Accidental stop sequences matching content inside the JSON itself. If you’ve set custom stop_sequences for some other part of your workflow and forgot they’re still active, and one of those sequences happens to appear naturally inside a JSON string value, Claude will stop generating right there — and stop_reason will read stop_sequence, which is worth checking for specifically.

Diagnosing Which One You Actually Have

Step-by-Step Fixes

Step 1: Check stop_reason before anything else

python

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": prompt}],
)

if response.stop_reason == "max_tokens":
    # Your limit was hit, not a model problem
    pass
elif response.stop_reason == "model_context_window_exceeded":
    # Context window filled — different fix needed
    pass

This single check tells you which of the causes above you’re actually dealing with, instead of guessing.

Step 2: Raise max_tokens if that’s the actual cause

Obvious, but worth saying directly: if stop_reason is max_tokens, the fix is raising the limit, not changing your prompt. Give yourself real headroom above what you think you need — JSON with nested structures and nullable fields tends to use more tokens than people expect.

One caveat if you’re on the Python SDK and pushing max_tokens above roughly 21,000: streaming becomes required at that point rather than optional, so factor that into how you’re calling the API.

Step 3: Switch to Structured Outputs instead of prompting for JSON

If your stop_reason is end_turn and the JSON is still malformed, prompting alone isn’t enough. Structured Outputs compiles your JSON Schema into a constraint on generation itself, so the response is guaranteed to match your schema rather than just being asked to. You can use this through the output_format parameter for direct JSON responses, or through strict: true on a tool definition if you’re already using tool calls to get structured data out of Claude.

Step 4: Implement the continuation pattern for genuinely long JSON

If your payload legitimately needs more tokens than a single response can hold, don’t just raise max_tokens indefinitely. Send a follow-up request with the partial response included and ask Claude to continue from where it left off:

python

messages = [
    {"role": "user", "content": prompt},
    {"role": "assistant", "content": partial_response},
    {"role": "user", "content": "Continue the JSON from exactly where it left off."},
]

Then concatenate the two outputs and parse the combined result.

Step 5: Fix streaming accumulation if that’s your setup

Make sure you’re appending text deltas from content_block_delta events into a single buffer, and that you’re only checking stop_reason from the message_delta event, not from any earlier event type where it’ll be null. This trips people up more often than the actual model output does.

What Actually Worked For Me

My first move was bumping max_tokens way up, since that’s the obvious guess. It helped some of the time but not consistently, which should’ve told me right away that I had more than one cause stacked on top of each other.

Once I actually logged stop_reason on every failure instead of just catching the JSON parse error, the picture got a lot clearer. About two-thirds of my failures were genuinely max_tokens, and raising the limit fixed those outright. But the rest were end_turn with subtly malformed JSON — missing commas, occasionally a field name that didn’t match my schema. Those weren’t truncation at all. Moving that part of the pipeline to Structured Outputs cleared up basically all of it, and in hindsight I probably should’ve started there instead of treating everything as a token-limit problem.

Advanced Fixes and Edge Cases

If you’re using model_context_window_exceeded deliberately, you can actually use it as a feature rather than an error. Setting a high max_tokens value and letting the context window itself be the limiting factor lets you request as much output as possible without pre-calculating exact input token counts. Just make sure your error handling treats this stop reason differently from a plain max_tokens hit, since the fix (reduce input) is the opposite of what you’d do for the other (raise limit).

If your JSON includes large arrays of extracted records, consider whether the task can be chunked into multiple smaller requests instead of one giant structured response. This sidesteps token-limit tuning entirely and tends to be more resilient overall.

If truncation only happens intermittently in production but never in testing, check for infrastructure timeouts sitting between your app and the API — a load balancer or reverse proxy cutting a long-running streaming connection can produce output that looks exactly like an API-side truncation but isn’t one at all.

Log the full response object, not just the parsed content, on every failure. This sounds basic, but from what I’ve seen, most people debugging this issue are working from the parse error alone and never actually look at stop_reason, stop_sequence, or usage on the failed response, which is where the real answer usually is.

Prevention Tips

• Always check stop_reason in your response handling, even on the happy path, not just when something breaks.
• Use Structured Outputs for anything going into a parser rather than relying on prompt instructions alone.
• Give max_tokens real headroom for variable-length JSON, and treat model_context_window_exceeded as a distinct case in your error handling rather than lumping it in with max_tokens.
• If you’re streaming, test with genuinely long responses during development, not just short ones, since accumulation bugs often only show up once a response spans multiple chunks.

FAQ

Does raising max_tokens cost more even if the response doesn’t use the full amount? No, you’re billed for tokens actually generated, not the ceiling you set.

Is Structured Outputs the same as just adding “respond in JSON” to my prompt? No. Prompt instructions are a request the model tries to follow; Structured Outputs constrains generation at the schema level, so malformed output that fails to match your schema shouldn’t happen at all under this feature.

Why does the same prompt sometimes truncate and sometimes not? Variable-length content in the response (longer lists, more nested objects) means token usage isn’t fixed run to run, so a max_tokens value that’s fine most of the time can occasionally fall short.

Can stop_sequences cause this even if I didn’t mean to use them for JSON at all? Yes, if a stop sequence set for an unrelated part of your workflow happens to match text that legitimately shows up inside a JSON string value. Worth double-checking if stop_reason comes back as stop_sequence unexpectedly.

Editor’s Opinion

honestly this whole problem mostly comes down to people not reading stop_reason before assuming the model messed up. once i started actually checking that field instead of just catching the parse exception, this stopped being mysterious pretty fast. structured outputs fixed the rest of it. wish id started there instead of spending a week tuning max_tokens for a problem that wasnt really about max_tokens at all.