This repository was archived by the owner on May 20, 2026. It is now read-only.
fix: rotate prompt_cache_key per call and treat 413 as retryable - #2
Open
Blue-B wants to merge 1 commit into
Open
fix: rotate prompt_cache_key per call and treat 413 as retryable#2Blue-B wants to merge 1 commit into
Blue-B wants to merge 1 commit into
Conversation
Calling codex_generate_image repeatedly in the same Pi session
eventually fails with '413 Request exceeds the maximum size' because:
1. prompt_cache_key is pinned to sessionId, so the Codex Responses
backend keeps replaying accumulated cached prefix for that key
across calls. image_generation calls don't benefit from prefix
caching (each call produces a fresh image) but they inherit the
accumulation.
2. 413 is not in isRetryableStatus, so the failure is surfaced
immediately with no recovery.
3. Even with a retry, reusing the same prompt_cache_key would hit
the same accumulated state and 413 again.
Changes:
- Add 408/413/425 to isRetryableStatus and add 'request exceeds |
payload too large' to the message-pattern matcher.
- New makeRequestCacheKey(sessionId) returns
${sessionId}:${timestamp}:${random} so every image call uses
its own cache slot. sessionId is still part of the key, so any
future per-session bucketing on the backend keeps working.
- buildRequestBody accepts an optional cacheKeyOverride so the
retry loop can rotate the key after a 413
(${sessionId}:retry${attempt}) and rebuild the body inside
the loop.
- The existing exponential-backoff + jitter loop is reused
unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rotate
prompt_cache_keyper call and treat 413 as retryableFixes #1
Problem
Calling
codex_generate_imagerepeatedly in the same Pi session eventuallyfails with
413 Request exceeds the maximum size, because:prompt_cache_keyis pinned tosessionId, so the Codex backend keepsreplaying accumulated cached prefix for that key across calls.
isRetryableStatus, so the failure is surfaced immediatelywith no recovery.
prompt_cache_keywould hit the sameaccumulated state and 413 again.
For
image_generationcalls the prompt-cache hit rate is effectively zero(each call produces a new image), so pinning the cache key only inherits the
downside.
Change
isRetryableStatusand addrequest exceeds | payload too largeto the message-pattern matcher.makeRequestCacheKey(sessionId)returns${sessionId}:${timestamp}:${random}so every image call uses its own cacheslot.
sessionIdis still part of the key, so any future per-sessionbucketing on the backend side keeps working.
buildRequestBodyaccepts an optionalcacheKeyOverrideso the retry loopcan rotate the key after a 413 (
${sessionId}:retry${attempt}) and rebuildthe body inside the loop.
No new dependencies. No behavior change for non-413 paths beyond the cache-key
rotation (which is a no-op for cache-miss workloads like image generation).
Repro / verification
Before: 3–5 sequential
codex_generate_imagecalls in the same Pi sessionreliably produce
Codex image generation request failed (413): Request exceeds the maximum size.After: same workload completes successfully; on the (now rare) case a 413 still
arrives, the retry rotates the cache key and succeeds on attempt 2.
Notes
prompt_cache_keyis documented as advisory: the server uses it as a hintfor prefix caching, so rotating it per call is safe.
sessionIdas the cache key and only rotating on 413,but that means every fresh session pays the 413 + retry round-trip the first
time it hits the limit. Per-call rotation makes the failure mode disappear
entirely without sacrificing anything image_generation actually benefits from.