Skip to content

fix: guard against empty choices and message=None across all model backends#48

Open
qizwiz wants to merge 1 commit into
velocitybolt:masterfrom
qizwiz:fix/guard-llm-response-choices
Open

fix: guard against empty choices and message=None across all model backends#48
qizwiz wants to merge 1 commit into
velocitybolt:masterfrom
qizwiz:fix/guard-llm-response-choices

Conversation

@qizwiz

@qizwiz qizwiz commented May 18, 2026

Copy link
Copy Markdown

What

Add explicit guards in all four model backend classes before accessing response.choices[0].message.content.

Why

client.chat.completions.create() can return two empty-response shapes, neither currently handled:

  1. choices = [] — on content-policy rejections, rate-limit errors, or provider failures
  2. choices[0].message = None — e.g. Gemini 2.5 Flash via the OpenAI-compatible endpoint returns HTTP 200 with finish_reason: PROHIBITED_CONTENT and message=None

Both crash with IndexError or AttributeError. All four model backends have identical unguarded access patterns.

Files changed

File Fix
common/models/openai_model.py Guard before choices[0].message.content
common/models/azure_model.py Guard before choices[0].message.content
common/models/cerebras_model.py Guard before choices[0].message.content
common/models/groq_model.py Guard before choices[0].message.content
# Before
return response.choices[0].message.content

# After
if not response.choices or response.choices[0].message is None:
    raise ValueError("LLM returned empty or filtered response")
return response.choices[0].message.content

Corpus context

Detected by pact (llm_response_unguarded mode), a Z3-verified static analyzer for LLM crash vectors. This pattern was found across 13.7k violations in 786 repos.

…ckends

All four model backends (OpenAI, Azure, Cerebras, Groq) access
response.choices[0].message.content without checking whether choices
is non-empty or message is non-None.

The API can return choices=[] on content-policy rejections or provider
errors, and choices[0].message=None (e.g. Gemini PROHIBITED_CONTENT via
OpenAI-compatible endpoint, HTTP 200 with finish_reason=PROHIBITED_CONTENT).
Both crash with IndexError or AttributeError. Explicit guards raise
ValueError instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant