fix(harness): recognize serde image rejection in image modality probe - #500
Open
michaelatamuk wants to merge 1 commit into
Open
Conversation
…ty probe** Text-only gateways reject the probe's image payload with a serde error (`"unknown variant `image_url`, expected `text`"`) that was not matched by the rejection patterns, so the verdict was never cached and the probe re-ran its failed call on every new agent. Add the pattern so the probe caches "image unsupported" after the first rejection.
|
head_sha: 变更摘要此变更修复了 主要改动
|
|
head_sha: 代码审查✅ 未发现问题 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paired: GitHub #500 ↔ GitCode !2307
What type of PR is this?
/kind bugfix
What does this PR do / why do we need it
This PR updates the image‑modality probe so that serde‑style deserialization errors (
unknown variant \image_url``) are correctly classified as image‑input unsupported.Without this pattern, text‑only gateways reject the probe payload but the failure is treated as inconclusive, causing the probe to re‑run on every new agent and waste extra API calls.
Problem
Non‑technical
Text‑only models reject the probe’s dummy image payload with a clear error, but the probe fails to recognize it.
Because the rejection is not classified as “image unsupported,” the probe never caches a verdict and fires again for every newly created agent.
This results in unnecessary repeated calls and noisy logs.
Technical
is_image_modality_rejection()matches against_IMAGE_INPUT_UNSUPPORTED_ERROR_PATTERNS.Serde‑style errors look like:
but this string was not included in the patterns.
The probe therefore treated the failure as inconclusive, skipped caching, and re‑ran the probe on every agent initialization.
Solution
Non‑technical
Add the
"unknown variant"pattern so the probe immediately recognizes the rejection as “image unsupported” and caches the verdict after the first failure.This eliminates redundant probe calls and correctly marks text‑only gateways as non‑image‑capable.
Technical
The fix adds
"unknown variant"to_IMAGE_INPUT_UNSUPPORTED_ERROR_PATTERNSinimage_modality_probe.py.Once matched, the probe caches
supported=Falseand will not re‑run for subsequent agents.Validation
python -m py_compile openjiuwen/harness/image_modality_probe.pypassesExpected Impact
Linked Closing Issues: