Skip to content

fix(harness): recognize serde image rejection in image modality probe - #500

Open
michaelatamuk wants to merge 1 commit into
openJiuwen-ai:developfrom
michaelatamuk:fix/image-modality-probe-unknown-variant
Open

fix(harness): recognize serde image rejection in image modality probe#500
michaelatamuk wants to merge 1 commit into
openJiuwen-ai:developfrom
michaelatamuk:fix/image-modality-probe-unknown-variant

Conversation

@michaelatamuk

@michaelatamuk michaelatamuk commented Aug 11, 2026

Copy link
Copy Markdown

Paired: GitHub #500GitCode !2307

What type of PR is this?
/kind bugfix


What does this PR do / why do we need it

This PR updates the image‑modality probe so that serde‑style deserialization errors (unknown variant \image_url``) are correctly classified as image‑input unsupported.
Without this pattern, text‑only gateways reject the probe payload but the failure is treated as inconclusive, causing the probe to re‑run on every new agent and waste extra API calls.


Problem

Non‑technical

Text‑only models reject the probe’s dummy image payload with a clear error, but the probe fails to recognize it.
Because the rejection is not classified as “image unsupported,” the probe never caches a verdict and fires again for every newly created agent.
This results in unnecessary repeated calls and noisy logs.

Technical

is_image_modality_rejection() matches against _IMAGE_INPUT_UNSUPPORTED_ERROR_PATTERNS.
Serde‑style errors look like:

unknown variant `image_url`, expected `text`

but this string was not included in the patterns.
The probe therefore treated the failure as inconclusive, skipped caching, and re‑ran the probe on every agent initialization.


Solution

Non‑technical

Add the "unknown variant" pattern so the probe immediately recognizes the rejection as “image unsupported” and caches the verdict after the first failure.
This eliminates redundant probe calls and correctly marks text‑only gateways as non‑image‑capable.

Technical

The fix adds "unknown variant" to _IMAGE_INPUT_UNSUPPORTED_ERROR_PATTERNS in image_modality_probe.py.
Once matched, the probe caches supported=False and will not re‑run for subsequent agents.


Validation


Expected Impact

  • Probe correctly detects text‑only models on the first attempt
  • Verdict is cached; no repeated probe calls
  • Reduced API usage and cleaner logs
  • Aligns behavior with other rejection patterns already handled by the probe

Linked Closing Issues:

…ty probe**

Text-only gateways reject the probe's image payload with a serde error (`"unknown variant `image_url`, expected `text`"`) that was not matched by the rejection patterns, so the verdict was never cached and the probe re-ran its failed call on every new agent. Add the pattern so the probe caches "image unsupported" after the first rejection.
@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

head_sha: dbb52559578b0d3fd7fb1d32f753152ab1f66943

变更摘要

此变更修复了 image_modality_probe.py 中图像模态探测器的错误识别逻辑:当纯文本模型以 serde 风格返回 unknown variant \image_url`反序列化错误时,探测器此前无法将其识别为"不支持图像输入",导致判定结果无法缓存、每次创建新 agent 时都会重复发起探测请求。通过在错误匹配模式中新增"unknown variant"字符串,使探测器能够正确捕获此类拒绝并缓存supported=False` 判定,从而消除冗余 API 调用和日志噪音。

主要改动

  • 新增 "unknown variant" 错误模式: 在 _IMAGE_INPUT_UNSUPPORTED_ERROR_PATTERNS 元组中添加 "unknown variant",使 is_image_modality_rejection() 能够匹配 serde 风格的反序列化错误(如 unknown variant \image_url`, expected `text``),将其正确归类为"不支持图像输入"。
  • 判定缓存行为修复: 由于错误模式补齐后匹配成功,探测器在首次失败后即可缓存 supported=False,避免对同一纯文本网关反复发起探测。

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

head_sha: dbb52559578b0d3fd7fb1d32f753152ab1f66943

代码审查

✅ 未发现问题

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: dbb52559578b0d3fd7fb1d32f753152ab1f66943

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Image-modality probe fails to detect text-only models and re- runs on every new agent

1 participant