Skip to content

fix(compress): count tool definitions toward compression trigger - #441

Open
openjiuwen-sync-bot[bot] wants to merge 1 commit into
openJiuwen-ai:mainfrom
openjiuwenai:sync/pr-2265
Open

fix(compress): count tool definitions toward compression trigger#441
openjiuwen-sync-bot[bot] wants to merge 1 commit into
openJiuwen-ai:mainfrom
openjiuwenai:sync/pr-2265

Conversation

@openjiuwen-sync-bot

@openjiuwen-sync-bot openjiuwen-sync-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Paired: GitHub #441GitCode !2265

Summary

Fixes #​1506: context compression was triggered by comparing message tokens only against tokens_threshold, so a context whose tool definitions already consumed most of the model window could stay over the limit even after compression ran.

Change (openjiuwen/core/context_engine/processor/compressor/current_round_compressor.py)

trigger_add_messages now adds the tool-definition token count (context.get_tools()token_counter.count_tools(...)) to the running total before comparing against the threshold.

Both access points are guarded so existing behavior is unchanged where the pieces are absent:

  • getattr(context, "get_tools", None) — contexts that do not expose get_tools() (e.g. SessionModelContext) skip the tools term entirely (regression fixed after initial attempt crashed them with AttributeError).
  • isinstance(tools_tokens, int) — lightweight token counters that only implement count_messages return a non-int via mocks/fallbacks; those are ignored rather than polluting the total.

Tests (tests/unit_tests/core/context_engine/test_current_round_compressor.py)

Two new cases:

  • messages below threshold but messages+tools above → trigger_add_messages returns True (regression for #​1506)
  • no tools registered → decision unchanged, count_tools never invoked

Verification

  • Windows (Python 3.12 + uv): uv run pytest tests/unit_tests/core/context_engine/test_current_round_compressor.py10 passed (8 existing + 2 new)
  • Linux (Docker, python:3.12-slim + uv): same — 10 passed

Linked Closing Issues:

trigger_add_messages compared only message tokens against tokens_threshold,
so a context whose tool definitions already consumed most of the window
could stay over the model limit even after compression ran (#1506).

Add the tool-definition token count (context.get_tools() +
token_counter.count_tools) to the trigger decision. The accessor and the
counter result are guarded: contexts without get_tools() and counters
without count_tools() behave exactly as before.

Closes #1506.

Co-Authored-By: AtomCode (deepseek-v4-flash) <noreply@atomgit.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: b5d5d5115026fcbaeb62fb90e0a333730075b2ea

变更摘要

本次 PR 修复了 #​1506:CurrentRoundCompressor.trigger_add_messages 在判断是否触发压缩时,原先仅统计消息的 token 数,导致工具定义已占用大量上下文窗口的场景下压缩无法及时触发。此变更将工具定义的 token 数也纳入阈值比较,确保 tokens_threshold 能真实反映消息与工具的总开销。

主要改动

  • 工具定义 token 计入压缩触发阈值:在 trigger_add_messages 中,通过 context.get_tools() 获取工具列表并调用 token_counter.count_tools() 计算其 token 数,累加到已有的消息 token 总数后再与 _token_threshold 比较。
  • 安全访问 get_tools:使用 getattr(context, "get_tools", None) 获取工具访问方法,对于不暴露 get_tools() 的上下文(如 SessionModelContext),静默跳过工具计数,避免 AttributeError
  • 工具计数器返回值类型校验:通过 isinstance(tools_tokens, int) 检查 count_tools 返回值,仅当返回整数时才累加,防止轻量级 token 计数器(仅实现 count_messages)的 mock/fallback 返回值污染总 token 数。
  • 新增回归测试用例test_tools_overhead_counts_toward_trigger_threshold 验证消息低于阈值但消息+工具超过阈值时触发压缩;test_tools_overhead_without_tools_keeps_old_behavior 验证无工具时行为不变且 count_tools 不会被调用。

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: b5d5d5115026fcbaeb62fb90e0a333730075b2ea

代码审查

✅ 未发现问题

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: b5d5d5115026fcbaeb62fb90e0a333730075b2ea

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: b5d5d5115026fcbaeb62fb90e0a333730075b2ea

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants