fix(async-context-compression): reuse summaries for idless requests#95
Conversation
Match idless request bodies against the persisted DB branch before rejecting snapshot reuse. This covers unfolded tool output, failed assistant filtering, and the terminal assistant placeholder inserted before middleware rebuilds the model-visible user-tip chain.
Address review findings for DB-backed idless request matching by making debug logging safe when body refs are unavailable and failing closed when folded output conversion raises. Add regression coverage for terminal/idless debug selection, output-omission matching, direct failed-assistant filtering, conversion failures, and trimmed assistant matching.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the Async Context Compression filter to version 1.7.1, introducing summary reuse for idless requests by matching them against the persisted DB active branch, handling terminal assistant placeholders, and adding a configurable timeout for summary LLM requests. Feedback on the changes includes opportunities to optimize performance by filtering messages before deep-copying and lifting imports out of loops, as well as a style guide violation regarding the casing of the new summary_llm_timeout_seconds valve field.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| direct_messages = chat_payload.get("messages") | ||
| if isinstance(direct_messages, list) and direct_messages: | ||
| return deepcopy(direct_messages) | ||
| return self._filter_model_visible_history_messages( | ||
| deepcopy(direct_messages) | ||
| ) |
There was a problem hiding this comment.
在对 direct_messages 进行过滤之前先执行了 deepcopy(direct_messages)。如果对话历史非常长，且其中包含已被过滤掉的失败 assistant 消息，这会导致不必要的深拷贝开销。
建议先进行过滤，然后再对过滤后的消息列表进行 deepcopy，以提高内存和时间效率。
direct_messages = chat_payload.get("messages")
if isinstance(direct_messages, list) and direct_messages:
filtered = self._filter_model_visible_history_messages(direct_messages)
return deepcopy(filtered)| def _unfold_db_branch_for_body_ref_fallback( | ||
| self, | ||
| db_messages: List[Dict[str, Any]], | ||
| ) -> tuple[List[Dict[str, Any]], List[int]]: | ||
| """Unfold DB messages and map folded DB boundaries to unfolded indices.""" | ||
| unfolded_messages: List[Dict[str, Any]] = [] | ||
| db_to_body_boundaries = [0] | ||
|
|
||
| for message in db_messages: | ||
| before_count = len(unfolded_messages) | ||
|
|
||
| if ( | ||
| isinstance(message, dict) | ||
| and message.get("role") == "assistant" | ||
| and isinstance(message.get("output"), list) | ||
| and message.get("output") | ||
| ): | ||
| expanded_messages = [] | ||
| try: | ||
| from open_webui.utils.misc import convert_output_to_messages | ||
|
|
||
| expanded_messages = convert_output_to_messages( | ||
| message["output"], raw=True | ||
| ) | ||
| except ImportError: | ||
| expanded_messages = [] | ||
| except Exception as exc: | ||
| logger.debug( | ||
| "[Summary Snapshot] Failed to unfold DB assistant output " | ||
| "for idless body fallback; rejecting DB ref fallback: %s", | ||
| exc, | ||
| ) | ||
| return [], [] |
There was a problem hiding this comment.
在循环内部进行 from open_webui.utils.misc import convert_output_to_messages 导入操作，在处理包含大量 assistant 消息的长对话历史时，会导致重复的导入开销。
建议将该导入操作提升到函数 _unfold_db_branch_for_body_ref_fallback 的开头，仅执行一次，并在循环中直接使用。这样既能保持优雅的异常处理，又能提升执行效率。
def _unfold_db_branch_for_body_ref_fallback(
self,
db_messages: List[Dict[str, Any]],
) -> tuple[List[Dict[str, Any]], List[int]]:
"""Unfold DB messages and map folded DB boundaries to unfolded indices."""
try:
from open_webui.utils.misc import convert_output_to_messages
except ImportError:
convert_output_to_messages = None
unfolded_messages: List[Dict[str, Any]] = []
db_to_body_boundaries = [0]
for message in db_messages:
before_count = len(unfolded_messages)
if (
isinstance(message, dict)
and message.get("role") == "assistant"
and isinstance(message.get("output"), list)
and message.get("output")
):
expanded_messages = []
if convert_output_to_messages is not None:
try:
expanded_messages = convert_output_to_messages(
message["output"], raw=True
)
except Exception as exc:
logger.debug(
"[Summary Snapshot] Failed to unfold DB assistant output "
"for idless body fallback; rejecting DB ref fallback: %s",
exc,
)
return [], []| summary_llm_timeout_seconds: float = Field( | ||
| default=180.0, | ||
| ge=0, | ||
| description="Maximum seconds to wait for the summary LLM request. Set to 0 to disable the timeout.", | ||
| ) |
There was a problem hiding this comment.
根据项目样式指南（Repository Style Guide），Valves 的字段应使用 UPPER_SNAKE_CASE 命名。
新引入的 summary_llm_timeout_seconds 字段目前使用的是 lower_snake_case。虽然这与类中现有的其他历史字段（如 summary_model）保持了文件内的一致性，但它违反了样式指南的规定。
建议在后续重构中，统一将 Valves 类中的所有字段转换为 UPPER_SNAKE_CASE，以完全符合项目规范。
References
- Required patterns: Valves(BaseModel) with UPPER_SNAKE_CASE fields (link)
2 participants
Summary
Fix async context compression falling back to raw long-history injection when Open WebUI sends a model request
body that no longer matches the persisted DB history shape exactly.
Branch-aware summary reuse depends on message IDs and fingerprints to prove a stored summary is safe for the
current branch. In practice, Open WebUI can send request messages without IDs, unfold assistant
outputintomodel-visible assistant/tool messages, filter failed assistant turns, and insert a new terminal assistant
This change lets idless requests use the DB active branch as proof. If the request body aligns with the
persisted branch after applying the same model-visible transformations, the existing summary is reused. If
alignment fails, the summary is still rejected.
Example
Before the fix, Open WebUI already had a compressed summary for a long chat, but the model request
Example DB history:
msg_1 user: Help me debug this error
msg_2 assistant: I’ll inspect the code first
msg_3 assistant:
msg_4 assistant: The issue is config option A
The summary safely covered up to msg_4, so the model should receive:
summary: Earlier, the user debugged an error and the assistant found config option A...
tail messages: only the recent messages after the summary
But Open WebUI might send the actual model request like this:
user: Help me debug this error
assistant: I’ll inspect the code first
assistant: I need to call a tool
tool: tool result...
assistant: The issue is config option A
assistant:
That differs because:
The request messages may not include msg_1, msg_2, etc.
Tool output stored in the DB as one folded assistant message may be unfolded into assistant + tool +
assistant.
Failed assistant turns may be filtered out before the model request.
The request may end with an empty assistant placeholder so the model can continue.
Those differences do not mean the chat history changed. They are just how Open WebUI prepares messages for
the model.
The old matching logic introduced by v1.7.0 was too strict, so it said:
The request messages do not exactly match the DB branch.
I cannot reuse the summary.
Then it fell back to:
Send the entire long raw history to the model
The fix changes the validation to say:
Even if the request messages have no IDs,
if they line up with the DB active branch after applying Open WebUI’s model-visible transformations,
then this is still the same chat branch.
So the system can safely send:
summary: compressed earlier context
tail messages: only the recent messages after the summary boundary
If the request really does not match the DB active branch, the summary is still rejected. So it stays
fail-closed: no risky reuse of a stale or wrong summary.
What Changed
tip branch only when the body proves it matches.
outputto unfolded model-visible body messages so DB coverage boundaries translateto the correct body tail index.
or wrong summary reuse.
Test Plan
pytest -q test_async_context_compression.py3c7046e3-1082-4f25-8965-4652acf1994a: the plugin selected the DB-backedsummary and sent ~24K tokens instead of falling back to the ~300K token raw history path.
摘要：让无 ID 请求也能复用摘要
修复 async context compression 在 Open WebUI 发出的模型请求 body 与数据库历史形态不完全一致时，误判已有 summary
不安全，从而退回注入原始长历史的问题。
branch-aware summary 复用依赖 message id 和 fingerprint 来证明某个已保存的 summary 仍然适用于当前分支。但 Open
WebUI 实际发给模型的 request body 有时不是数据库里的原始消息形态：消息可能没有 id，assistant 的
output可能已展开为多条模型可见消息，middleware 会过滤失败的 assistant 消息，并且请求开始前 DB 里可能已经插入了一个“正在生成
中”的 terminal assistant 占位消息。
这个改动让 idless request 可以通过 DB active branch 做严格对齐确认。对齐成功就复用已有 summary；对齐失败仍然拒
绝复用，避免错误使用过期或不属于当前分支的摘要。
举例说明
比如数据库里保存的是：
msg_1 user: 帮我看这个报错
msg_2 assistant: 我先检查代码
msg_3 assistant: <工具调用和工具结果被折叠在这条 assistant 里>
msg_4 assistant: 问题在配置项 A
摘要已经覆盖到了 msg_4，正常应该发给模型：
summary: 前面用户在排查配置项 A 的报错，助手已经定位到……
tail messages: 最近几条新消息
但 Open WebUI 真正发模型请求时，可能变成：
user: 帮我看这个报错
assistant: 我先检查代码
assistant: 我要调用工具
tool: 工具结果……
assistant: 问题在配置项 A
assistant:
这里有几个差异：
这些差异不是“聊天内容变了”，只是 Open WebUI 在发给模型前把消息整理成了模型可见格式。
v1.7.0 的摘要匹配逻辑太依赖 ID 和严格匹配，所以会说：
请求里的 messages 和数据库里的 branch 对不上。
不能复用 summary。
然后退回到：
把完整的几十万 token 历史全部发给模型
这就是之前的问题。
这次修复后的逻辑是：
虽然请求 messages 没有 ID，
但如果它们按 Open WebUI 的模型可见规则处理后，
能和数据库里的 active branch 对齐，
那就说明还是同一条聊天分支。
于是可以安全复用摘要：
summary: 已压缩的前文
tail messages: 摘要之后真正需要保留的最近消息
但如果请求内容真的对不上，比如 body 里少了一段关键用户消息、换了分支、消息内容和数据库 active branch 不一
致，那还是会拒绝 summary，不会乱用旧摘要。
改动内容
尝试去掉最后这个占位消息匹配。
output到 unfolded model-visible body 的边界映射，确保 DB 覆盖到第 N 条消息时，能正确换算 body 里 tail 应该从哪里开始保留。
error的 failed assistant，贴近 Open WebUI middleware 实际发给模型的消息。测试
pytest -q test_async_context_compression.py3c7046e3-1082-4f25-8965-4652acf1994a验证：插件成功选中 DB-backed summary，发送上下文约24K tokens，没有再退回约 300K tokens 的原始历史路径。