Branch-aware summary never applied at inlet for reasoning models (idless body ≠ DB branch via process_messages_with_output)

[Tuxie]

Branch-aware summary never applied at inlet for reasoning models (idless body ≠ DB branch via process_messages_with_output)

Summary

With branch-aware compression (v1.7.0+), a saved summary is never applied on the inlet for chats whose model is a reasoning model (assistant turns persisted with an output array plus a rendered <details type="reasoning">…</details> block in content). The outlet validates the snapshot against the DB branch and accepts it (so it stops rebuilding), but the inlet validates against the idless request body and always rejects it → the full uncompressed history is sent on every request, while the UI still reports a summary exists.

Environment

• Function: Async Context Compression v1.7.2 (commit 439b9f4)
• Open WebUI: 0.9.6
• DB: PostgreSQL
• Model: a reasoning model over an OpenAI-compatible endpoint with function_calling: native. Assistant turns are stored with an output array and a <details type="reasoning"> block inside content.

Symptom

• Every chat request sends the full history (verified via the upstream proxy's prompt_tokens), despite a valid chat_summary row covering the older turns.
• The summary row is correct and current-branch-anchored — replaying the stored covered refs against the live DB history nodes matches 100%.
• With debug_mode enabled, the inlet logs:

  [Summary Snapshot] Rejecting snapshot because current messages do not expose stable refs through covered prefix count=<N>.
  [Summary Snapshot] DB active branch fallback skipped: request body is not a compatible idless view (body_count=<N>, db_count=<N>).
  

  Note body_count == db_count, yet the idless view is judged incompatible.

Root cause

The request-body messages carry no Open WebUI ids, so _select_applicable_summary_snapshot → _current_branch_refs returns None and the code falls back to _compatible_db_branch_for_body_ref_fallback. That fallback fails for reasoning models:

• Open WebUI rebuilds each assistant message from its output array before sending, via process_messages_with_output() → convert_output_to_messages(output, ...), which skips reasoning by default (reasoning_format=None). So the model-visible assistant content is regenerated from output (reasoning excluded) and does not equal the stored content (which contains the <details type="reasoning"> fold).
• _body_message_matches_db_branch_message compares stored content (folded, with reasoning) against body content (output-derived, no reasoning). Even via the include_output=False path, the content differs, so the folded match fails.
• _unfold_db_branch_for_body_ref_fallback calls convert_output_to_messages(output, raw=True), but that does not reproduce the installed OWUI build's process_messages_with_output result (the raw flag / reasoning_format / message-collapsing differ), so the unfolded comparison also fails.

Net: neither the folded nor unfolded comparison matches → snapshot rejected on every inlet → full context sent. The outlet, which validates against DB-reconstructed messages, accepts the snapshot and skips rebuild, so the stale-but-valid summary persists unused.

Reproduction

1. Use a reasoning model with function_calling: native so assistant turns are stored with an output array and a <details type="reasoning"> block in content.
2. Let a long chat exceed the compression threshold so a summary is generated and saved.
3. Observe subsequent requests: the summary is never applied on the inlet (full history sent), while the outlet keeps the existing snapshot and skips rebuild.

Suggested fix

Reconcile the idless body↔DB comparison with how Open WebUI actually builds the model payload for reasoning models:

• Mirror process_messages_with_output() exactly in _unfold_db_branch_for_body_ref_fallback (use the same convert_output_to_messages arguments — raw and reasoning_format — as the installed OWUI version), or
• When the body is idless and 1:1 in count/role with the DB active branch, validate the snapshot against the DB branch (as the outlet does) and inject the summary by position, rather than requiring a content-level match of reasoning-stripped bodies.

Happy to provide more detail or test a patch.

[Fu-Jie]

Fixed — please help test 🙏

The issue is fixed on the fix/outlet-idless-summary-persist branch, shipped as v1.7.3 via PR #99 (not merged yet — waiting for community validation first).

Root cause

Reasoning models (DeepSeek-R1 / o1-like / Qwen3-thinking, etc.) store assistant content with a folded <details type="reasoning"> block in the DB, but OpenWebUI reconstructs the request body via process_messages_with_output → convert_output_to_messages(output, reasoning_format=...) before sending it to the model:

• OpenAI-compatible (reasoning_format=None): reasoning is stripped
• Ollama (think_tags): reasoning is re-tagged as <think>
• llama.cpp (reasoning_content): reasoning is routed to a separate reasoning_content field

As a result, body content ≠ DB content, and both existing alignment paths (folded / unfolded content match) fail. The inlet rejects the cached summary every turn and re-sends the full uncompressed history.

Fix — Path 3 (position-based fallback)

When Path 1 and Path 2 both fail, a third alignment path accepts the snapshot iff all of the following hold:

• body and DB branches have equal length
• all body messages are idless (the frontend sends plain chat-completion messages without ids)
• role / tool_calls / tool_call_id match position-by-position
• DB messages that carry an output array are exempted from content comparison (their content was genuinely reconstructed)
• DB messages without an output array still require exact content equality (fail-closed against edits)

Fail-closed preserved — the following cases are still rejected: edited body content when DB has no output, tampered tool_calls, mismatched tool_call_id, role drift, tool-call expansion (body count > DB count).

Verification (not a simple unit test)

A new test_issue98_e2e.py module inlines verbatim the real OpenWebUI main-branch functions (convert_output_to_messages, process_messages_with_output, reconcile_tool_pairs), constructs 4 real reasoning-model data flows (OpenAI-compatible / Ollama / llama.cpp / with tool_calls), rebuilds the body through the real OpenWebUI pipeline, then calls the plugin's real inlet() entry point. All 14 end-to-end tests pass.

How to test

1. Switch to the fix/outlet-idless-summary-persist branch, or in OpenWebUI replace the async_context_compression filter content with async_context_compression.py from that branch.
2. Have a long conversation with a reasoning model and trigger compression.
3. Observe subsequent turns: previously every turn re-sent the full history; now the cached summary should be reused.

Also fixed in this release

This version also fixes chat_summary table initialization failing on freshly deployed PostgreSQL (DuplicateTable: relation "ix_chat_summary_chat_id" already exists), which caused ⚠️ Summary generated but was not persisted. Root cause was a stale unique-index definition retained in SQLAlchemy metadata; the init now deduplicates it before CREATE TABLE.

---

Please help test — once confirmed I'll merge PR #99. Reply here if anything still looks off. Thanks!

[Tuxie]

Tested the fix/outlet-idless-summary-persist branch (v1.7.3) on a real reasoning-model chat — thanks for the fast turnaround. Unfortunately it doesn't resolve the inlet rejection in my setup. I think Path 3 has a guard that's too strict, plus a further issue underneath it.

Environment: OWUI 0.9.6, PostgreSQL, model = a reasoning model (mimo-v2.5-pro) over an OpenAI-compatible endpoint (LiteLLM), function_calling: native on the model. ~800-message chat; existing summary covers 693.

Result: every turn still sends the full uncompressed history. Inlet logs (debug_mode):

[Summary Snapshot] Rejecting snapshot because current messages do not expose stable refs through covered prefix count=693.
[Summary Snapshot] DB active branch fallback skipped: request body is not a compatible idless view (body_count=796, db_count=796).

The outlet, by contrast, accepts it: [Outlet] Reinjected summary placeholder at boundary=693. So the outlet (validating against DB messages, which carry ids → Path 1) accepts and skips rebuild, while the inlet (idless-body fallback) rejects — the summary sits valid-but-unused.

Issue 1 — Path 3's all-idless guard is too strict for reasoning chats. process_messages_with_output() rebuilds assistant-with-output messages via convert_output_to_messages() (idless), but user / system / no-output messages keep their DB node id — process_messages_with_output's clean_message = {k: v for k, v in message.items() if k != 'output'} only strips output, so id survives. The request body is therefore mixed-id, not all-idless. So:

and all(not self._get_message_id(m) for m in body_messages)

is False, and Path 3 bails before the position check. (Confirmed offline: ~half the body messages carry ids.)

Issue 2 — even with that guard removed, it still rejects. I commented out the all-idless condition; the inlet still rejects every turn (full context sent), so a per-position _body_position_matches_db_message is failing too. I couldn't isolate which position from logs (the failure is silent). For reference: the body is balanced (398 user / 398 assistant / 0 system, no tool calls, body_count == db_count), and an offline replay that rebuilds the body with the real process_messages_with_output(..., reasoning_format=None) and compares position-by-position shows 0 mismatches — so the live in-flight body diverges from that model in some way not visible in logs (the request body content isn't logged anywhere). One possibly-relevant detail: the request body reports function_calling=unset even though the model is configured native.

Happy to test the next iteration. I can also instrument _body_position_matches_db_message to log the exact failing index/field and report back if useful.

[Fu-Jie]

Thanks for the detailed report — both issues are real, fixed in the v1.7.3 branch

@Tuxie Your analysis was spot-on. The fix is folded into the same fix/outlet-idless-summary-persist branch (still v1.7.3, not a new version — the release isn't out yet). Please re-pull and retest when you get a chance.

Issue 1 — Path 3's all-idless guard was wrong ✅ fixed

You're correct: process_messages_with_output only strips output, not id. The real request body is mixed-id — user / system / no-output assistant messages keep their DB node id, only rebuilt assistant-with-output messages are idless. The initial Path 3 guard all(not _get_message_id(m) for m in body_messages) rejected every real body before the position check ran.

Fix: removed the guard entirely. Reaching Path 3 already requires _current_branch_refs(messages) is None upstream (the body as a whole exposes no usable ref sequence because the idless assistant-with-output messages break the ref chain), so the all-idless check was both wrong and redundant.

My e2e test was also at fault — it artificially stripped all ids via a _strip_ids helper, which masked this real-world shape. Fixed: _build_body_from_db now mirrors the real OpenWebUI pipeline exactly (no id stripping), producing genuinely mixed-id bodies. New regression tests test_openai_reasoning_body_is_mixed_id and test_mixed_id_body_path3_accepts lock this in.

Issue 2 — silent per-position rejection ✅ diagnostics added

You mentioned that even with the all-idless guard commented out, Path 3 still rejected every turn, and the failure was silent. Rather than guess what diverges in the live in-flight body (your offline replay showed 0 mismatches), I added diagnostic logging so we can see exactly what differs:

New: when Path 3 is eligible (equal length, unfoldable output) but a per-position check fails, debug_mode now logs:

[Summary Snapshot] Path 3 position fallback rejected at index=N: <reason>

where <reason> is one of:

• role mismatch (body=..., db=...)
• tool_calls mismatch
• tool_call_id mismatch (body=..., db=...)
• content mismatch on no-output message (body=..., db=...) (120-char preview)

This makes the previously-silent rejection observable without code instrumentation — exactly what you offered to do.

What I need from you

1. Re-pull the fix/outlet-idless-summary-persist branch (it's force-updated; the tip is now a single v1.7.3 commit e1a082b).
2. Enable debug_mode on the filter.
3. Trigger a turn on the same ~800-message reasoning chat.
4. If the summary is now reused — great, confirmed.
5. If it still rejects, look for the Path 3 position fallback rejected at index=N: <reason> line and paste it here. That will tell us exactly which position and field diverges in the live body, so we can fix the underlying mismatch (your function_calling=unset vs native observation is a strong lead — OpenWebUI may be transforming the body in a way neither of us has pinned down yet).

Why I didn't relax the content check blindly

_body_position_matches_db_message still requires exact content equality for DB messages WITHOUT an output array. This is the fail-closed guard against edited/tampered bodies. Your offline replay shows 0 mismatches for a faithful process_messages_with_output reconstruction, so if the live body is faithful, this check should pass. If it's failing, something else is transforming the body — and the new logging will tell us where. I'd rather see the data first than loosen a safety guard speculatively.

Thanks again for the high-quality report — the function_calling=unset detail and the offline-vs-live divergence observation were exactly the clues needed.

[Tuxie]

v1.7.4 (2066046) tested — Issue 1 confirmed fixed: Path 3 is now reached (the new diagnostic fires). It still rejects, and the new logging pinpoints it:

[Summary Snapshot] Path 3 position fallback rejected at index=126: role mismatch (body='assistant', db='user')
[Summary Snapshot] DB active branch fallback skipped: request body is not a compatible idless view (body_count=812, db_count=812)

So at index 126 the body and db_messages diverge by role (body=assistant, db=user), at equal total length — a sequence/branch misalignment, not content.

Offline cross-check (this localizes it): I rebuilt the body from the chat's currentId branch via the real process_messages_with_output(reasoning_format=None) and compared role-by-role to that same branch:

• 815 vs 815, 0 role divergences, strict assistant/user alternation end to end;
• 0 assistant turns whose output expands to ≠1 message (no tool-call expansion in this chat).

Key point: in that currentId walk, index 126 = assistant — which matches the live body (body='assistant'). The live db[126]='user' does not. So the request body tracks the currentId branch, while the filter's _load_full_chat_messages(chat_id) is off-by-one — it reconstructs a different/offset active branch (an extra or shifted message somewhere before 126, likely around a regenerated/edited node). That mismatch — not content, not reasoning — is what fails Path 3's position check.

So the fix likely belongs in _load_full_chat_messages / the DB active-branch reconstruction, so it yields the same sequence OWUI actually sends (the currentId branch). function_calling=unset (vs the model's native) is still present and may influence which branch/assembly path runs.

Happy to dump the role sequences from both _load_full_chat_messages and the live body around index 126 if you add a one-line log of each — that'll show exactly where the two walks split.

[Fu-Jie]

Update — another alignment gap fixed, please retest

@Tuxie Your index=126 diagnosis was the key clue. I traced it to a second alignment gap in _load_full_chat_messages and just pushed the fix to PR #99 (commit c3aefb2, still on fix/outlet-idless-summary-persist).

What was wrong

_load_full_chat_messages applied a _filter_model_visible_history_messages step that dropped assistant messages carrying an error field, on the assumption that OpenWebUI's middleware does the same. It does not — load_messages_from_db only strips each message to (role, content, output, files) and keeps every node on the parentId chain, failed or not:

return [{k: v for k, v in msg.items() if k in ("role", "content", "output", "files")} for msg in db_messages]

So whenever the active parentId chain contained a failed/regenerated assistant turn, the filter's DB walk silently dropped it while the live body kept it — every index after that point shifted. That is exactly the "off-by-one / different active branch" shape you described.

Fix

Removed the filter call in _load_full_chat_messages (and the now-dead _is_failed_assistant_message / _filter_model_visible_history_messages helpers). The DB walk now reproduces OpenWebUI's load_messages_from_db behaviour verbatim — same anchor, same parentId walk, no filtering. Two tests that codified the wrong behaviour were flipped to assert the kept-sequence contract.

Caveat — equal-length mismatch

Your log showed body_count=812 == db_count=812 (equal length) with a mid-sequence role swap. A pure drop would make db_count < body_count, so on a chat with no failed assistant on the active chain this fix is a no-op and the equal-length swap would persist. If that turns out to be your case, the remaining suspect is that the body and the DB walk are taking different parentId chains despite the same anchor — e.g. the anchor fix not taking effect, or a regenerated/edited fork where user_message_id and currentId land on different branches.

Please retest and share these console lines

The inlet now emits (DevTools console, via __event_call__):

[Inlet] 📍 DB walk anchor: user_message_id=<id>
[Inlet] 🧭 DB walk used user_message_id: anchor=<id> currentId=<id> (same branch as currentId | ⚠️ TRUE DIVERGENCE ...)
[Inlet] 📦 Summary snapshot: FOUND (will inject) | NONE (full context sent)
[Inlet] ⚠️ Path 3 rejected at index=<N>: <reason>     # only if still rejected

The 🧭 line is the discriminant:

• (same branch as currentId) + still rejected → the equal-length swap is real and lives elsewhere; I'll add a one-line role-sequence dump around the reject index so we can see exactly where the two walks split.
• ⚠️ TRUE DIVERGENCE → user_message_id is not on the currentId ancestor chain, which is the fork case and needs a different fix.

Pull c3aefb2, retry the failing turn, and paste those 3–4 lines. That will tell us whether to keep digging in the filter or look at branch selection upstream.