fix+feat: v0.37.0 — recovery completeness, enumeration truth, provider compat, engine correctness, de-ossification

[hxddh]

What changed

The v0.37.0 batch from the four-angle mining round (persistence, S3/provider, agent de-ossification, engine math). Sixteen items; every security-floor bound untouched, one tightened.

Bugs fixed (Tier 1)

• Evidence import crash-wedge. A hard kill mid-download left the row importing forever (no path back to confirmed/planned). Startup reconciliation now fails orphaned imports + their files, mirroring the existing run reconciler.
• list_objects unreachable keys. The 500-key echo cap sat below the 1000-key S3 page while next_token advanced past the whole page — keys 501–1000 were silently lost with no way to page back. Echo cap now equals the page cap; the elastic tool-output budget still backstops it.
• Silent final-answer truncation. The answer was hard-sliced at 48 000 chars with no marker (the real slice lived in parse_agent_contract, one layer below the visible one). The cap is now model-elastic (≥ 4 chars/token of the completion budget — it can never cut an answer the model was allowed to emit) and appends an explicit [TRUNCATED …] marker when hit.
• test_conditional_get false "changed". Providers that ignore If-None-Match return 200 with the same ETag; the tool reported that as a change. It now compares quote-normalized ETags: equal on 200 → "unchanged + conditional unsupported" (rule 18).
• list_multipart_uploads on prefix-scoped providers. No prefix argument meant the scope check always denied it. Now takes prefix, threaded to both the scope check and S3 Prefix=.
• get_object_lock_status over-broad "none". InvalidRequest maps to "no lock" only in its object-lock message flavor; other flavors surface as errors instead of reading "cleanly deletable".
• Inventory engine: storage-class-skew finding suppressed when the column is absent (null-group guard); average_object_size uses floor division (int64 precision above 2^53).
• DuckDB layer: read-only open of a missing DB is a clean error and no longer creates a stray empty .duckdb; writer lock contention gets the same retryable message readers had.
• session_datasets NULL-filename dedupe (IS, not =).

Security

• Bare AK/SK pair pastes fully scrubbed. A bare 40-char secret key pasted with its AKIA… id survived label-anchored redaction. New narrow rule: bare 40-char base64 tokens are masked only when an AWS key-id shape co-occurs in the same text — ordinary 40-char strings without that hint are untouched (regression-tested both ways).
• session_messages JSON columns (tool_activity/grounding/proposed_actions) now pass through redact() at the persistence boundary like every sibling repo (rule-14 defense in depth).

De-ossification (no security-floor change)

• Completion budget bounded only by the per-model provider max-output — the module-wide 32k ceiling removed (it only ever clamped models with higher real caps: claude-3-7/gemini-2.5 64k, o-series 100k).
• Survey/config-review summary echo scales with the model window (floor 2000, ceiling 16k).
• Deterministic session summary: persisted store holds 200 facts/findings (was 50), context echo model-elastic (floor 50), human digest stays at 50 with an explicit "+N more" note.
• KiB/MiB/GiB labels for the 1024-based size math (labels said KB/MB while dividing by 1024).
• Docs note: gated 1M context windows (e.g. Claude beta) go through the explicit context_window override.

How to run it

cd sidecar && pytest -q
ruff check app

Checks run

• pytest -q — 634 passed (20 new in tests/test_v0370_fixes.py).
• ruff check app — clean.
• Existing tests updated only where behavior intentionally changed: full-page echo, provider-cap-only completion budget, object-lock message-flavor, stream_events_for signature, gemini-2.5 budget, KiB bucket labels, drift-sync now >=.
• Frontend untouched.

Known gaps

• parallel_tool_calls=False stays global (real DeepSeek workaround; provider-conditionalizing deferred).
• The Claude window table stays at the 200k GA default by design — the 1M window is a gated opt-in, covered by the explicit context_window override (documented).

🤖 Generated with Claude Code

https://claude.ai/code/session_01M3YojFPzjkfMj6YYSwvzdx

---

Generated by Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d430fa2047

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Scope bare-secret redaction to the credential pair

When one message or log contains an AWS access-key ID plus any unrelated 40-character hex/base64 value—such as a SHA-1 checksum or object identifier—paired_paste causes every such value in the entire string to be replaced. This irreversibly removes legitimate diagnostic evidence from persisted messages and reports; constrain the bare-secret match to a nearby or structurally paired value instead of globally substituting all 40-character tokens.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Avoid treating four characters as a hard token maximum

For completions whose tokenizer averages more than four characters per token—common with repetitive object-key prefixes, code, URLs, or whitespace—the model can legally emit an answer longer than this computed cap. _finalize_contract then truncates that valid answer despite the stated guarantee, so a requested full enumeration can still lose its tail; use a conservative tokenizer/provider-specific upper bound rather than the four-character heuristic.

Useful? React with 👍 / 👎.