Skip to content

feat(usage): per-model and per-day prefix-cache hit-rate observability#165

Open
OnlyTerp wants to merge 3 commits into
sybil-solutions:mainfrom
OnlyTerp:terp/wp-016-cache-metrics
Open

feat(usage): per-model and per-day prefix-cache hit-rate observability#165
OnlyTerp wants to merge 3 commits into
sybil-solutions:mainfrom
OnlyTerp:terp/wp-016-cache-metrics

Conversation

@OnlyTerp

Copy link
Copy Markdown

Summary

Surface prefix-cache hit-rate in the usage dashboard, per model and per day. The aggregate already records cache_read/cache_write tokens; this exposes them in the breakdowns and UI.

Why (local-model)

Lets you see whether local prefix caching is actually helping — e.g. after the tool-order normalization PR — and how the hit rate trends over time. Inference-efficiency signal, not API cost.

Changes

  • controller/src/stores/inference-request-store.ts: add cache_read/cache_write/cache_hit_rate to the by_model, daily, and daily_by_model breakdowns; extract a shared calcCacheHitRate (also used by the global hit-rate).
  • controller/src/modules/system/usage/pi-sessions.ts: per-model / per-day cache accumulation in the parallel Pi-sessions aggregator.
  • shared/contracts/usage.ts + frontend normalizer: additive cache fields (default 0).
  • UI: a sortable "Cache hit %" column in the per-model table, plus a per-day readout and an "avg cache hit" summary stat in the daily chart.

Verification

  • cd controller && bun test src/stores/inference-request-store.test.ts → pass; bun run typecheck → clean.
  • frontend typecheck + lint + build → clean.

Non-goals

  • Additive only; no contract removals. Frontend normalizer defaults missing fields to 0 for older payloads.

terp and others added 3 commits June 23, 2026 00:40
Add per-day cache hit-rate readout below each bar in the daily usage
chart (when data is present), and an average cache hit-rate summary
stat alongside the existing total tokens/requests/peak day row.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@OnlyTerp OnlyTerp requested a review from 0xSero as a code owner June 23, 2026 04:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant