Dashboards re-issue the same aggregation/histogram queries every N seconds,
and seq-db recomputes them from scratch on every refresh — the most
expensive read path (docs/en/11-benchmarks.md) with the most repetitive
workload. Caching the results seems natural (cf. Elasticsearch's shard
request cache), and we'd like to know where the maintainers see it living:
-
In seq-db (store level, per sealed fraction). Sealed fractions are
immutable, so a result fully covering a fraction can be cached without
TTL and stays exact; sliding windows reuse interior fractions. We built
a prototype this way: an opt-in aggs layer in the existing cache
subsystem, hit ≈ 2.7µs vs 14ms recompute on a 200k-doc fraction. Happy
to submit a PR if this direction is acceptable.
-
In seq-ui (Redis, TTL). Simpler, follows the existing
logs_lifespan/events pattern, but results go stale within the TTL,
a shifted time window is always a full miss, and other API consumers
don't benefit.
Is per-fraction caching in seq-db something you'd accept, or do you consider
result caching out of scope for the engine and better kept in seq-ui?
Dashboards re-issue the same aggregation/histogram queries every N seconds,
and seq-db recomputes them from scratch on every refresh — the most
expensive read path (
docs/en/11-benchmarks.md) with the most repetitiveworkload. Caching the results seems natural (cf. Elasticsearch's shard
request cache), and we'd like to know where the maintainers see it living:
In seq-db (store level, per sealed fraction). Sealed fractions are
immutable, so a result fully covering a fraction can be cached without
TTL and stays exact; sliding windows reuse interior fractions. We built
a prototype this way: an opt-in
aggslayer in the existing cachesubsystem, hit ≈ 2.7µs vs 14ms recompute on a 200k-doc fraction. Happy
to submit a PR if this direction is acceptable.
In seq-ui (Redis, TTL). Simpler, follows the existing
logs_lifespan/events pattern, but results go stale within the TTL,a shifted time window is always a full miss, and other API consumers
don't benefit.
Is per-fraction caching in seq-db something you'd accept, or do you consider
result caching out of scope for the engine and better kept in seq-ui?