Skip to content

feat: Session and Scan wrappers for cache - #489

Draft
dkharms wants to merge 6 commits into
mainfrom
0-scoped-cache
Draft

feat: Session and Scan wrappers for cache#489
dkharms wants to merge 6 commits into
mainfrom
0-scoped-cache

Conversation

@dkharms

@dkharms dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member

Description

In this PR I've introduced interface cache.Wrapper into cache package.

This interface allows us to solve several problems by providing the most reasonable implementations at the call-sites. The hottest problems are:

  • Too high cache hit-rate (it is very noticeable with lids cache layer) because we receive objects one-at-a-time (from the one block) and therefore generate a lot of cache hits;
  • Compaction used cache as well and our current cache implementation is not tolerant to scan-and-forget workloads (what compaction basically is) therefore search queries suffered;

It was a little bit tricky to collect sound measurements because compaction is using cache.Scan wrapper and the overall RAM usage dropped and therefore GC was more sensitive to RAM usage spikes (example, GC-rate) that's why I've set GOGC=500 and GOMEMLIMIT=4GiB.

So I guess the most important metrics are:

  • Cache hit-rate is higher in 0-scoped-cache branch because compaction does not evict cached blocks and search latencies are better across all percentiles;
  • CPU usage is a little bit lower (well, because we do not waste CPU cycles on block unpacking);
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
base comp diff base comp diff base comp diff base comp diff base comp diff base comp diff base comp diff
bulk
hot 25.84 25.81 -0.15% 5.32 5.20 -2.37% 25 25 0.00% 38 36 -5.26% 59 53 -10.17% 32329 32328 -0.00% 0 0 0.00%
service:payment-backend-eu
AND k8s_namespace:prod
AND level:[0 to 3]
AND (
    message:'failed'
    OR message:'timeout'
)
hot 43.38 41.80 -3.65% 5.01 5.16 +2.86% 43.28 41.55 -4.00% 56.59 54.35 -3.96% 75.87 70.06 -7.66% 4785 4785 0.00% 0 0 0.00%

  • I have read and followed all requirements in CONTRIBUTING.md;
  • I used LLM/AI assistance to make this pull request;

If you have used LLM/AI assistance please provide model name and full prompt:

Model: {{model-name}}
Prompt: {{prompt}}

@codecov-commenter

codecov-commenter commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.14062% with 79 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.24%. Comparing base (d872202) to head (9262aea).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
frac/sealed/seqids/loader.go 64.86% 7 Missing and 6 partials ⚠️
storage/index_reader.go 60.00% 8 Missing and 4 partials ⚠️
frac/sealed_loader.go 16.66% 8 Missing and 2 partials ⚠️
frac/sealed/token/block_loader.go 33.33% 6 Missing and 2 partials ⚠️
cache/wrapper.go 78.78% 4 Missing and 3 partials ⚠️
frac/sealed/token/table_loader.go 65.00% 3 Missing and 4 partials ⚠️
frac/remote.go 75.00% 6 Missing ⚠️
frac/sealed.go 77.77% 6 Missing ⚠️
frac/sealed/lids/loader.go 71.42% 1 Missing and 1 partial ⚠️
frac/sealed_source.go 93.10% 1 Missing and 1 partial ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #489      +/-   ##
==========================================
+ Coverage   71.10%   71.24%   +0.14%     
==========================================
  Files         232      234       +2     
  Lines       18572    19026     +454     
==========================================
+ Hits        13205    13556     +351     
- Misses       4374     4437      +63     
- Partials      993     1033      +40     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dkharms

dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start bulk main --duration=10m

@seqbenchbot

seqbenchbot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark 1b350ba7.

Show details
  • scenario bulk;
  • against main;
  • running for 10m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 1b350ba7.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark 1b350ba7 (scenario bulk) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 26.33 5.62 26 41 59 21585 0

Compare it against another run with @seqbenchbot compare 1b350ba7 <other-id>.

@dkharms

dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start bulk 0-scoped-cache --duration=10m

@seqbenchbot

seqbenchbot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark 90121b2e.

Show details
  • scenario bulk;
  • against 0-scoped-cache;
  • running for 10m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 90121b2e.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark 90121b2e (scenario bulk) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 29.21 7.17 29 56 71 21585 0

Compare it against another run with @seqbenchbot compare 90121b2e <other-id>.

@dkharms

dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start mixed main --duration=15m

@seqbenchbot

seqbenchbot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark 2636018c.

Show details
  • scenario mixed;
  • against main;
  • running for 15m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 2636018c.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark 2636018c (scenario mixed) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 27.02 5.89 26 42 63 32328 0
service:payment-backend-eu
AND k8s_namespace:prod
AND level:[0 to 3]
AND (
    message:'failed'
    OR message:'timeout'
)
hot 43.80 5.37 43.56 60.07 77.17 4785 0

Compare it against another run with @seqbenchbot compare 2636018c <other-id>.

@dkharms

dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start mixed 0-scoped-cache --duration=15m

@seqbenchbot

seqbenchbot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark cab2b4e0.

Show details
  • scenario mixed;
  • against 0-scoped-cache;
  • running for 15m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop cab2b4e0.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark cab2b4e0 (scenario mixed) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 29.21 6.89 29 53 72 32328 0
service:payment-backend-eu
AND k8s_namespace:prod
AND level:[0 to 3]
AND (
    message:'failed'
    OR message:'timeout'
)
hot 44.70 6.47 44.41 69.55 87.65 4785 0

Compare it against another run with @seqbenchbot compare cab2b4e0 <other-id>.

@dkharms

dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start mixed 0-scoped-cache --duration=15m

@seqbenchbot

seqbenchbot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark 785f5d30.

Show details
  • scenario mixed;
  • against 0-scoped-cache;
  • running for 15m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 785f5d30.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
Indexer-4 a73114 107efc
678752766.00 B/op 766786365.00 B/op 1.13 🔴

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark 785f5d30 (scenario mixed) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 25.81 5.20 25 36 53 32328 0
service:payment-backend-eu
AND k8s_namespace:prod
AND level:[0 to 3]
AND (
    message:'failed'
    OR message:'timeout'
)
hot 41.80 5.16 41.55 54.35 70.06 4785 0

Compare it against another run with @seqbenchbot compare 785f5d30 <other-id>.

@dkharms

dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start mixed main --duration=15m

@seqbenchbot

seqbenchbot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark f0f3659f.

Show details
  • scenario mixed;
  • against main;
  • running for 15m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop f0f3659f.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark f0f3659f (scenario mixed) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 25.84 5.32 25 38 59 32329 0
service:payment-backend-eu
AND k8s_namespace:prod
AND level:[0 to 3]
AND (
    message:'failed'
    OR message:'timeout'
)
hot 43.38 5.01 43.28 56.59 75.87 4785 0

Compare it against another run with @seqbenchbot compare f0f3659f <other-id>.

@dkharms

dkharms commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot compare f0f3659f 785f5d30

@seqbenchbot

seqbenchbot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Comparison of baseline f0f3659f against comparison 785f5d30:

Show comparison
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
base comp diff base comp diff base comp diff base comp diff base comp diff base comp diff base comp diff
bulk
hot 25.84 25.81 -0.15% 5.32 5.20 -2.37% 25 25 0.00% 38 36 -5.26% 59 53 -10.17% 32329 32328 -0.00% 0 0 0.00%
service:payment-backend-eu
AND k8s_namespace:prod
AND level:[0 to 3]
AND (
    message:'failed'
    OR message:'timeout'
)
hot 43.38 41.80 -3.65% 5.01 5.16 +2.86% 43.28 41.55 -4.00% 56.59 54.35 -3.96% 75.87 70.06 -7.66% 4785 4785 0.00% 0 0 0.00%

See both runs overlaid (baseline aligned to comparison) on the Grafana dashboard.

Have a great time!

@dkharms dkharms added the performance Features or improvements that positively affect seq-db performance label Aug 11, 2026
@cheb0

cheb0 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

@seqbenchbot start search-regular main 'message:*' --duration 3m

@seqbenchbot

seqbenchbot commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Nice, @cheb0 <(-^,^-)=b!

Started benchmark dfa15547.

Show details
  • scenario search-regular;
  • against main;
  • running for 3m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop dfa15547.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @cheb0 <(-^,^-)=b!

Benchmark dfa15547 (scenario search-regular) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
message:*
hot 6495.15 186.22 6474.38 7073.97 7787.12 167 10063671

Compare it against another run with @seqbenchbot compare dfa15547 <other-id>.

@cheb0

cheb0 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

@seqbenchbot start search-regular 0-scoped-cache 'message:*' --duration 3m

@seqbenchbot

seqbenchbot commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Nice, @cheb0 <(-^,^-)=b!

Started benchmark 5d4219b9.

Show details
  • scenario search-regular;
  • against 0-scoped-cache;
  • running for 3m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 5d4219b9.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @cheb0 <(-^,^-)=b!

Benchmark 5d4219b9 (scenario search-regular) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
message:*
hot 6617.52 227.34 6581.83 7031.41 7626.72 163 10063717

Compare it against another run with @seqbenchbot compare 5d4219b9 <other-id>.

@dkharms

dkharms commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start bulk 0-scoped-cache --duration=10m

@seqbenchbot

seqbenchbot commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark 8ac09a89.

Show details
  • scenario bulk;
  • against 0-scoped-cache;
  • running for 10m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 8ac09a89.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
Sealing_WithSort-4 a73114 b9c254
40324408.00 B/op 72327872.00 B/op 1.79 🔴
5389.00 allocs/op 2005403.00 allocs/op 372.13 🔴

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark 8ac09a89 (scenario bulk) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 29.27 7.36 29 54 71 21585 0

Compare it against another run with @seqbenchbot compare 8ac09a89 <other-id>.

@dkharms

dkharms commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start bulk main --duration=10m

@seqbenchbot

seqbenchbot commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark debb7b21.

Show details
  • scenario bulk;
  • against main;
  • running for 10m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop debb7b21.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark debb7b21 (scenario bulk) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 26.80 5.98 26 42 61 21585 0

Compare it against another run with @seqbenchbot compare debb7b21 <other-id>.

@dkharms

dkharms commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start bulk 0-scoped-cache --duration=10m

@seqbenchbot

seqbenchbot commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark e364a3c5.

Show details
  • scenario bulk;
  • against 0-scoped-cache;
  • running for 10m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop e364a3c5.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
Sealing_WithSort-4 a73114 c9d52c
40324408.00 B/op 72324152.00 B/op 1.79 🔴
5389.00 allocs/op 2005387.00 allocs/op 372.13 🔴

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark e364a3c5 (scenario bulk) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 25.64 5.27 25 37 54 21585 0

Compare it against another run with @seqbenchbot compare e364a3c5 <other-id>.

@dkharms

dkharms commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@seqbenchbot start bulk 0-scoped-cache --duration=10m

@seqbenchbot

seqbenchbot commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Started benchmark 189238e7.

Show details
  • scenario bulk;
  • against 0-scoped-cache;
  • running for 10m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 189238e7.

Here is a list of helpful links:

  • Take a look at Grafana dashboard;
  • Live-tailing logs are also available;

Have a great time!

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
Sealing_WithSort-4 a73114 04c4f4
40324408.00 B/op 73381472.00 B/op 1.82 🔴
5389.00 allocs/op 2005406.00 allocs/op 372.13 🔴

@seqbenchbot

Copy link
Copy Markdown
Collaborator

Nice, @dkharms <(-^,^-)=b!

Benchmark 189238e7 (scenario bulk) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary
Query Type mean (ms) stddev (ms) p(50) (ms) p(95) (ms) p(99) (ms) iterations total
bulk
hot 25.80 5.25 25 38 60 21585 0

Compare it against another run with @seqbenchbot compare 189238e7 <other-id>.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Features or improvements that positively affect seq-db performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants