Skip to content

storage,logicalplan: lazy iterator allocation with batching for range… - #721

Merged
yeya24 merged 1 commit into
thanos-io:mainfrom
PaurushGarg:lazy-iterator-allocation
Jul 22, 2026
Merged

storage,logicalplan: lazy iterator allocation with batching for range…#721
yeya24 merged 1 commit into
thanos-io:mainfrom
PaurushGarg:lazy-iterator-allocation

Conversation

@PaurushGarg

@PaurushGarg PaurushGarg commented Jul 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Problem
Ruler instant queries with high cardinality hit OOM because matrixSelector.loadSeries() allocates iterators and buffers for ALL series upfront. With N series, this creates N merge iterators simultaneously. In Prometheus, this doesn't happen because its engine processes one series at a time and releases between series. The Thanos engine holds all iterators simultaneously.

Fix

  1. Lazy iterator allocation (matrix_selector.go)
    Move s.Iterator(nil) and newBuffer() from loadSeries() to Next(). Iterators are now created per-batch (controlled by seriesBatchSize). For instant queries, iterators are released after each batch (scanner.iterator = nil).
  2. Optimizer rewrite (set_batch_size.go)
    Rewrote the SelectorBatchSize optimizer - to enable batching for:
  • Range vector functions (rate, increase, present_over_time, etc.)
  • Set operations (or, and, unless)
  • One-to-one and scalar-vector binary ops
  • group_left/group_right (many-to-one/one-to-many): batching disabled as requires cross-series matching within batch
  • histogram_quantile, topk, bottomk: batching disabled as needs full series visibility

Benchmark

Instant queries (3K series, SelectorBatchSize=256): #721 (comment)

  • Range function queries (rate, sum_rate, sum_by_rate): 6-11% latency improvement
  • No regression on non-batched queries (sort, binary ops, histogram_quantile)
  • Memory/allocations: negligible overhead (+0.7%)

Range queries (3K series, SelectorBatchSize=256): #721 (comment)

  • Range function queries: 2-6% latency improvement
  • No regression elsewhere
  • Memory: neutral

@PaurushGarg

PaurushGarg commented Jul 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author 
> dev-dsk-paurushg-2a-87a5ee42 %   ~/go/bin/benchstat benchmarks/instant_baseline.out benchmarks/instant_change.out
> 
> goos: linux
> goarch: amd64
> pkg: github.com/thanos-io/promql-engine/engine
> cpu: AMD EPYC 7R13 Processor
>                                                              │ benchmarks/instant_baseline.out │    benchmarks/instant_change.out     │
>                                                              │             sec/op              │    sec/op      vs base               │
> InstantQuery/count_over_time_6h/new_engine-32                                     14.28m ± ∞ ¹    14.01m ± ∞ ¹        ~ (p=0.151 n=5)
> InstantQuery/sum_over_time_6h/new_engine-32                                       15.06m ± ∞ ¹    14.82m ± ∞ ¹   -1.58% (p=0.008 n=5)
> InstantQuery/avg_over_time_6h/new_engine-32                                       15.76m ± ∞ ¹    15.37m ± ∞ ¹   -2.45% (p=0.008 n=5)
> InstantQuery/min_over_time_6h/new_engine-32                                       14.65m ± ∞ ¹    14.47m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/max_over_time_6h/new_engine-32                                       14.63m ± ∞ ¹    14.42m ± ∞ ¹   -1.42% (p=0.032 n=5)
> InstantQuery/stddev_over_time_6h/new_engine-32                                    16.44m ± ∞ ¹    16.17m ± ∞ ¹   -1.66% (p=0.008 n=5)
> InstantQuery/stdvar_over_time_6h/new_engine-32                                    16.46m ± ∞ ¹    16.14m ± ∞ ¹   -1.96% (p=0.008 n=5)
> InstantQuery/present_over_time_6h/new_engine-32                                   14.52m ± ∞ ¹    14.10m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/last_over_time_6h/new_engine-32                                      14.60m ± ∞ ¹    14.06m ± ∞ ¹   -3.71% (p=0.008 n=5)
> InstantQuery/vector_selector/old_engine-32                                        9.934m ± ∞ ¹   10.217m ± ∞ ¹   +2.85% (p=0.032 n=5)
> InstantQuery/vector_selector/new_engine-32                                        4.061m ± ∞ ¹    4.077m ± ∞ ¹        ~ (p=0.841 n=5)
> InstantQuery/count/old_engine-32                                                  9.880m ± ∞ ¹   10.058m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/count/new_engine-32                                                  3.813m ± ∞ ¹    3.829m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/count_values/old_engine-32                                           12.18m ± ∞ ¹    12.45m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/count_values/new_engine-32                                           5.191m ± ∞ ¹    5.219m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/round/old_engine-32                                                  11.36m ± ∞ ¹    11.46m ± ∞ ¹        ~ (p=0.548 n=5)
> InstantQuery/round/new_engine-32                                                  4.676m ± ∞ ¹    4.908m ± ∞ ¹   +4.98% (p=0.008 n=5)
> InstantQuery/round_with_argument/old_engine-32                                    11.18m ± ∞ ¹    10.88m ± ∞ ¹        ~ (p=1.000 n=5)
> InstantQuery/round_with_argument/new_engine-32                                    4.671m ± ∞ ¹    4.874m ± ∞ ¹   +4.35% (p=0.008 n=5)
> InstantQuery/avg/old_engine-32                                                    9.770m ± ∞ ¹   10.025m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/avg/new_engine-32                                                    3.817m ± ∞ ¹    3.880m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/sum/old_engine-32                                                    9.784m ± ∞ ¹   10.093m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/sum/new_engine-32                                                    3.790m ± ∞ ¹    3.855m ± ∞ ¹   +1.71% (p=0.008 n=5)
> InstantQuery/sum_by_pod/old_engine-32                                             10.47m ± ∞ ¹    10.76m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/sum_by_pod/new_engine-32                                             5.481m ± ∞ ¹    5.528m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/rate/old_engine-32                                                   10.84m ± ∞ ¹    10.78m ± ∞ ¹        ~ (p=1.000 n=5)
> InstantQuery/rate/new_engine-32                                                   4.297m ± ∞ ¹    3.982m ± ∞ ¹   -7.33% (p=0.008 n=5)
> InstantQuery/rate_with_long_window/old_engine-32                                  10.83m ± ∞ ¹    10.92m ± ∞ ¹        ~ (p=0.548 n=5)
> InstantQuery/rate_with_long_window/new_engine-32                                  4.287m ± ∞ ¹    4.012m ± ∞ ¹   -6.41% (p=0.008 n=5)
> InstantQuery/sum_rate/old_engine-32                                               10.73m ± ∞ ¹    10.73m ± ∞ ¹        ~ (p=0.548 n=5)
> InstantQuery/sum_rate/new_engine-32                                               4.065m ± ∞ ¹    3.732m ± ∞ ¹   -8.20% (p=0.008 n=5)
> InstantQuery/sum_by_rate/old_engine-32                                            11.13m ± ∞ ¹    11.26m ± ∞ ¹        ~ (p=0.310 n=5)
> InstantQuery/sum_by_rate/new_engine-32                                            5.652m ± ∞ ¹    5.014m ± ∞ ¹  -11.28% (p=0.008 n=5)
> InstantQuery/binary_operation_with_many_to_one/old_engine-32                      17.12m ± ∞ ¹    17.61m ± ∞ ¹        ~ (p=0.151 n=5)
> InstantQuery/binary_operation_with_many_to_one/new_engine-32                      6.600m ± ∞ ¹    6.693m ± ∞ ¹   +1.40% (p=0.008 n=5)
> InstantQuery/unary_negation/old_engine-32                                         10.20m ± ∞ ¹    10.37m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/unary_negation/new_engine-32                                         4.729m ± ∞ ¹    4.853m ± ∞ ¹   +2.62% (p=0.008 n=5)
> InstantQuery/vector_and_scalar_comparison/old_engine-32                           11.45m ± ∞ ¹    11.76m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/vector_and_scalar_comparison/new_engine-32                           4.409m ± ∞ ¹    4.521m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/sort/old_engine-32                                                   11.38m ± ∞ ¹    11.91m ± ∞ ¹        ~ (p=0.056 n=5)
> InstantQuery/sort/new_engine-32                                                   4.645m ± ∞ ¹    4.752m ± ∞ ¹        ~ (p=0.151 n=5)
> InstantQuery/sort_desc/old_engine-32                                              11.53m ± ∞ ¹    12.01m ± ∞ ¹        ~ (p=0.222 n=5)
> InstantQuery/sort_desc/new_engine-32                                              4.666m ± ∞ ¹    4.661m ± ∞ ¹        ~ (p=0.841 n=5)
> InstantQuery/subquery_sum_over_time/old_engine-32                                 23.32m ± ∞ ¹    23.66m ± ∞ ¹        ~ (p=0.310 n=5)
> InstantQuery/subquery_sum_over_time/new_engine-32                                 6.353m ± ∞ ¹    6.487m ± ∞ ¹   +2.11% (p=0.008 n=5)
> InstantQuery/double_exponential_smoothing/old_engine-32                           10.80m ± ∞ ¹    11.15m ± ∞ ¹        ~ (p=0.095 n=5)
> InstantQuery/double_exponential_smoothing/new_engine-32                           4.245m ± ∞ ¹    3.978m ± ∞ ¹   -6.29% (p=0.008 n=5)
> geomean                                                                           8.401m          8.402m         +0.01%
> ¹ need >= 6 samples for confidence interval at level 0.95
> 
>                                                              │ benchmarks/instant_baseline.out │    benchmarks/instant_change.out    │
>                                                              │              B/op               │     B/op       vs base              │
> InstantQuery/count_over_time_6h/new_engine-32                                    6.125Mi ± ∞ ¹   6.187Mi ± ∞ ¹  +1.01% (p=0.008 n=5)
> InstantQuery/sum_over_time_6h/new_engine-32                                      6.216Mi ± ∞ ¹   6.278Mi ± ∞ ¹  +1.00% (p=0.008 n=5)
> InstantQuery/avg_over_time_6h/new_engine-32                                      6.353Mi ± ∞ ¹   6.416Mi ± ∞ ¹  +0.98% (p=0.008 n=5)
> InstantQuery/min_over_time_6h/new_engine-32                                      6.124Mi ± ∞ ¹   6.187Mi ± ∞ ¹  +1.02% (p=0.008 n=5)
> InstantQuery/max_over_time_6h/new_engine-32                                      6.124Mi ± ∞ ¹   6.187Mi ± ∞ ¹  +1.02% (p=0.008 n=5)
> InstantQuery/stddev_over_time_6h/new_engine-32                                   6.216Mi ± ∞ ¹   6.278Mi ± ∞ ¹  +1.00% (p=0.008 n=5)
> InstantQuery/stdvar_over_time_6h/new_engine-32                                   6.216Mi ± ∞ ¹   6.278Mi ± ∞ ¹  +1.00% (p=0.008 n=5)
> InstantQuery/present_over_time_6h/new_engine-32                                  6.124Mi ± ∞ ¹   6.187Mi ± ∞ ¹  +1.02% (p=0.008 n=5)
> InstantQuery/last_over_time_6h/new_engine-32                                     5.804Mi ± ∞ ¹   5.866Mi ± ∞ ¹  +1.08% (p=0.008 n=5)
> InstantQuery/vector_selector/old_engine-32                                       1.502Mi ± ∞ ¹   1.597Mi ± ∞ ¹       ~ (p=0.421 n=5)
> InstantQuery/vector_selector/new_engine-32                                       3.591Mi ± ∞ ¹   3.602Mi ± ∞ ¹  +0.29% (p=0.008 n=5)
> InstantQuery/count/old_engine-32                                                 1.640Mi ± ∞ ¹   1.650Mi ± ∞ ¹  +0.64% (p=0.008 n=5)
> InstantQuery/count/new_engine-32                                                 3.134Mi ± ∞ ¹   3.144Mi ± ∞ ¹  +0.33% (p=0.008 n=5)
> InstantQuery/count_values/old_engine-32                                          2.829Mi ± ∞ ¹   2.840Mi ± ∞ ¹  +0.37% (p=0.008 n=5)
> InstantQuery/count_values/new_engine-32                                          3.893Mi ± ∞ ¹   3.903Mi ± ∞ ¹  +0.27% (p=0.008 n=5)
> InstantQuery/round/old_engine-32                                                 2.782Mi ± ∞ ¹   2.792Mi ± ∞ ¹  +0.37% (p=0.008 n=5)
> InstantQuery/round/new_engine-32                                                 4.078Mi ± ∞ ¹   4.088Mi ± ∞ ¹  +0.26% (p=0.008 n=5)
> InstantQuery/round_with_argument/old_engine-32                                   2.783Mi ± ∞ ¹   2.793Mi ± ∞ ¹  +0.37% (p=0.008 n=5)
> InstantQuery/round_with_argument/new_engine-32                                   4.078Mi ± ∞ ¹   4.089Mi ± ∞ ¹  +0.26% (p=0.008 n=5)
> InstantQuery/avg/old_engine-32                                                   1.641Mi ± ∞ ¹   1.648Mi ± ∞ ¹  +0.42% (p=0.008 n=5)
> InstantQuery/avg/new_engine-32                                                   3.134Mi ± ∞ ¹   3.145Mi ± ∞ ¹  +0.33% (p=0.008 n=5)
> InstantQuery/sum/old_engine-32                                                   1.638Mi ± ∞ ¹   1.648Mi ± ∞ ¹  +0.64% (p=0.008 n=5)
> InstantQuery/sum/new_engine-32                                                   3.134Mi ± ∞ ¹   3.144Mi ± ∞ ¹  +0.33% (p=0.008 n=5)
> InstantQuery/sum_by_pod/old_engine-32                                            2.068Mi ± ∞ ¹   2.079Mi ± ∞ ¹  +0.51% (p=0.008 n=5)
> InstantQuery/sum_by_pod/new_engine-32                                            4.259Mi ± ∞ ¹   4.269Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
> InstantQuery/rate/old_engine-32                                                  1.658Mi ± ∞ ¹   1.669Mi ± ∞ ¹  +0.63% (p=0.008 n=5)
> InstantQuery/rate/new_engine-32                                                  4.982Mi ± ∞ ¹   5.055Mi ± ∞ ¹  +1.47% (p=0.008 n=5)
> InstantQuery/rate_with_long_window/old_engine-32                                 1.658Mi ± ∞ ¹   1.669Mi ± ∞ ¹  +0.63% (p=0.008 n=5)
> InstantQuery/rate_with_long_window/new_engine-32                                 4.982Mi ± ∞ ¹   5.055Mi ± ∞ ¹  +1.47% (p=0.008 n=5)
> InstantQuery/sum_rate/old_engine-32                                              1.712Mi ± ∞ ¹   1.723Mi ± ∞ ¹  +0.61% (p=0.008 n=5)
> InstantQuery/sum_rate/new_engine-32                                              4.525Mi ± ∞ ¹   4.598Mi ± ∞ ¹  +1.61% (p=0.008 n=5)
> InstantQuery/sum_by_rate/old_engine-32                                           2.102Mi ± ∞ ¹   2.115Mi ± ∞ ¹  +0.61% (p=0.008 n=5)
> InstantQuery/sum_by_rate/new_engine-32                                           5.650Mi ± ∞ ¹   5.723Mi ± ∞ ¹  +1.29% (p=0.008 n=5)
> InstantQuery/binary_operation_with_many_to_one/old_engine-32                     4.763Mi ± ∞ ¹   4.773Mi ± ∞ ¹  +0.21% (p=0.008 n=5)
> InstantQuery/binary_operation_with_many_to_one/new_engine-32                     6.385Mi ± ∞ ¹   6.395Mi ± ∞ ¹  +0.16% (p=0.008 n=5)
> InstantQuery/unary_negation/old_engine-32                                        1.657Mi ± ∞ ¹   1.668Mi ± ∞ ¹  +0.63% (p=0.008 n=5)
> InstantQuery/unary_negation/new_engine-32                                        4.076Mi ± ∞ ¹   4.087Mi ± ∞ ¹  +0.25% (p=0.008 n=5)
> InstantQuery/vector_and_scalar_comparison/old_engine-32                          2.782Mi ± ∞ ¹   2.793Mi ± ∞ ¹  +0.37% (p=0.008 n=5)
> InstantQuery/vector_and_scalar_comparison/new_engine-32                          4.205Mi ± ∞ ¹   4.215Mi ± ∞ ¹  +0.25% (p=0.008 n=5)
> InstantQuery/sort/old_engine-32                                                  2.782Mi ± ∞ ¹   2.792Mi ± ∞ ¹  +0.37% (p=0.008 n=5)
> InstantQuery/sort/new_engine-32                                                  3.592Mi ± ∞ ¹   3.602Mi ± ∞ ¹  +0.29% (p=0.008 n=5)
> InstantQuery/sort_desc/old_engine-32                                             2.782Mi ± ∞ ¹   2.792Mi ± ∞ ¹  +0.37% (p=0.008 n=5)
> InstantQuery/sort_desc/new_engine-32                                             3.592Mi ± ∞ ¹   3.602Mi ± ∞ ¹  +0.29% (p=0.008 n=5)
> InstantQuery/subquery_sum_over_time/old_engine-32                                1.663Mi ± ∞ ¹   1.667Mi ± ∞ ¹  +0.27% (p=0.008 n=5)
> InstantQuery/subquery_sum_over_time/new_engine-32                                3.694Mi ± ∞ ¹   3.705Mi ± ∞ ¹  +0.28% (p=0.008 n=5)
> InstantQuery/double_exponential_smoothing/old_engine-32                          1.935Mi ± ∞ ¹   1.945Mi ± ∞ ¹  +0.54% (p=0.008 n=5)
> InstantQuery/double_exponential_smoothing/new_engine-32                          4.937Mi ± ∞ ¹   5.010Mi ± ∞ ¹  +1.48% (p=0.008 n=5)
> geomean                                                                          3.404Mi         3.429Mi        +0.74%
> ¹ need >= 6 samples for confidence interval at level 0.95
> 
>                                                              │ benchmarks/instant_baseline.out │   benchmarks/instant_change.out    │
>                                                              │            allocs/op            │  allocs/op    vs base              │
> InstantQuery/count_over_time_6h/new_engine-32                                     78.68k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=0.246 n=5)
> InstantQuery/sum_over_time_6h/new_engine-32                                       78.67k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=1.000 n=5)
> InstantQuery/avg_over_time_6h/new_engine-32                                       78.67k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=0.651 n=5)
> InstantQuery/min_over_time_6h/new_engine-32                                       78.67k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=0.595 n=5)
> InstantQuery/max_over_time_6h/new_engine-32                                       78.67k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=0.905 n=5)
> InstantQuery/stddev_over_time_6h/new_engine-32                                    78.67k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=1.000 n=5)
> InstantQuery/stdvar_over_time_6h/new_engine-32                                    78.67k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=0.563 n=5)
> InstantQuery/present_over_time_6h/new_engine-32                                   78.67k ± ∞ ¹   78.67k ± ∞ ¹       ~ (p=0.690 n=5)
> InstantQuery/last_over_time_6h/new_engine-32                                      69.67k ± ∞ ¹   69.67k ± ∞ ¹       ~ (p=0.317 n=5)
> InstantQuery/vector_selector/old_engine-32                                        19.28k ± ∞ ¹   20.80k ± ∞ ¹       ~ (p=0.492 n=5)
> InstantQuery/vector_selector/new_engine-32                                        36.19k ± ∞ ¹   36.30k ± ∞ ¹  +0.30% (p=0.008 n=5)
> InstantQuery/count/old_engine-32                                                  17.71k ± ∞ ¹   17.82k ± ∞ ¹  +0.61% (p=0.008 n=5)
> InstantQuery/count/new_engine-32                                                  33.28k ± ∞ ¹   33.39k ± ∞ ¹  +0.32% (p=0.008 n=5)
> InstantQuery/count_values/old_engine-32                                           29.73k ± ∞ ¹   29.84k ± ∞ ¹  +0.36% (p=0.008 n=5)
> InstantQuery/count_values/new_engine-32                                           38.36k ± ∞ ¹   38.47k ± ∞ ¹  +0.28% (p=0.008 n=5)
> InstantQuery/round/old_engine-32                                                  23.72k ± ∞ ¹   23.83k ± ∞ ¹  +0.46% (p=0.008 n=5)
> InstantQuery/round/new_engine-32                                                  45.23k ± ∞ ¹   45.34k ± ∞ ¹  +0.24% (p=0.008 n=5)
> InstantQuery/round_with_argument/old_engine-32                                    23.75k ± ∞ ¹   23.85k ± ∞ ¹  +0.45% (p=0.008 n=5)
> InstantQuery/round_with_argument/new_engine-32                                    45.26k ± ∞ ¹   45.37k ± ∞ ¹  +0.24% (p=0.008 n=5)
> InstantQuery/avg/old_engine-32                                                    17.71k ± ∞ ¹   17.82k ± ∞ ¹  +0.60% (p=0.008 n=5)
> InstantQuery/avg/new_engine-32                                                    33.29k ± ∞ ¹   33.39k ± ∞ ¹  +0.32% (p=0.008 n=5)
> InstantQuery/sum/old_engine-32                                                    17.71k ± ∞ ¹   17.82k ± ∞ ¹  +0.61% (p=0.008 n=5)
> InstantQuery/sum/new_engine-32                                                    33.28k ± ∞ ¹   33.39k ± ∞ ¹  +0.32% (p=0.008 n=5)
> InstantQuery/sum_by_pod/old_engine-32                                             19.74k ± ∞ ¹   19.85k ± ∞ ¹  +0.55% (p=0.008 n=5)
> InstantQuery/sum_by_pod/new_engine-32                                             51.33k ± ∞ ¹   51.44k ± ∞ ¹  +0.21% (p=0.008 n=5)
> InstantQuery/rate/old_engine-32                                                   20.71k ± ∞ ¹   20.82k ± ∞ ¹  +0.52% (p=0.008 n=5)
> InstantQuery/rate/new_engine-32                                                   57.26k ± ∞ ¹   57.36k ± ∞ ¹  +0.19% (p=0.008 n=5)
> InstantQuery/rate_with_long_window/old_engine-32                                  20.71k ± ∞ ¹   20.82k ± ∞ ¹  +0.52% (p=0.008 n=5)
> InstantQuery/rate_with_long_window/new_engine-32                                  57.25k ± ∞ ¹   57.36k ± ∞ ¹  +0.19% (p=0.008 n=5)
> InstantQuery/sum_rate/old_engine-32                                               17.74k ± ∞ ¹   17.84k ± ∞ ¹  +0.61% (p=0.008 n=5)
> InstantQuery/sum_rate/new_engine-32                                               54.35k ± ∞ ¹   54.46k ± ∞ ¹  +0.20% (p=0.008 n=5)
> InstantQuery/sum_by_rate/old_engine-32                                            19.77k ± ∞ ¹   19.88k ± ∞ ¹  +0.55% (p=0.008 n=5)
> InstantQuery/sum_by_rate/new_engine-32                                            72.40k ± ∞ ¹   72.51k ± ∞ ¹  +0.15% (p=0.008 n=5)
> InstantQuery/binary_operation_with_many_to_one/old_engine-32                      42.73k ± ∞ ¹   42.83k ± ∞ ¹  +0.24% (p=0.008 n=5)
> InstantQuery/binary_operation_with_many_to_one/new_engine-32                      55.80k ± ∞ ¹   55.90k ± ∞ ¹  +0.18% (p=0.008 n=5)
> InstantQuery/unary_negation/old_engine-32                                         20.71k ± ∞ ¹   20.81k ± ∞ ¹  +0.52% (p=0.008 n=5)
> InstantQuery/unary_negation/new_engine-32                                         45.23k ± ∞ ¹   45.34k ± ∞ ¹  +0.24% (p=0.008 n=5)
> InstantQuery/vector_and_scalar_comparison/old_engine-32                           23.71k ± ∞ ¹   23.82k ± ∞ ¹  +0.46% (p=0.008 n=5)
> InstantQuery/vector_and_scalar_comparison/new_engine-32                           36.27k ± ∞ ¹   36.37k ± ∞ ¹  +0.30% (p=0.008 n=5)
> InstantQuery/sort/old_engine-32                                                   23.73k ± ∞ ¹   23.83k ± ∞ ¹  +0.46% (p=0.008 n=5)
> InstantQuery/sort/new_engine-32                                                   36.20k ± ∞ ¹   36.30k ± ∞ ¹  +0.30% (p=0.008 n=5)
> InstantQuery/sort_desc/old_engine-32                                              23.73k ± ∞ ¹   23.83k ± ∞ ¹  +0.46% (p=0.008 n=5)
> InstantQuery/sort_desc/new_engine-32                                              36.20k ± ∞ ¹   36.31k ± ∞ ¹  +0.30% (p=0.008 n=5)
> InstantQuery/subquery_sum_over_time/old_engine-32                                 17.76k ± ∞ ¹   17.87k ± ∞ ¹  +0.60% (p=0.008 n=5)
> InstantQuery/subquery_sum_over_time/new_engine-32                                 34.84k ± ∞ ¹   34.95k ± ∞ ¹  +0.31% (p=0.008 n=5)
> InstantQuery/double_exponential_smoothing/old_engine-32                           26.75k ± ∞ ¹   26.86k ± ∞ ¹  +0.40% (p=0.008 n=5)
> InstantQuery/double_exponential_smoothing/new_engine-32                           48.28k ± ∞ ¹   48.39k ± ∞ ¹  +0.22% (p=0.008 n=5)
> geomean                                                                           36.87k         37.03k        +0.45%
> ¹ need >= 6 samples for confidence interval at level 0.95
> 
>

@PaurushGarg

PaurushGarg commented Jul 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author 

dev-dsk-paurushg-2a-87a5ee42 %   ~/go/bin/benchstat benchmarks/range_baseline.out benchmarks/range_change.out

goos: linux
goarch: amd64
pkg: github.com/thanos-io/promql-engine/engine
cpu: AMD EPYC 7R13 Processor
                                                                 │ benchmarks/range_baseline.out │     benchmarks/range_change.out     │
                                                                 │            sec/op             │    sec/op     vs base               │
RangeQuery/vector_selector/old_engine-32                                            42.25m ± ∞ ¹   43.26m ± ∞ ¹        ~ (p=0.095 n=5)
RangeQuery/vector_selector/new_engine-32                                            22.66m ± ∞ ¹   23.01m ± ∞ ¹        ~ (p=0.056 n=5)
RangeQuery/sum/old_engine-32                                                        46.27m ± ∞ ¹   47.54m ± ∞ ¹        ~ (p=0.151 n=5)
RangeQuery/sum/new_engine-32                                                        10.20m ± ∞ ¹   10.40m ± ∞ ¹   +1.97% (p=0.032 n=5)
RangeQuery/sum_by_pod/old_engine-32                                                 54.08m ± ∞ ¹   57.17m ± ∞ ¹        ~ (p=0.056 n=5)
RangeQuery/sum_by_pod/new_engine-32                                                 31.40m ± ∞ ¹   30.65m ± ∞ ¹   -2.38% (p=0.008 n=5)
RangeQuery/topk/old_engine-32                                                       49.51m ± ∞ ¹   50.20m ± ∞ ¹        ~ (p=0.056 n=5)
RangeQuery/topk/new_engine-32                                                       13.60m ± ∞ ¹   13.70m ± ∞ ¹        ~ (p=0.310 n=5)
RangeQuery/bottomk/old_engine-32                                                    50.25m ± ∞ ¹   51.50m ± ∞ ¹        ~ (p=0.151 n=5)
RangeQuery/bottomk/new_engine-32                                                    13.44m ± ∞ ¹   13.77m ± ∞ ¹   +2.47% (p=0.008 n=5)
RangeQuery/limitk/old_engine-32                                                     44.19m ± ∞ ¹   45.76m ± ∞ ¹   +3.54% (p=0.016 n=5)
RangeQuery/limitk/new_engine-32                                                     8.998m ± ∞ ¹   9.001m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/limit_ratio/old_engine-32                                                109.1m ± ∞ ¹   109.3m ± ∞ ¹        ~ (p=0.222 n=5)
RangeQuery/limit_ratio/new_engine-32                                                44.47m ± ∞ ¹   44.18m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/rate/old_engine-32                                                       105.0m ± ∞ ¹   105.3m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/rate/new_engine-32                                                       22.15m ± ∞ ¹   21.46m ± ∞ ¹   -3.14% (p=0.008 n=5)
RangeQuery/rate_with_longer_window/old_engine-32                                    48.83m ± ∞ ¹   48.43m ± ∞ ¹        ~ (p=0.421 n=5)
RangeQuery/rate_with_longer_window/new_engine-32                                   10.039m ± ∞ ¹   9.579m ± ∞ ¹   -4.58% (p=0.008 n=5)
RangeQuery/subquery/old_engine-32                                                   141.1m ± ∞ ¹   143.0m ± ∞ ¹        ~ (p=0.151 n=5)
RangeQuery/subquery/new_engine-32                                                   73.36m ± ∞ ¹   73.22m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/sum_rate/old_engine-32                                                   107.3m ± ∞ ¹   107.2m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/sum_rate/new_engine-32                                                   16.67m ± ∞ ¹   16.45m ± ∞ ¹        ~ (p=0.222 n=5)
RangeQuery/sum_by_rate/old_engine-32                                                114.6m ± ∞ ¹   114.1m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/sum_by_rate/new_engine-32                                                31.77m ± ∞ ¹   30.28m ± ∞ ¹   -4.69% (p=0.008 n=5)
RangeQuery/quantile_with_variable_parameter/old_engine-32                           169.3m ± ∞ ¹   170.6m ± ∞ ¹        ~ (p=0.421 n=5)
RangeQuery/quantile_with_variable_parameter/new_engine-32                           47.45m ± ∞ ¹   46.75m ± ∞ ¹        ~ (p=0.095 n=5)
RangeQuery/binary_operation_with_one_to_one/old_engine-32                           100.6m ± ∞ ¹   100.6m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/binary_operation_with_one_to_one/new_engine-32                           18.67m ± ∞ ¹   18.73m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/binary_operation_with_many_to_one/old_engine-32                          315.3m ± ∞ ¹   318.0m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/binary_operation_with_many_to_one/new_engine-32                          42.63m ± ∞ ¹   42.53m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/binary_operation_with_vector_and_scalar/old_engine-32                    137.3m ± ∞ ¹   138.4m ± ∞ ¹        ~ (p=0.222 n=5)
RangeQuery/binary_operation_with_vector_and_scalar/new_engine-32                    33.61m ± ∞ ¹   34.63m ± ∞ ¹        ~ (p=0.310 n=5)
RangeQuery/unary_negation/old_engine-32                                             44.03m ± ∞ ¹   45.59m ± ∞ ¹        ~ (p=0.095 n=5)
RangeQuery/unary_negation/new_engine-32                                             22.59m ± ∞ ¹   22.75m ± ∞ ¹        ~ (p=0.310 n=5)
RangeQuery/vector_and_scalar_comparison/old_engine-32                               137.4m ± ∞ ¹   137.4m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/vector_and_scalar_comparison/new_engine-32                               34.12m ± ∞ ¹   34.95m ± ∞ ¹        ~ (p=0.222 n=5)
RangeQuery/positive_offset_vector/old_engine-32                                     40.11m ± ∞ ¹   41.20m ± ∞ ¹   +2.71% (p=0.032 n=5)
RangeQuery/positive_offset_vector/new_engine-32                                     21.31m ± ∞ ¹   21.33m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/at_modifier_/old_engine-32                                               15.53m ± ∞ ¹   15.52m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/at_modifier_/new_engine-32                                               13.46m ± ∞ ¹   13.55m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/at_modifier_with_positive_offset_vector/old_engine-32                    15.40m ± ∞ ¹   15.09m ± ∞ ¹   -2.03% (p=0.032 n=5)
RangeQuery/at_modifier_with_positive_offset_vector/new_engine-32                    13.39m ± ∞ ¹   13.71m ± ∞ ¹   +2.34% (p=0.008 n=5)
RangeQuery/clamp/old_engine-32                                                      138.9m ± ∞ ¹   133.1m ± ∞ ¹   -4.16% (p=0.032 n=5)
RangeQuery/clamp/new_engine-32                                                      30.87m ± ∞ ¹   30.27m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/clamp_min/old_engine-32                                                  132.0m ± ∞ ¹   133.8m ± ∞ ¹        ~ (p=0.421 n=5)
RangeQuery/clamp_min/new_engine-32                                                  28.57m ± ∞ ¹   28.03m ± ∞ ¹        ~ (p=0.310 n=5)
RangeQuery/complex_func_query/old_engine-32                                         248.0m ± ∞ ¹   244.5m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/complex_func_query/new_engine-32                                         42.22m ± ∞ ¹   41.46m ± ∞ ¹   -1.80% (p=0.016 n=5)
RangeQuery/func_within_func_query/old_engine-32                                     82.52m ± ∞ ¹   77.54m ± ∞ ¹   -6.03% (p=0.016 n=5)
RangeQuery/func_within_func_query/new_engine-32                                     11.47m ± ∞ ¹   10.78m ± ∞ ¹   -5.97% (p=0.008 n=5)
RangeQuery/aggr_within_func_query/old_engine-32                                     75.30m ± ∞ ¹   75.25m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/aggr_within_func_query/new_engine-32                                    10.321m ± ∞ ¹   9.852m ± ∞ ¹   -4.54% (p=0.008 n=5)
RangeQuery/histogram_quantile/old_engine-32                                         576.0m ± ∞ ¹   579.4m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/histogram_quantile/new_engine-32                                         143.1m ± ∞ ¹   142.2m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/sort/old_engine-32                                                       213.1m ± ∞ ¹   218.7m ± ∞ ¹        ~ (p=0.095 n=5)
RangeQuery/sort/new_engine-32                                                       22.06m ± ∞ ¹   21.88m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/sort_desc/old_engine-32                                                  213.5m ± ∞ ¹   220.4m ± ∞ ¹        ~ (p=0.056 n=5)
RangeQuery/sort_desc/new_engine-32                                                  22.42m ± ∞ ¹   22.23m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/absent_and_exists/old_engine-32                                          51.85m ± ∞ ¹   52.50m ± ∞ ¹        ~ (p=0.222 n=5)
RangeQuery/absent_and_exists/new_engine-32                                          8.508m ± ∞ ¹   8.514m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/absent_and_doesnt_exist/old_engine-32                                    53.06µ ± ∞ ¹   53.92µ ± ∞ ¹        ~ (p=0.151 n=5)
RangeQuery/absent_and_doesnt_exist/new_engine-32                                    809.6µ ± ∞ ¹   800.7µ ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/double_exponential_smoothing/old_engine-32                               145.5m ± ∞ ¹   145.5m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/double_exponential_smoothing/new_engine-32                               22.90m ± ∞ ¹   21.92m ± ∞ ¹   -4.30% (p=0.008 n=5)
RangeQuery/count_over_time_5m/old_engine-32                                         88.18m ± ∞ ¹   88.34m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/count_over_time_5m/new_engine-32                                         22.69m ± ∞ ¹   22.18m ± ∞ ¹        ~ (p=0.222 n=5)
RangeQuery/count_over_time_1h/old_engine-32                                         87.91m ± ∞ ¹   90.93m ± ∞ ¹        ~ (p=0.151 n=5)
RangeQuery/count_over_time_1h/new_engine-32                                         28.41m ± ∞ ¹   28.40m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/count_over_time_6h/old_engine-32                                         85.59m ± ∞ ¹   84.44m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/count_over_time_6h/new_engine-32                                         31.00m ± ∞ ¹   31.47m ± ∞ ¹        ~ (p=0.056 n=5)
RangeQuery/sum_over_time_5m/old_engine-32                                           108.6m ± ∞ ¹   107.8m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/sum_over_time_5m/new_engine-32                                           22.37m ± ∞ ¹   22.14m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/sum_over_time_1h/old_engine-32                                           260.9m ± ∞ ¹   258.7m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/sum_over_time_1h/new_engine-32                                           47.81m ± ∞ ¹   48.14m ± ∞ ¹   +0.68% (p=0.008 n=5)
RangeQuery/sum_over_time_6h/old_engine-32                                           309.3m ± ∞ ¹   309.2m ± ∞ ¹        ~ (p=0.421 n=5)
RangeQuery/sum_over_time_6h/new_engine-32                                           57.38m ± ∞ ¹   58.16m ± ∞ ¹   +1.38% (p=0.008 n=5)
RangeQuery/avg_over_time_5m/old_engine-32                                           120.2m ± ∞ ¹   117.8m ± ∞ ¹   -1.97% (p=0.016 n=5)
RangeQuery/avg_over_time_5m/new_engine-32                                           22.77m ± ∞ ¹   22.41m ± ∞ ¹   -1.60% (p=0.008 n=5)
RangeQuery/avg_over_time_1h/old_engine-32                                           352.2m ± ∞ ¹   352.1m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/avg_over_time_1h/new_engine-32                                           57.54m ± ∞ ¹   58.07m ± ∞ ¹        ~ (p=0.151 n=5)
RangeQuery/avg_over_time_6h/old_engine-32                                           432.1m ± ∞ ¹   431.4m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/avg_over_time_6h/new_engine-32                                           70.26m ± ∞ ¹   70.91m ± ∞ ¹        ~ (p=0.095 n=5)
RangeQuery/min_over_time_5m/old_engine-32                                           110.7m ± ∞ ¹   110.4m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/min_over_time_5m/new_engine-32                                           22.48m ± ∞ ¹   21.68m ± ∞ ¹   -3.55% (p=0.008 n=5)
RangeQuery/min_over_time_1h/old_engine-32                                           260.8m ± ∞ ¹   259.8m ± ∞ ¹        ~ (p=0.310 n=5)
RangeQuery/min_over_time_1h/new_engine-32                                           36.84m ± ∞ ¹   36.78m ± ∞ ¹        ~ (p=0.421 n=5)
RangeQuery/min_over_time_6h/old_engine-32                                           311.9m ± ∞ ¹   309.6m ± ∞ ¹        ~ (p=0.690 n=5)
RangeQuery/min_over_time_6h/new_engine-32                                           42.04m ± ∞ ¹   42.31m ± ∞ ¹        ~ (p=0.151 n=5)
RangeQuery/max_over_time_5m/old_engine-32                                           111.2m ± ∞ ¹   112.9m ± ∞ ¹   +1.56% (p=0.008 n=5)
RangeQuery/max_over_time_5m/new_engine-32                                           22.65m ± ∞ ¹   22.33m ± ∞ ¹   -1.43% (p=0.008 n=5)
RangeQuery/max_over_time_1h/old_engine-32                                           250.2m ± ∞ ¹   249.2m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/max_over_time_1h/new_engine-32                                           35.23m ± ∞ ¹   35.64m ± ∞ ¹   +1.16% (p=0.032 n=5)
RangeQuery/max_over_time_6h/old_engine-32                                           295.8m ± ∞ ¹   294.8m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/max_over_time_6h/new_engine-32                                           39.55m ± ∞ ¹   40.85m ± ∞ ¹   +3.30% (p=0.008 n=5)
RangeQuery/stddev_over_time_5m/old_engine-32                                        182.1m ± ∞ ¹   182.1m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/stddev_over_time_5m/new_engine-32                                        25.07m ± ∞ ¹   24.88m ± ∞ ¹        ~ (p=0.056 n=5)
RangeQuery/stddev_over_time_1h/old_engine-32                                        895.0m ± ∞ ¹   893.0m ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/stddev_over_time_1h/new_engine-32                                        79.00m ± ∞ ¹   79.30m ± ∞ ¹        ~ (p=0.548 n=5)
RangeQuery/stddev_over_time_6h/old_engine-32                                         1.156 ± ∞ ¹    1.160 ± ∞ ¹        ~ (p=0.841 n=5)
RangeQuery/stddev_over_time_6h/new_engine-32                                        98.48m ± ∞ ¹   98.46m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/stdvar_over_time/old_engine-32                                           177.9m ± ∞ ¹   178.3m ± ∞ ¹        ~ (p=1.000 n=5)
RangeQuery/stdvar_over_time/new_engine-32                                           25.14m ± ∞ ¹   24.86m ± ∞ ¹   -1.10% (p=0.032 n=5)
RangeQuery/last_over_time/old_engine-32                                             91.55m ± ∞ ¹   90.82m ± ∞ ¹        ~ (p=0.421 n=5)
RangeQuery/last_over_time/new_engine-32                                             22.39m ± ∞ ¹   21.76m ± ∞ ¹   -2.81% (p=0.008 n=5)
RangeQuery/present_over_time/old_engine-32                                          89.82m ± ∞ ¹   79.12m ± ∞ ¹  -11.92% (p=0.008 n=5)
RangeQuery/present_over_time/new_engine-32                                          22.77m ± ∞ ¹   22.26m ± ∞ ¹   -2.24% (p=0.032 n=5)
geomean                                                                             52.02m         51.90m         -0.24%
¹ need >= 6 samples for confidence interval at level 0.95

                                                                 │ benchmarks/range_baseline.out │     benchmarks/range_change.out     │
                                                                 │             B/op              │     B/op       vs base              │
RangeQuery/vector_selector/old_engine-32                                           13.51Mi ± ∞ ¹   13.51Mi ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/vector_selector/new_engine-32                                           18.04Mi ± ∞ ¹   18.04Mi ± ∞ ¹       ~ (p=0.690 n=5)
RangeQuery/sum/old_engine-32                                                       2.038Mi ± ∞ ¹   2.038Mi ± ∞ ¹       ~ (p=0.889 n=5)
RangeQuery/sum/new_engine-32                                                       5.839Mi ± ∞ ¹   5.839Mi ± ∞ ¹       ~ (p=0.222 n=5)
RangeQuery/sum_by_pod/old_engine-32                                                6.319Mi ± ∞ ¹   6.324Mi ± ∞ ¹       ~ (p=0.381 n=5)
RangeQuery/sum_by_pod/new_engine-32                                                11.48Mi ± ∞ ¹   11.48Mi ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/topk/old_engine-32                                                      2.472Mi ± ∞ ¹   2.479Mi ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/topk/new_engine-32                                                      6.452Mi ± ∞ ¹   6.448Mi ± ∞ ¹  -0.06% (p=0.008 n=5)
RangeQuery/bottomk/old_engine-32                                                   2.474Mi ± ∞ ¹   2.462Mi ± ∞ ¹       ~ (p=0.278 n=5)
RangeQuery/bottomk/new_engine-32                                                   6.448Mi ± ∞ ¹   6.448Mi ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/limitk/old_engine-32                                                    3.796Mi ± ∞ ¹   3.786Mi ± ∞ ¹       ~ (p=0.310 n=5)
RangeQuery/limitk/new_engine-32                                                    7.797Mi ± ∞ ¹   7.787Mi ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/limit_ratio/old_engine-32                                               20.99Mi ± ∞ ¹   20.98Mi ± ∞ ¹       ~ (p=0.135 n=5)
RangeQuery/limit_ratio/new_engine-32                                               14.63Mi ± ∞ ¹   14.62Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/rate/old_engine-32                                                      14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/rate/new_engine-32                                                      20.95Mi ± ∞ ¹   21.00Mi ± ∞ ¹  +0.25% (p=0.008 n=5)
RangeQuery/rate_with_longer_window/old_engine-32                                   4.405Mi ± ∞ ¹   4.395Mi ± ∞ ¹  -0.23% (p=0.008 n=5)
RangeQuery/rate_with_longer_window/new_engine-32                                   9.610Mi ± ∞ ¹   9.663Mi ± ∞ ¹  +0.55% (p=0.008 n=5)
RangeQuery/subquery/old_engine-32                                                  22.09Mi ± ∞ ¹   22.08Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/subquery/new_engine-32                                                  23.78Mi ± ∞ ¹   23.83Mi ± ∞ ¹  +0.22% (p=0.008 n=5)
RangeQuery/sum_rate/old_engine-32                                                  3.482Mi ± ∞ ¹   3.474Mi ± ∞ ¹       ~ (p=0.540 n=5)
RangeQuery/sum_rate/new_engine-32                                                  8.747Mi ± ∞ ¹   8.799Mi ± ∞ ¹  +0.60% (p=0.008 n=5)
RangeQuery/sum_by_rate/old_engine-32                                               7.718Mi ± ∞ ¹   7.708Mi ± ∞ ¹       ~ (p=0.143 n=5)
RangeQuery/sum_by_rate/new_engine-32                                               14.39Mi ± ∞ ¹   14.44Mi ± ∞ ¹  +0.36% (p=0.008 n=5)
RangeQuery/quantile_with_variable_parameter/old_engine-32                          93.98Mi ± ∞ ¹   93.94Mi ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/quantile_with_variable_parameter/new_engine-32                          20.55Mi ± ∞ ¹   20.53Mi ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/binary_operation_with_one_to_one/old_engine-32                          7.135Mi ± ∞ ¹   7.144Mi ± ∞ ¹       ~ (p=0.127 n=5)
RangeQuery/binary_operation_with_one_to_one/new_engine-32                          9.841Mi ± ∞ ¹   9.837Mi ± ∞ ¹  -0.04% (p=0.008 n=5)
RangeQuery/binary_operation_with_many_to_one/old_engine-32                         62.40Mi ± ∞ ¹   62.39Mi ± ∞ ¹       ~ (p=0.103 n=5)
RangeQuery/binary_operation_with_many_to_one/new_engine-32                         23.28Mi ± ∞ ¹   23.27Mi ± ∞ ¹  -0.04% (p=0.008 n=5)
RangeQuery/binary_operation_with_vector_and_scalar/old_engine-32                   15.98Mi ± ∞ ¹   15.97Mi ± ∞ ¹       ~ (p=0.167 n=5)
RangeQuery/binary_operation_with_vector_and_scalar/new_engine-32                   20.36Mi ± ∞ ¹   20.35Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/unary_negation/old_engine-32                                            14.93Mi ± ∞ ¹   14.92Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/unary_negation/new_engine-32                                            19.88Mi ± ∞ ¹   19.87Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/vector_and_scalar_comparison/old_engine-32                              15.93Mi ± ∞ ¹   15.92Mi ± ∞ ¹       ~ (p=0.063 n=5)
RangeQuery/vector_and_scalar_comparison/new_engine-32                              19.97Mi ± ∞ ¹   19.96Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/positive_offset_vector/old_engine-32                                    14.13Mi ± ∞ ¹   14.13Mi ± ∞ ¹  -0.04% (p=0.008 n=5)
RangeQuery/positive_offset_vector/new_engine-32                                    18.67Mi ± ∞ ¹   18.66Mi ± ∞ ¹  -0.03% (p=0.008 n=5)
RangeQuery/at_modifier_/old_engine-32                                              24.99Mi ± ∞ ¹   24.98Mi ± ∞ ¹  -0.01% (p=0.008 n=5)
RangeQuery/at_modifier_/new_engine-32                                              16.03Mi ± ∞ ¹   16.03Mi ± ∞ ¹  -0.02% (p=0.008 n=5)
RangeQuery/at_modifier_with_positive_offset_vector/old_engine-32                   24.99Mi ± ∞ ¹   24.99Mi ± ∞ ¹  -0.01% (p=0.008 n=5)
RangeQuery/at_modifier_with_positive_offset_vector/new_engine-32                   16.03Mi ± ∞ ¹   16.03Mi ± ∞ ¹  -0.02% (p=0.008 n=5)
RangeQuery/clamp/old_engine-32                                                     15.99Mi ± ∞ ¹   15.97Mi ± ∞ ¹       ~ (p=0.413 n=5)
RangeQuery/clamp/new_engine-32                                                     19.88Mi ± ∞ ¹   19.87Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/clamp_min/old_engine-32                                                 15.98Mi ± ∞ ¹   15.97Mi ± ∞ ¹       ~ (p=0.278 n=5)
RangeQuery/clamp_min/new_engine-32                                                 19.88Mi ± ∞ ¹   19.87Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/complex_func_query/old_engine-32                                        17.14Mi ± ∞ ¹   17.08Mi ± ∞ ¹       ~ (p=0.087 n=5)
RangeQuery/complex_func_query/new_engine-32                                        20.84Mi ± ∞ ¹   20.83Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/func_within_func_query/old_engine-32                                    3.159Mi ± ∞ ¹   3.149Mi ± ∞ ¹  -0.32% (p=0.008 n=5)
RangeQuery/func_within_func_query/new_engine-32                                    7.160Mi ± ∞ ¹   7.212Mi ± ∞ ¹  +0.73% (p=0.008 n=5)
RangeQuery/aggr_within_func_query/old_engine-32                                    3.158Mi ± ∞ ¹   3.148Mi ± ∞ ¹  -0.32% (p=0.008 n=5)
RangeQuery/aggr_within_func_query/new_engine-32                                    6.931Mi ± ∞ ¹   6.984Mi ± ∞ ¹  +0.76% (p=0.008 n=5)
RangeQuery/histogram_quantile/old_engine-32                                        22.24Mi ± ∞ ¹   22.25Mi ± ∞ ¹       ~ (p=0.190 n=5)
RangeQuery/histogram_quantile/new_engine-32                                        37.58Mi ± ∞ ¹   37.59Mi ± ∞ ¹  +0.03% (p=0.008 n=5)
RangeQuery/sort/old_engine-32                                                      16.01Mi ± ∞ ¹   16.00Mi ± ∞ ¹       ~ (p=0.111 n=5)
RangeQuery/sort/new_engine-32                                                      19.39Mi ± ∞ ¹   19.38Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/sort_desc/old_engine-32                                                 16.01Mi ± ∞ ¹   16.00Mi ± ∞ ¹       ~ (p=0.373 n=5)
RangeQuery/sort_desc/new_engine-32                                                 19.39Mi ± ∞ ¹   19.38Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/absent_and_exists/old_engine-32                                         4.023Mi ± ∞ ¹   4.013Mi ± ∞ ¹       ~ (p=0.500 n=5)
RangeQuery/absent_and_exists/new_engine-32                                         7.181Mi ± ∞ ¹   7.171Mi ± ∞ ¹  -0.14% (p=0.008 n=5)
RangeQuery/absent_and_doesnt_exist/old_engine-32                                   66.21Ki ± ∞ ¹   66.21Ki ± ∞ ¹       ~ (p=0.484 n=5)
RangeQuery/absent_and_doesnt_exist/new_engine-32                                   178.6Ki ± ∞ ¹   178.6Ki ± ∞ ¹  -0.01% (p=0.032 n=5)
RangeQuery/double_exponential_smoothing/old_engine-32                              81.16Mi ± ∞ ¹   81.15Mi ± ∞ ¹  -0.01% (p=0.008 n=5)
RangeQuery/double_exponential_smoothing/new_engine-32                              20.81Mi ± ∞ ¹   20.86Mi ± ∞ ¹  +0.25% (p=0.008 n=5)
RangeQuery/count_over_time_5m/old_engine-32                                        14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/count_over_time_5m/new_engine-32                                        21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/count_over_time_1h/old_engine-32                                        14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/count_over_time_1h/new_engine-32                                        41.32Mi ± ∞ ¹   41.37Mi ± ∞ ¹  +0.13% (p=0.008 n=5)
RangeQuery/count_over_time_6h/old_engine-32                                        14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/count_over_time_6h/new_engine-32                                        64.69Mi ± ∞ ¹   64.75Mi ± ∞ ¹  +0.08% (p=0.008 n=5)
RangeQuery/sum_over_time_5m/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/sum_over_time_5m/new_engine-32                                          21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/sum_over_time_1h/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/sum_over_time_1h/new_engine-32                                          41.32Mi ± ∞ ¹   41.37Mi ± ∞ ¹  +0.13% (p=0.008 n=5)
RangeQuery/sum_over_time_6h/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/sum_over_time_6h/new_engine-32                                          64.69Mi ± ∞ ¹   64.75Mi ± ∞ ¹  +0.08% (p=0.008 n=5)
RangeQuery/avg_over_time_5m/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.06% (p=0.008 n=5)
RangeQuery/avg_over_time_5m/new_engine-32                                          21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/avg_over_time_1h/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/avg_over_time_1h/new_engine-32                                          41.32Mi ± ∞ ¹   41.37Mi ± ∞ ¹  +0.13% (p=0.008 n=5)
RangeQuery/avg_over_time_6h/old_engine-32                                          14.95Mi ± ∞ ¹   14.94Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/avg_over_time_6h/new_engine-32                                          64.69Mi ± ∞ ¹   64.75Mi ± ∞ ¹  +0.08% (p=0.008 n=5)
RangeQuery/min_over_time_5m/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/min_over_time_5m/new_engine-32                                          21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/min_over_time_1h/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/min_over_time_1h/new_engine-32                                          41.32Mi ± ∞ ¹   41.37Mi ± ∞ ¹  +0.13% (p=0.008 n=5)
RangeQuery/min_over_time_6h/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/min_over_time_6h/new_engine-32                                          64.69Mi ± ∞ ¹   64.75Mi ± ∞ ¹  +0.08% (p=0.008 n=5)
RangeQuery/max_over_time_5m/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/max_over_time_5m/new_engine-32                                          21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/max_over_time_1h/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/max_over_time_1h/new_engine-32                                          41.32Mi ± ∞ ¹   41.37Mi ± ∞ ¹  +0.13% (p=0.008 n=5)
RangeQuery/max_over_time_6h/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/max_over_time_6h/new_engine-32                                          64.69Mi ± ∞ ¹   64.75Mi ± ∞ ¹  +0.08% (p=0.008 n=5)
RangeQuery/stddev_over_time_5m/old_engine-32                                       14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/stddev_over_time_5m/new_engine-32                                       21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/stddev_over_time_1h/old_engine-32                                       14.95Mi ± ∞ ¹   14.94Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/stddev_over_time_1h/new_engine-32                                       41.32Mi ± ∞ ¹   41.37Mi ± ∞ ¹  +0.13% (p=0.008 n=5)
RangeQuery/stddev_over_time_6h/old_engine-32                                       14.97Mi ± ∞ ¹   14.96Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/stddev_over_time_6h/new_engine-32                                       64.69Mi ± ∞ ¹   64.75Mi ± ∞ ¹  +0.08% (p=0.008 n=5)
RangeQuery/stdvar_over_time/old_engine-32                                          14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/stdvar_over_time/new_engine-32                                          21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/last_over_time/old_engine-32                                            14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/last_over_time/new_engine-32                                            21.56Mi ± ∞ ¹   21.62Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
RangeQuery/present_over_time/old_engine-32                                         14.94Mi ± ∞ ¹   14.93Mi ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/present_over_time/new_engine-32                                         21.91Mi ± ∞ ¹   21.96Mi ± ∞ ¹  +0.24% (p=0.008 n=5)
geomean                                                                            14.67Mi         14.67Mi        +0.02%
¹ need >= 6 samples for confidence interval at level 0.95

                                                                 │ benchmarks/range_baseline.out │     benchmarks/range_change.out      │
                                                                 │           allocs/op           │  allocs/op    vs base                │
RangeQuery/vector_selector/old_engine-32                                            24.11k ± ∞ ¹   24.11k ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/vector_selector/new_engine-32                                            41.77k ± ∞ ¹   41.77k ± ∞ ¹       ~ (p=0.968 n=5)
RangeQuery/sum/old_engine-32                                                        21.13k ± ∞ ¹   21.13k ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/sum/new_engine-32                                                        38.94k ± ∞ ¹   38.94k ± ∞ ¹       ~ (p=0.238 n=5)
RangeQuery/sum_by_pod/old_engine-32                                                 23.16k ± ∞ ¹   23.16k ± ∞ ¹       ~ (p=0.738 n=5)
RangeQuery/sum_by_pod/new_engine-32                                                 56.99k ± ∞ ¹   56.99k ± ∞ ¹       ~ (p=0.444 n=5)
RangeQuery/topk/old_engine-32                                                       22.36k ± ∞ ¹   22.36k ± ∞ ¹       ~ (p=0.937 n=5)
RangeQuery/topk/new_engine-32                                                       39.68k ± ∞ ¹   39.68k ± ∞ ¹       ~ (p=0.397 n=5)
RangeQuery/bottomk/old_engine-32                                                    22.12k ± ∞ ¹   22.12k ± ∞ ¹       ~ (p=0.190 n=5)
RangeQuery/bottomk/new_engine-32                                                    39.68k ± ∞ ¹   39.68k ± ∞ ¹       ~ (p=0.317 n=5)
RangeQuery/limitk/old_engine-32                                                     38.87k ± ∞ ¹   38.78k ± ∞ ¹       ~ (p=0.119 n=5)
RangeQuery/limitk/new_engine-32                                                     56.44k ± ∞ ¹   56.34k ± ∞ ¹  -0.16% (p=0.008 n=5)
RangeQuery/limit_ratio/old_engine-32                                                41.62k ± ∞ ¹   41.53k ± ∞ ¹       ~ (p=0.087 n=5)
RangeQuery/limit_ratio/new_engine-32                                                194.2k ± ∞ ¹   194.1k ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/rate/old_engine-32                                                       41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/rate/new_engine-32                                                       79.83k ± ∞ ¹   79.74k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/rate_with_longer_window/old_engine-32                                    41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/rate_with_longer_window/new_engine-32                                    78.52k ± ∞ ¹   78.43k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/subquery/old_engine-32                                                   47.19k ± ∞ ¹   47.10k ± ∞ ¹  -0.19% (p=0.008 n=5)
RangeQuery/subquery/new_engine-32                                                   100.5k ± ∞ ¹   100.4k ± ∞ ¹  -0.09% (p=0.008 n=5)
RangeQuery/sum_rate/old_engine-32                                                   38.16k ± ∞ ¹   38.07k ± ∞ ¹  -0.22% (p=0.040 n=5)
RangeQuery/sum_rate/new_engine-32                                                   77.00k ± ∞ ¹   76.91k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/sum_by_rate/old_engine-32                                                40.19k ± ∞ ¹   40.10k ± ∞ ¹  -0.22% (p=0.040 n=5)
RangeQuery/sum_by_rate/new_engine-32                                                95.05k ± ∞ ¹   94.96k ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/quantile_with_variable_parameter/old_engine-32                           1.042M ± ∞ ¹   1.042M ± ∞ ¹       ~ (p=0.087 n=5)
RangeQuery/quantile_with_variable_parameter/new_engine-32                           159.9k ± ∞ ¹   159.7k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/binary_operation_with_one_to_one/old_engine-32                           30.57k ± ∞ ¹   30.54k ± ∞ ¹       ~ (p=0.095 n=5)
RangeQuery/binary_operation_with_one_to_one/new_engine-32                           48.00k ± ∞ ¹   47.97k ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/binary_operation_with_many_to_one/old_engine-32                          545.9k ± ∞ ¹   545.9k ± ∞ ¹       ~ (p=0.103 n=5)
RangeQuery/binary_operation_with_many_to_one/new_engine-32                          87.34k ± ∞ ¹   87.28k ± ∞ ¹  -0.08% (p=0.008 n=5)
RangeQuery/binary_operation_with_vector_and_scalar/old_engine-32                    41.15k ± ∞ ¹   41.06k ± ∞ ¹       ~ (p=0.063 n=5)
RangeQuery/binary_operation_with_vector_and_scalar/new_engine-32                    67.97k ± ∞ ¹   67.88k ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/unary_negation/old_engine-32                                             41.12k ± ∞ ¹   41.03k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/unary_negation/new_engine-32                                             67.81k ± ∞ ¹   67.72k ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/vector_and_scalar_comparison/old_engine-32                               41.14k ± ∞ ¹   41.05k ± ∞ ¹       ~ (p=0.063 n=5)
RangeQuery/vector_and_scalar_comparison/new_engine-32                               58.96k ± ∞ ¹   58.87k ± ∞ ¹  -0.16% (p=0.008 n=5)
RangeQuery/positive_offset_vector/old_engine-32                                     32.44k ± ∞ ¹   32.38k ± ∞ ¹  -0.18% (p=0.008 n=5)
RangeQuery/positive_offset_vector/new_engine-32                                     50.10k ± ∞ ¹   50.04k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/at_modifier_/old_engine-32                                               50.78k ± ∞ ¹   50.75k ± ∞ ¹  -0.06% (p=0.008 n=5)
RangeQuery/at_modifier_/new_engine-32                                               42.31k ± ∞ ¹   42.28k ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/at_modifier_with_positive_offset_vector/old_engine-32                    50.79k ± ∞ ¹   50.76k ± ∞ ¹  -0.06% (p=0.008 n=5)
RangeQuery/at_modifier_with_positive_offset_vector/new_engine-32                    42.31k ± ∞ ¹   42.28k ± ∞ ¹  -0.08% (p=0.008 n=5)
RangeQuery/clamp/old_engine-32                                                      41.17k ± ∞ ¹   41.08k ± ∞ ¹       ~ (p=0.087 n=5)
RangeQuery/clamp/new_engine-32                                                      67.87k ± ∞ ¹   67.78k ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/clamp_min/old_engine-32                                                  41.16k ± ∞ ¹   41.06k ± ∞ ¹       ~ (p=0.087 n=5)
RangeQuery/clamp_min/new_engine-32                                                  67.86k ± ∞ ¹   67.76k ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/complex_func_query/old_engine-32                                         41.27k ± ∞ ¹   41.17k ± ∞ ¹       ~ (p=0.087 n=5)
RangeQuery/complex_func_query/new_engine-32                                         77.10k ± ∞ ¹   77.00k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/func_within_func_query/old_engine-32                                     38.22k ± ∞ ¹   38.13k ± ∞ ¹  -0.24% (p=0.008 n=5)
RangeQuery/func_within_func_query/new_engine-32                                     78.65k ± ∞ ¹   78.56k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/aggr_within_func_query/old_engine-32                                     38.22k ± ∞ ¹   38.13k ± ∞ ¹  -0.24% (p=0.008 n=5)
RangeQuery/aggr_within_func_query/new_engine-32                                     78.65k ± ∞ ¹   78.56k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/histogram_quantile/old_engine-32                                         152.9k ± ∞ ¹   152.9k ± ∞ ¹       ~ (p=0.651 n=5)
RangeQuery/histogram_quantile/new_engine-32                                         277.8k ± ∞ ¹   277.8k ± ∞ ¹       ~ (p=0.603 n=5)
RangeQuery/sort/old_engine-32                                                       41.62k ± ∞ ¹   41.53k ± ∞ ¹       ~ (p=0.111 n=5)
RangeQuery/sort/new_engine-32                                                       58.77k ± ∞ ¹   58.68k ± ∞ ¹  -0.16% (p=0.008 n=5)
RangeQuery/sort_desc/old_engine-32                                                  41.62k ± ∞ ¹   41.53k ± ∞ ¹       ~ (p=0.063 n=5)
RangeQuery/sort_desc/new_engine-32                                                  58.77k ± ∞ ¹   58.68k ± ∞ ¹  -0.15% (p=0.008 n=5)
RangeQuery/absent_and_exists/old_engine-32                                          38.13k ± ∞ ¹   38.04k ± ∞ ¹       ~ (p=0.087 n=5)
RangeQuery/absent_and_exists/new_engine-32                                          55.79k ± ∞ ¹   55.70k ± ∞ ¹  -0.16% (p=0.008 n=5)
RangeQuery/absent_and_doesnt_exist/old_engine-32                                     587.0 ± ∞ ¹    587.0 ± ∞ ¹       ~ (p=1.000 n=5) ²
RangeQuery/absent_and_doesnt_exist/new_engine-32                                    1.439k ± ∞ ¹   1.438k ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/double_exponential_smoothing/old_engine-32                               1.487M ± ∞ ¹   1.487M ± ∞ ¹  -0.01% (p=0.008 n=5)
RangeQuery/double_exponential_smoothing/new_engine-32                               73.86k ± ∞ ¹   73.77k ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/count_over_time_5m/old_engine-32                                         41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/count_over_time_5m/new_engine-32                                         76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/count_over_time_1h/old_engine-32                                         41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/count_over_time_1h/new_engine-32                                         85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/count_over_time_6h/old_engine-32                                         41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/count_over_time_6h/new_engine-32                                         85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/sum_over_time_5m/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/sum_over_time_5m/new_engine-32                                           76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/sum_over_time_1h/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/sum_over_time_1h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/sum_over_time_6h/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/sum_over_time_6h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/avg_over_time_5m/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/avg_over_time_5m/new_engine-32                                           76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/avg_over_time_1h/old_engine-32                                           41.14k ± ∞ ¹   41.05k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/avg_over_time_1h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/avg_over_time_6h/old_engine-32                                           41.14k ± ∞ ¹   41.05k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/avg_over_time_6h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/min_over_time_5m/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/min_over_time_5m/new_engine-32                                           76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/min_over_time_1h/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/min_over_time_1h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/min_over_time_6h/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/min_over_time_6h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/max_over_time_5m/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/max_over_time_5m/new_engine-32                                           76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/max_over_time_1h/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/max_over_time_1h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/max_over_time_6h/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/max_over_time_6h/new_engine-32                                           85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/stddev_over_time_5m/old_engine-32                                        41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/stddev_over_time_5m/new_engine-32                                        76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/stddev_over_time_1h/old_engine-32                                        41.14k ± ∞ ¹   41.05k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/stddev_over_time_1h/new_engine-32                                        85.83k ± ∞ ¹   85.75k ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/stddev_over_time_6h/old_engine-32                                        41.15k ± ∞ ¹   41.06k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/stddev_over_time_6h/new_engine-32                                        85.84k ± ∞ ¹   85.75k ± ∞ ¹  -0.10% (p=0.008 n=5)
RangeQuery/stdvar_over_time/old_engine-32                                           41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/stdvar_over_time/new_engine-32                                           76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
RangeQuery/last_over_time/old_engine-32                                             41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/last_over_time/new_engine-32                                             67.83k ± ∞ ¹   67.74k ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/present_over_time/old_engine-32                                          41.13k ± ∞ ¹   41.04k ± ∞ ¹  -0.22% (p=0.008 n=5)
RangeQuery/present_over_time/new_engine-32                                          76.83k ± ∞ ¹   76.74k ± ∞ ¹  -0.12% (p=0.008 n=5)
geomean                                                                             54.69k         54.62k        -0.14%
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

(26-07-14 7:27:22) <0> [~/workplace/promql-engine]  
dev-dsk-paurushg-2a-87a5ee42 %

@PaurushGarg
PaurushGarg marked this pull request as ready for review July 14, 2026 14:20
yeya24
yeya24 reviewed Jul 14, 2026
View reviewed changes

@yeya24 yeya24 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think we might miss some fuzz tests. Can we revisit the existing fuzzy tests in this repo and make sure it has set batch size enabled so this code change is exercised? I want to understand if there is any edge case we might hit that breaks the series batch optimizer

Comment thread storage/prometheus/matrix_selector.go

if o.opts.IsInstantQuery() {
scanner.buffer.Reset(math.MaxInt64, 0)
scanner.iterator = nil

@yeya24 yeya24 Jul 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One further improvement. If we want to reduce allocation further, we don't have to eagerly allocate the iterator upfront. Can reuse the iterator created for the previous scanner. Need to be careful if it works for us.

	for i := firstSeries; i < lastInBatch; i++ {
		if o.scanners[i].iterator == nil {
			o.scanners[i].iterator = o.scanners[i].rawSeries.Iterator(nil)
			o.scanners[i].buffer = o.newBuffer(ctx)
		}
	}

@PaurushGarg PaurushGarg Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Added the TODO.

Comment thread engine/bench_test.go
Comment thread logicalplan/set_batch_size.go
@fpetkovski

fpetkovski commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

I merged #718 which seems to have created a conflict but it should be easy to resolve. That PR removes a few fields from the matrix selector.

@PaurushGarg
storage,logicalplan: lazy iterator allocation with batching for range…
84c1a8a 
… functions

Two changes that together fix ruler OOM from unbounded iterator allocation:

1. logicalplan: enable series batching for range vector functions

   The SelectorBatchSize optimizer previously disabled batching for ALL
   FunctionCall nodes. Range vector functions (rate, increase,
   present_over_time, etc.) are per-series independent — each series'
   result depends only on that series' own range data. This change only
   disables batching for instant vector functions (no matrix arg) while
   allowing it for range vector functions.

2. storage: defer iterator and buffer allocation to batch time

   Previously, loadSeries() called s.Iterator(nil) and newBuffer() for
   ALL series upfront. With high-cardinality tenants (4.5M series), this
   allocates 4.5M merge iterators simultaneously, causing 22GB+ heap
   and OOM kills on ruler pods.

   Move iterator and buffer allocation from loadSeries() to Next(),
   where they are initialized per-batch. With seriesBatchSize=256, only
   256 merge iterators exist at any time instead of the full series count.

Together: the optimizer enables batching for the OOM-causing query
patterns, and the lazy allocation makes batching actually reduce peak
memory from O(N) to O(batchSize).

Signed-off-by: Paurush Garg <paurushg@amazon.com>
@PaurushGarg
PaurushGarg force-pushed the lazy-iterator-allocation branch from 44818b6 to 84c1a8a Compare July 18, 2026 00:02
@PaurushGarg

PaurushGarg commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

I merged #718 which seems to have created a conflict but it should be easy to resolve. That PR removes a few fields from the matrix selector.

@fpetkovski Thanks. Updated.

@PaurushGarg

PaurushGarg commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

Overall I think we might miss some fuzz tests. Can we revisit the existing fuzzy tests in this repo and make sure it has set batch size enabled so this code change is exercised? I want to understand if there is any edge case we might hit that breaks the series batch optimizer

@yeya24 Thanks. I added SelectorBatchSize: 256 to all 4 fuzz tests.

@PaurushGarg
PaurushGarg requested a review from yeya24 July 18, 2026 00:06
@yeya24
yeya24 merged commit 4e111a7 into thanos-io:main Jul 22, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yeya24 yeya24 yeya24 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@PaurushGarg @fpetkovski @yeya24