Add scalar function performance baselines - #9136
Draft
connortsui20 wants to merge 1 commit into
Draft
Conversation
Merging this PR will not alter performance
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| 🆕 | Simulation | inline[4096] |
N/A | 61.6 µs | N/A |
| 🆕 | Simulation | inline[65536] |
N/A | 787.6 µs | N/A |
| 🆕 | Simulation | nullable_out_of_line[4096] |
N/A | 73.8 µs | N/A |
| 🆕 | Simulation | nullable_out_of_line[65536] |
N/A | 793.8 µs | N/A |
| 🆕 | Simulation | out_of_line[4096] |
N/A | 61.8 µs | N/A |
| 🆕 | Simulation | out_of_line[65536] |
N/A | 787.6 µs | N/A |
| 🆕 | Simulation | column_x_column_points |
N/A | 14.1 ms | N/A |
| 🆕 | Simulation | column_x_column_polygons |
N/A | 23.6 ms | N/A |
| 🆕 | Simulation | constant_x_nullable_points |
N/A | 12.6 ms | N/A |
| 🆕 | Simulation | constant_x_nullable_polygons_disjoint |
N/A | 17.9 ms | N/A |
| 🆕 | Simulation | constant_x_points |
N/A | 13.7 ms | N/A |
| 🆕 | Simulation | constant_x_polygons_disjoint |
N/A | 17.8 ms | N/A |
| 🆕 | Simulation | constant_x_polygons_overlapping |
N/A | 2 s | N/A |
| 🆕 | Simulation | column_x_column_polygons |
N/A | 22.1 ms | N/A |
| 🆕 | Simulation | nullable_points_x_constant |
N/A | 12.7 ms | N/A |
| 🆕 | Simulation | nullable_polygons_disjoint_x_constant |
N/A | 16 ms | N/A |
| 🆕 | Simulation | points_x_constant |
N/A | 13.7 ms | N/A |
| 🆕 | Simulation | polygons_disjoint_x_constant |
N/A | 15.6 ms | N/A |
| 🆕 | Simulation | polygons_overlapping_x_constant |
N/A | 26.8 ms | N/A |
| 🆕 | Simulation | multipoint_non_nullable |
N/A | 65.5 ms | N/A |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Comparing ct/scalar-fn-baselines (880ddac) with ct/scalar-fn-factory-ext (bf43ecf)
Footnotes
-
12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
connortsui20
force-pushed
the
ct/scalar-fn-baselines
branch
from
August 1, 2026 17:49
1fb231a to
c82b8f3
Compare
connortsui20
force-pushed
the
ct/scalar-fn-baselines
branch
from
August 2, 2026 00:53
c82b8f3 to
71787a7
Compare
Adds stable CodSpeed benchmark names for `byte_length`, tensor scalar functions, and the geo binary predicates. The benchmarks use public construction and execution APIs, which lets later implementation changes compare directly against this baseline. CodSpeed runs tensor and geo in a dedicated shard. Each benchmark binary uses vendored `mimalloc` because scalar function execution allocates its output inside the timed trace. `L2Denorm` keeps its checked inherent constructor because `ScalarFnFactoryExt` cannot represent its normalized-row invariant. Signed-off-by: "Connor Tsui" <connor.tsui20@gmail.com> Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/scalar-fn-baselines
branch
from
August 2, 2026 18:03
71787a7 to
880ddac
Compare
Contributor
|
This benchmark has a too long runtime |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracking Issue: #9128
Adds stable scalar function performance baselines for
byte_length,l2_norm,l2_denorm,inner_product,cosine_similarity,GeoContains, andGeoIntersects.The benchmarks use the public construction and execution paths, with no
RowFninternals or forced execution strategies. The tensor arms vary width and nullability. The cosine arms also cover both constant-vector representations. The geo arms cover column and constant operands, mostly-disjoint and mostly-overlapping polygons, and nullable inputs.The
l2_denormarms measure decoding theL2Denormarray encoding rather than a scalar function, since #9138 reclassified it. The benchmark and arm names are unchanged so CodSpeed can still line them up againstdevelop.The benchmark sizes are all comfortably above the roughly 146-instruction Divan harness floor from #9011. Each benchmark binary also uses vendored
mimalloc, since scalar function execution allocates its output inside the timed trace and glibc differences between runner images caused the flakes fixed in #8861.vortex-tensorandvortex-georun in a dedicated CodSpeed shard.CodSpeed can only compare a later implementation change when the same benchmark name already exists on
develop, so these baselines need to land before theRowFnports. This PR is stacked on the constructor standardization because the ordinary scalar functions useScalarFnFactoryExt.Checks:
cargo check -p vortex-array -p vortex-tensor -p vortex-geo --benchescargo bench -p vortex-array --bench byte_length -- --testcargo bench -p vortex-tensor --bench cosine_similarity --bench l2_norm --bench l2_denorm --bench inner_product -- --testcargo bench -p vortex-geo --bench binary_predicates -- --testbyte_lengthand all tensor benchmarks (byte-length medians stayed within 0.4%, tensor medians stayed within 3.3%)cargo clippy -p vortex-array -p vortex-tensor -p vortex-geo --all-targets --all-featurescargo clippy --all-targets --all-featurescargo +nightly fmt --allyamllint --strict -c .yamllint.yaml .github/workflows/codspeed.ymlgit diff --check