implement btree engine for aggregating string comparisons#50
Merged
Conversation
8 tasks
KiKoS0
force-pushed
the
riadh/optimize-string-comparisons-expr
branch
from
4bc0d52 to
00da5b9
Compare
KiKoS0
force-pushed
the
riadh/optimize-string-comparisons-expr
branch
from
00da5b9 to
19a1310
Compare
KiKoS0
added a commit
to inngest/inngest
that referenced
this pull request
May 13, 2026
## Description - **cache mixed expressions cel programs** - **eliminate per-eval allocations in CEL expression evaluation** - **cache attribute paths and CEL patterns per expression** - **implement btree engine for aggregating string comparisons**: inngest/expr#50 ``` go test -run=^$ -bench=BenchmarkPauseMixed ./pkg/expressions/expragg/ -benchtime=10000x -benchmem goos: darwin goarch: arm64 pkg: github.com/inngest/inngest/pkg/expressions/expragg cpu: Apple M4 Max BenchmarkPauseMixed-14 10000 60536 ns/op 50062 B/op 802 allocs/op --- BENCH: BenchmarkPauseMixed-14 pause_benchmark_test.go:176: Adding 1 expressions (useSameVersion=false, useMixedExpression=true)... pause_benchmark_test.go:205: Aggregate evaluator has 1 total evaluables (Fast: 1, Mixed: 0, Slow: 0) pause_benchmark_test.go:176: Adding 10000 expressions (useSameVersion=false, useMixedExpression=true)... pause_benchmark_test.go:205: Aggregate evaluator has 10000 total evaluables (Fast: 10000, Mixed: 0, Slow: 0) PASS ok github.com/inngest/inngest/pkg/expressions/expragg 1.174s ``` while before ``` go test -run=^$ -bench=BenchmarkPauseMixed ./pkg/expressions/expragg/ -benchtime=10000x -benchmem goos: darwin goarch: arm64 pkg: github.com/inngest/inngest/pkg/expressions/expragg cpu: Apple M4 Max BenchmarkPauseMixed-14 10000 5030243 ns/op 24819610 B/op 504199 allocs/op --- BENCH: BenchmarkPauseMixed-14 pause_benchmark_test.go:70: Adding 1 expressions (useSameVersion=false, useMixedExpression=true)... pause_benchmark_test.go:99: Aggregate evaluator has 1 total evaluables (Fast: 0, Mixed: 1, Slow: 0) pause_benchmark_test.go:70: Adding 10000 expressions (useSameVersion=false, useMixedExpression=true)... pause_benchmark_test.go:99: Aggregate evaluator has 10000 total evaluables (Fast: 0, Mixed: 10000, Slow: 0) PASS ok github.com/inngest/inngest/pkg/expressions/expragg 50.790s ``` ## Type of change (choose one) - [x] Chore (refactors, upgrades, etc.) - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] Security fix (non-breaking change that fixes a potential vulnerability) - [ ] Docs - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality not to work as expected) ## Checklist - [ ] I've linked any associated issues to this PR. - [ ] I've tested my own changes. *[Check our Pull Request Guidelines](https://github.com/inngest/inngest/blob/main/docs/PULL_REQUEST_GUIDELINES.md)* <!-- MENDRAL_SUMMARY --> --- > [!NOTE] > This PR optimizes CEL expression evaluation for pause matching via three strategies: (1) caching compiled `cel.Program` instances per expression string, (2) pre-computing attribute paths and `AttributePattern` slices at parse time to avoid per-evaluation allocations, and (3) refactoring `unknownDecorator` from plan-time to eval-time via the new `runtimeUnknownCall` struct. It also bumps `github.com/inngest/expr` to pick up a btree engine for string comparison aggregation, yielding an ~83× throughput improvement on the mixed-expression benchmark. > > <sup>Written by [Mendral](https://mendral.com) for commit b99fe80.</sup> <!-- /MENDRAL_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
String comparisons like
"2025-10-28T23:21:37.821Z" <= async.data.createdAtare lexicographically ordered, so they can be matched with the same B-tree strategy used for numbers; Without going through CEL at all.