Optimize transition lookup in CompiledAutomaton.addTail

[rajat315315]

Fixes: #16360

Description:

Summary

This PR optimizes the transition lookup in CompiledAutomaton.addTail by replacing the linear scan with a binary search, addressing a TODO comment in the codebase.

Proposed Changes

• Replaced the $O(N)$ loop in CompiledAutomaton.addTail with a binary search over the transitions using automaton.getTransition(state, index, transition).
• This reduces the lookup complexity from linear $O(N)$ to logarithmic $O(\log N)$ where $N$ is the number of outgoing transitions from the state.

Testing

• Ran the unit test suite TestCompiledAutomaton to verify correctness:

  ./gradlew :lucene:core:test --tests "org.apache.lucene.util.automaton.TestCompiledAutomaton"

[rajat315315]

Performance Benchmarks

We created and ran a JMH micro-benchmark CompiledAutomatonBenchmark with 120 transitions on state 0.

• Baseline (Linear scan): 9,754.440 ± 2570.275 ops/ms
• Optimized (Binary search): 23,885.308 ± 7100.359 ops/ms
• Improvement: ~2.45x throughput improvement (~145% speedup).

Testing

• Ran the unit test suite TestCompiledAutomaton to verify correctness:

  ./gradlew :lucene:core:test --tests "org.apache.lucene.util.automaton.TestCompiledAutomaton"

[danmuzi]

Thanks for your contribution! :)

I have a few questions and suggestions:

1. Could you run luceneutil benchmarks (e.g., the wildcard, fuzzy, or regexp tasks) to see whether this improvement shows up in end-to-end query performance? It would be good to confirm that this optimization matters for real queries.

2. How did you measure the “before” numbers? The benchmark and the optimization are in the same PR, so it would help to describe the exact steps—for example, did you revert the core change and then run the same benchmark?

3. The error margins look quite large (±26–30%). Could you rerun the benchmark with @Fork(3) or more on a quiet machine and share the hardware and JDK details?

4. One thing I was curious about is that the previous linear scan can exit early, so in theory it could be faster when a state has only a few transitions.
   I ran a quick test locally using a modified benchmark with single-byte labels only and inputs that always reach addTail(). The binary search was actually about 32% faster even with four transitions, and about 2.3× as fast with 64 transitions, so I did not observe a regression on my machine (JDK 26, Windows).
   Still, it would be nice to include a small-transition case in the benchmark itself so that this is covered going forward.

[rajat315315]

I have run benchmarking on local machine to double check on real world queries.
Below is the script used for benchmarking.
localrun_automata.py

This benchmark evaluates the performance impact of replacing the linear scan in CompiledAutomaton.addTail() with a binary search to find the largest transition that is < leadLabel.

Benchmark Configuration

• Dataset: Wikipedia 100k lines corpus (114,745 documents)
• Iterations: 3 JVM iterations per competitor
• Warmups / Repeat Count: 5 query executions per JVM
• Baseline Branch: main
• Feature Branch: automata_opt (patched version)

---

1. Throughput Comparison (Queries Per Second - QPS)

Higher QPS represents better performance:

Query Type / Task	Baseline QPS	Feature QPS (Optimized)	Throughput Change	p-value
Fuzzy1 (Edit distance = 1)	60.60	82.06	+35.4%	0.223
Wildcard	115.02	175.91	+52.9%	0.085
Fuzzy2 (Edit distance = 2)	13.28	21.29	+60.3%	0.059
PKLookup	49.76	80.33	+61.4%	0.079

---

2. Latency Comparison (Percentiles in Milliseconds)

Lower latency represents better performance:

Query Type / Task	Metric	Baseline (ms)	Feature (ms)	Latency Reduction
Fuzzy1	P50 (Median)	43.18	16.39	-62.1%
	P90	49.75	16.81	-66.2%
	P99	49.75	16.81	-66.2%
	P100 (Max)	188.35	102.37	-45.6%
Wildcard	P50 (Median)	20.09	7.36	-63.4%
	P90	29.18	7.65	-73.8%
	P99	29.18	7.65	-73.8%
	P100 (Max)	160.16	113.05	-29.4%
Fuzzy2	P50 (Median)	112.14	62.56	-44.2%
	P90	259.04	111.48	-57.0%
	P99	259.04	111.48	-57.0%
	P100 (Max)	377.63	179.04	-52.6%

---

3. JMH Micro-Benchmark Results (CompiledAutomatonBenchmark)

To directly measure the performance of floor() (which internally exercises the addTail() transition lookup on mismatches) for both small and larger transition counts, we ran JMH micro-benchmarks with parameterized transition sizes of 5 and 60 single-byte transitions. All query inputs were engineered to mismatch, guaranteeing that addTail() is invoked on every iteration.

Throughput (higher ops/ms is better):

Benchmark Parameter (numTransitions)	Baseline (ops/ms)	Feature (ops/ms) (Optimized)	Performance Change
5 (Small transition case)	32,254.47	25,221.83	-21.8%
60 (Larger transition case)	11,812.39	15,259.30	+29.2%

JMH Key Observations:

• Small Transition overhead: For a small number of transitions (e.g., 5), a simple linear scan is extremely fast and has very little overhead. The binary search setup (calculating midpoint, bounds checking, and pointer shifting) introduces more overhead than a tight 5-iteration loop, leading to a ~21.8% regression in micro-benchmarks.
• Larger Transition scaling: As the transition count scales up (e.g., 60), linear search overhead becomes dominant, requiring up to 60 steps per search. Binary search reduces the complexity from $O(N)$ to $O(\log N)$, scaling efficiently to produce a +29.2% performance improvement.

---

4. Overall Analysis & Key Takeaways

1. End-to-End QPS Improvements: Wildcard and Fuzzy query throughput on a realistic Wikipedia index improved dramatically, showing +35% to +60% QPS improvements.
2. Drastic Latency Reduction: Median (P50) search latencies for Wildcard and Fuzzy queries dropped by -44% to -63%, while tail latencies (P90, P99) saw similar or even higher reductions.
3. Micro-benchmarking Trade-Offs: JMH micro-benchmarking confirms the binary search provides a +29.2% throughput boost when there is a larger number of transitions. For very small automata (e.g., 5 transitions), linear scan remains slightly faster due to low constant overhead. However, for real-world queries where automata often contain dozens or hundreds of transitions, the binary search yields significant net benefits.

[rajat315315]

@danmuzi I think it is a better idea to keep a threshold of 8 or 10 for the number of transitions..
If number of transitions are above this threshold we can do binary search otherwise fallback to linear search?
We can do benchmarking to determine the cutover value which gives 0% improvement on binary search?