Optimize XPath step

[tompng]

Refactor step so that nodeset materialization is deferred. Instead of building the full nodeset up front and filtering through predicates,
each axis returns a scan descriptor ([generator_name, generator_argument]), and step picks a scan strategy based on the predicates' shape.

Predicates are classified into three groups:

kind	examples	strategy passed to the generator
position-independent	[@a="1"], [name()="foo"], [@a=@b]	:uniq — emit deduplicated matching nodes
simple positional	[N], [position()=N], [position()>N], [position()<N]	[op, value] — positional scan with one comparison
complex / position-dependent	[position()*@a], [last()-1], ...	:nodesets — fall back to per-anchor nodesets + the previous evaluate_predicate pipeline

Mixed predicate lists are split: position-independent predicates before the first positional predicate are folded into the node test;
predicates after it are applied per-node on the result.

Each axis can implement zero, one, or all of the three strategies. If a strategy is not implemented, the generator falls back to producing
:nodesets and the common slow path (non_optimized_raw_nodesets_select) handles dedup / positional filtering on flattened nodesets — i.e. the
same behavior as before this PR.

This pull request adds fast paths for:

• descendant / descendant-or-self: :uniq (single DFS with a seen-set; this is what speeds up //a//a//a//a)
• ancestor / ancestor-or-self: :uniq (parent-chain walk with a seen-set)
• preceding-sibling / following-sibling: :uniq and [op, value] (sibling scan with anchor-index tracking)

Other axes (child, parent, self, attribute) keep the previous behavior via the fallback path; they can be optimized in
follow-ups without changing call sites.

Detail

For //a//a//a style queries, the previous code built nodesets keyed by each anchor, including the same descendant once per anchor. The new
:uniq path scans every node at most once per step.

For [position() > N] style predicates on wide trees (e.g. //a/preceding-sibling::*[position()>2]), we previously built the full
preceding-sibling nodeset for each anchor and then ran evaluate_predicate. The new [op, value] path scans children once per parent and uses
anchor-index bookkeeping to recover per-anchor positions.

Note: general XPath cannot be linear — e.g. *[position() * number(@a) % number(@b) = 1] is genuinely O(n²) — so the goal is only to fast-path
position-independent and simple-positional predicates.

Benchmark

DEPTH = 500
xml   = '<a>' * DEPTH + '</a>' * DEPTH
doc   = REXML::Document.new(xml)
WIDTH = 1000
xml_wide = '<root>' + '<child/>' * WIDTH + '</root>'
doc_wide = REXML::Document.new(xml_wide)

REXML::XPath.match(doc, "//a//a");
# processing time: 30.756939s → 0.126807s

REXML::XPath.match(doc_wide, "//*/preceding-sibling::*[position()=10]");
# processing time: 2.446333s → 0.083954s

Also fixes #25

[Copilot]

Pull request overview

This PR refactors XPath step evaluation in REXML::XPathParser to defer nodeset materialization and introduce axis “scan strategies” that can fast-path common predicate shapes (position-independent and simple positional predicates), improving performance for deep and wide-tree queries.

Changes:

• Reworked step to drive axis scans via [scanner_method, scanner_argument] and choose scanning strategies (:uniq, [op, value], :nodesets) based on predicate classification.
• Added optimized scanners for descendant(-or-self), ancestor(-or-self), and sibling axes, plus a shared fallback selection path.
• Added/updated XPath predicate and sibling-axis tests, including float literal predicates and positional comparisons.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
lib/rexml/xpath_parser.rb	Implements deferred scanning/strategy selection in step, predicate classification, and optimized axis scanners.
test/xpath/test_predicate.rb	Adds coverage for float-literal positional predicates and variable-as-position predicates.
test/xpath/test_base.rb	Adds test for following-sibling::* with position() < N and adjusts nested-predicate test behavior.
test/xpath/test_axis_preceding_sibling.rb	Adds tests for preceding-sibling:: with < and <= position predicates.

Comments suppressed due to low confidence (1)

lib/rexml/xpath_parser.rb:834

• descendant axis fast path calls raw_node.children when include_self is false without checking the node type or whether children exists. If the context node is not an element/document (e.g., an attribute node), this will raise NoMethodError; previously it would just yield an empty descendant set. Guard this branch so it only iterates children for element/document nodes (or use the same node_type check as in recursive).

        raw_nodes.each do |raw_node|
          if include_self
            recursive.call(raw_node)
          else
            raw_node.children.each(&recursive)
          end

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.