perf(gfql): Cypher seeded-lookup abstraction tax -- 10-800x over native g.hop() (chain-executor combine_steps, not index-miss)

[lmeyerov]

Summary (perf / planner-lowering)

On LDBC SNB SF0.1 seeded point/short queries, the g.gfql("MATCH...") cypher surface is 10-800x slower than the ideal native g.hop() path for the same query, same engine, same session -- even though the #1658 CSR index is resident and engaged. Worst case IS1 seed-lookup pandas: cypher 233.66ms vs native 0.28ms = 832x. Median cypher/native tax by engine: pandas 67x (max 832x, TAX 460ms @ic8), polars 10x, cudf 12x, polars-gpu 27x (full table: plans/gfql-benchmark-numbers/native-tax-sf01.md).

Because native hits sub-ms via #1658, this is not a core data-structure/kernel gap -- it is a compile+lowering+chain-executor abstraction tax (plan criterion 5(c)).

Root cause (evidence-backed, not inference)

Profiled on dgx-spark (pandas, image 26.02-gfql-polars, bench checkout @983d2aff) on an in-container Message->Person HAS_CREATOR graph (50k/200k/200k) with the resident #1658 index. Probe + raw output: plans/gfql-benchmark-numbers/scratch_tax_probe.py / scratch_tax_probe_output.txt; writeup: plans/gfql-benchmark-numbers/seeded-lookup-tax-rootcause.md.

The tax is the chain.py two-pass executor (forward wavefront + combine_steps reverse reconciliation) re-merging the FULL node/edge frames, plus full-column object/label filters -- NOT cypher parsing and NOT an index miss.

Evidence 1 -- decomposition (warm median ms)

Path	ms	vs bare-hop
IS5 bare g.hop() (native ceiling)	0.131	1x
IS5 native gfql([n,e,n]) chain, no type filters	14.532	111x
IS5 native gfql([n,e,n]) chain, typed	21.127	161x
IS5 cypher MATCH (m:Message {id})-[:HAS_CREATOR]->(p:Person)	29.086	222x
IS5 cypher compile-only (cached)	~0.000	0% of cypher
IS1 cypher node lookup	10.554	38x vs native 0.276

bare hop 0.13ms -> chain-machinery 14.5ms is the dominant jump (~111x, the combine_steps/boundary-merge cost). Typed filters +6.6ms; cypher projection/postprocess +8ms; parse/compile ~0ms (fully cached, cache_owner=g).

Evidence 2 -- plan diff (gfql_explain): the index engages through cypher

Native IS5 chain and cypher IS5 emit the identical index trace step -- the cypher path is not scanning:

IS5 native chain : used_index=true  step={"op":"hop","direction":"forward","hops":1,
                     "frontier_n":1,"seed_deg_sum":1,"path":"index",
                     "decision_reason":"frontier below cost gate -> index"}
IS5 cypher       : used_index=true  step={...IDENTICAL...}

(IS1 is node-only -> no hop -> used_index=false for BOTH; its 38x tax is pure filter_by_dict full-column compare + merge.) This refines #1658: on this build the cypher-lowered seeded hop does engage the index; the residual gap is the chain executor wrapped around it.

Evidence 3 -- cProfile hot frames (IS5 cypher, 5 calls, 0.221s cum)

0.211  chain.py:761 chain -> _chain_impl -> chain.py:547 _handle_boundary_calls
0.102  Engine.py:761 safe_merge            <-- 55 calls  (~46%)
0.100  pandas merge                        <-- 55 merges
0.089  chain.py:171 combine_steps          <-- reverse reconciliation pass (~40%)
0.036  pandas comp_method_OBJECT_ARRAY     <-- full-column label == filters (~16%)

combine_steps (chain.py:171) re-merges full_nodes/_edges per step to reconcile wavefront boundaries even when the seeded result is a single row. Cypher parsing does not appear warm.

Proposed fix direction

1. Seeded-chain fast path (biggest win): when a chain's hops already took the index (gfql_explain path=index, tiny frontier), skip the combine_steps full-frame reverse-reconciliation -- the index hop already returns the exact boundary set. Route straight through the bare-hop result (0.13ms) vs the two-pass executor (14.5ms).
2. Push :Label/[:REL] predicates into the seed, not a full-column post-scan (comp_method_OBJECT_ARRAY over the whole graph for a 1-seed query).
3. Make combine_steps cardinality-aware: gate the full-frame merges on frontier size, reusing the planner's existing frontier below cost gate signal.
4. Parse/compile already cached (~0ms warm) -- no work needed there.

Refs

• plans/gfql-benchmark-numbers/native-tax-sf01.md (4-engine tax table)
• plans/gfql-benchmark-numbers/seeded-lookup-tax-rootcause.md (this root cause)
• feat(gfql): physical adjacency indexes for O(degree) seeded traversal #1658 (CSR seeded-hop index)
• plan criterion 5(c)