feat(gfql): native polars label_node_hops on the plain BFS (#1741 groundwork) by lmeyerov · Pull Request #1746 · graphistry/pygraphistry

lmeyerov · 2026-07-20T09:02:59Z

What

Implements native label_node_hops / label_seeds in the polars hop's plain (shortest-path) BFS. Previously every hop-label request raised NotImplementedError on polars.

Why

This is step 1 of the real fix for #1741, replacing the interim decline in #1742.

#1741 is: on the polars engine, MATCH (a)-[*1..2]-(b) over-flags the alias — a seed re-reached by backtracking gets aliased where pandas leaves it unaliased. My first diagnosis ("polars lacks pandas' backtrack avoidance") was wrong: A/B shows both engines return identical node sets and identical final _nodes; only the alias flag diverges.

The actual pandas rule is a hop-distance gate (compute/chain.py ~452-503): for a named node preceded by an edge op, label_mask &= hop >= min_hop & hop <= max_hop. The seed's hop label is null, so it fails the gate and loses the alias. polars cannot run that gate because it has no hop labels at all — hence this PR.

Step 2 (separate PR) ports the gate itself into _apply_node_names and closes #1741; #1742 then closes unmerged.

The rule, and why it is direction-dependent

pandas labels differently per direction, and that asymmetry is the #1741 behavior:

direction	labeled set per hop	consequence
forward / reverse	every destination, first-wins (`hop.py:540`)	a seed re-entered at hop 1 is labeled 1
undirected	destinations minus everything already visited	a seed re-reached by backtracking stays null

label_seeds writes hop 0 for seeds and marks them seen (hop.py:361-363).

Verification

Differential fuzz against the pandas oracle: 4800/4800 exact matches over random graphs × {forward, reverse, undirected} × hops 1–4 × label_seeds {on, off}. Both asymmetric shapes are pinned as explicit named tests, not just fuzz coverage.

Full local run: 1277 passed across test_engine_polars_hop.py, test_compute_hops.py, test_engine_polars_chain.py. mypy clean, ruff clean. (Three test_engine_polars_conformance_matrix.py temporal failures in this environment reproduce identically on origin/master — pre-existing, unrelated.)

Still declined — honest NIE, no pandas bridge

label_edge_hops at every shape. With labels on, the pandas edge output duplicates an undirected edge traversed in both directions (448/4800 differential cases). That looks like a pandas artifact rather than intended semantics; baking it into polars is not parity worth having, and GFQL polars chain: undirected bounded varlen (-[:T*1..k]-) silently diverges from pandas — missing backtrack avoidance includes the seed #1741 needs only node labels. Worth a follow-up issue on the pandas side.
Labels under min_hops > 1, whose labels come from the layered backward walk (hop.py:676-778) — a different rule, not yet ported.

Note

test_polars_hop_unsupported_raises loses its label_node_hops and label_seeds cases — they are now supported. label_edge_hops and min_hops>1 stay in the decline list.

🤖 Generated with Claude Code

…undwork) The polars hop declined every hop-label request, which is why the polars chain cannot reproduce pandas' hop-distance alias gate — the root cause behind #1741 (undirected varlen + alias silently over-flags a backtracked-to seed). Emits node hop labels from the existing BFS layers. pandas' rule is direction-dependent, and the asymmetry is exactly the #1741 behavior: * forward/reverse — every destination of a hop is labeled, first-wins, so a seed re-entered at hop 1 IS labeled 1 (hop.py:540 new_node_ids); * undirected — destinations minus everything already visited, so a seed re-reached by walking back along its arrival edge stays NULL. label_seeds writes hop 0 for seeds and makes them "seen" (hop.py:361-363). Verified against the pandas oracle by differential fuzz: 4800/4800 exact matches over random graphs x {forward, reverse, undirected} x hops 1-4 x label_seeds {on, off}. Tests pin the two asymmetric shapes explicitly. Still declined (honest NIE, no pandas bridge): * label_edge_hops at every shape — with labels on, pandas DUPLICATES an undirected edge traversed in both directions (448/4800 differential cases); reproducing that artifact is not parity worth having, and #1741 needs only node labels. * labels under min_hops>1, whose labels come from the layered backward walk (hop.py:676-778), a different rule not yet ported. Label requests route to the eager loop rather than the single-hop lazy plan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

label_edge_hops declines unconditionally, so track_edge_hops was always False and the collect/attach machinery was unreachable — it showed up as the changed-line coverage miss. Node labels are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

lmeyerov · 2026-07-20T15:17:26Z

Cross-engine validation posted on the stack tip #1747: 4-engine differential sweep on dgx-spark, A/B vs clean master c355660a — 0 regressions, 70 of 98 mismatches fixed, polars and polars-gpu identical, pandas/cudf untouched at 144/144. Residual 28 (forward -[*2..3]->) is pre-existing on master, filed as #1748.

…+ label-col collision (#1741/#1746) Two correctness fixes to the native polars label_node_hops path, moved down from the stacked chain PR so #1746 is right standalone: 1. Undirected seed pre-seed (was in #1747). Without pre-seeding label_seen_nodes with the seeds, an undirected walk in wave-front mode (visited starts empty) re-labels a seed it backtracks into, and a seed filtered out of the frontier by source_node_match also gets re-labeled. pandas excludes all original seeds from undirected labeling. Differential vs pandas over direction x hops x return_as_wave_front x source_node_match x destination_node_match x 10 graphs: was 456/24, now 480/480. 2. Label column-name collision parity. A requested label_node_hops name that already exists on the node table is now redirected to `<name>_1` (matching pandas' resolve_label_col) instead of the polars left-join auto-suffix `<name>_right` / DuplicateError. Amplifies TestHopLabelsDifferential with the seed-filter / wave-front / multi-seed axes that were previously unvaried (exactly the diverging cases), plus a collision test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

A node named after a variable-length edge (`MATCH (a)-[*1..2]-(b)`) must carry its alias only when its hop distance falls inside the edge's [min_hop, max_hop] window (pandas chain.py:456-501). The polars chain had no distance to gate on, so it aliased a backtracked-to seed pandas leaves unaliased — silently wrong RETURN rows. Auto-injects the hop-distance label (name resolved against the user's node columns via generate_safe_column_name, so a user column literally named __gfql_chain_node_hop__ is never clobbered — a clash would crash the int/str compare) and applies pandas' alias window in _apply_node_names. The underlying hop-label correctness (undirected seed pre-seed) now lives in the base PR #1746. Retitled closes->refs #1741: forward/reverse min_hops>1 aliases still run ungated (a whole family of silently-wrong shapes) tracked as #1748 and pinned here with strict xfail tests so the gap is visible in CI and flips to failure the moment it lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

lmeyerov · 2026-07-20T15:39:34Z

Restructured after review — the hop-label correctness fix now lives HERE

The #1746 review correctly flagged that this PR, at its own head, shipped silently-wrong undirected hop labels whenever a seed was filtered out of the frontier (source_node_match) or the walk ran in wave-front mode (return_as_wave_front=True) — the node/edge set matched pandas but the labels diverged. Confirmed by direct A/B against the pandas oracle on the pre-fix head 4c981e2a:

direction × hops × return_as_wave_front × source_node_match × destination_node_match × 10 graphs = 480 cases
before: 456 agree / 24 disagree — every failure undirected + wave-front, seed re-labeled
after this commit: 480 / 480

The undirected seed pre-seed fix (previously stacked in #1747) has been moved down into this PR so #1746 is correct standalone, and a second parity fix was added: a label_node_hops name that collides with an existing column is now redirected to <name>_1 like pandas' resolve_label_col, instead of the polars left-join auto-suffix <name>_right / DuplicateError.

TestHopLabelsDifferential is amplified with exactly the seed-filter / wave-front / multi-seed axes that were previously unvaried, plus a collision test. Full polars hop suite: 325 passed, ruff clean.

… (review c1) Adds an explicit same-graph forward-vs-undirected label-asymmetry test (the subtle direction-dependent rule the hop_eager fix bakes in, now legible in one pin) and the CHANGELOG [Development]/Fixed entry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

A node named after a variable-length edge (`MATCH (a)-[*1..2]-(b)`) must carry its alias only when its hop distance falls inside the edge's [min_hop, max_hop] window (pandas chain.py:456-501). The polars chain had no distance to gate on, so it aliased a backtracked-to seed pandas leaves unaliased — silently wrong RETURN rows. Auto-injects the hop-distance label (name resolved against the user's node columns via generate_safe_column_name, so a user column literally named __gfql_chain_node_hop__ is never clobbered — a clash would crash the int/str compare) and applies pandas' alias window in _apply_node_names. The underlying hop-label correctness (undirected seed pre-seed) now lives in the base PR #1746. Retitled closes->refs #1741: forward/reverse min_hops>1 aliases still run ungated (a whole family of silently-wrong shapes) tracked as #1748 and pinned here with strict xfail tests so the gap is visible in CI and flips to failure the moment it lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

lmeyerov mentioned this pull request Jul 20, 2026

polars chain: forward/reverse varlen with min_hops>1 returns nodes outside the hop window (silent wrong) #1748

Open

lmeyerov merged commit e8bd4cf into master Jul 20, 2026
57 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(gfql): native polars label_node_hops on the plain BFS (#1741 groundwork)#1746

feat(gfql): native polars label_node_hops on the plain BFS (#1741 groundwork)#1746
lmeyerov merged 4 commits into
masterfrom
fix/1741-polars-hop-labels

lmeyerov commented Jul 20, 2026

Uh oh!

lmeyerov commented Jul 20, 2026

Uh oh!

lmeyerov commented Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

lmeyerov commented Jul 20, 2026

What

Why

The rule, and why it is direction-dependent

Verification

Still declined — honest NIE, no pandas bridge

Note

Uh oh!

lmeyerov commented Jul 20, 2026

Uh oh!

lmeyerov commented Jul 20, 2026

Restructured after review — the hop-label correctness fix now lives HERE

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant