feat(gfql): native polars UNWIND of a carried collect() list column#1752
Merged
Conversation
Extend `unwind_polars` to explode a List-dtype column reference (a `collect()` output / carried binding), not just a scalar-literal list. This is the row- pipeline analogue of the IC6 `WITH collect(x) AS xs UNWIND xs AS y` shape: previously the second UNWIND declined (NIE) because the expr was an Identifier, not a ListLiteral, forcing engine='pandas'. Mirrors the pandas oracle (`RowPipelineMixin.unwind` list-column branch) exactly: empty-list and null cells contribute 0 rows; nulls WITHIN a list survive as real elements; the source column is retained and exploded values append as `as_`. Implemented as with_columns(copy) -> filter(list.len() > 0) -> explode, which also makes the result independent of the polars 2.0 `empty_as_null` default. Non-list columns, name collisions, unknown identifiers, and nested-list literals still decline (NIE) — no pandas bridge, no silent-wrong. NOTE (out of scope): the collect->UNWIND->MATCH re-entry form (IC6 with a trailing MATCH) is compile-time rewritten to whole-row MATCH-after-WITH re-entry; polars declines there because the whole-entity projection does not attach `_cypher_entity_projection_meta` and `binding_rows_polars` declines on `_gfql_start_nodes`. That is the separate WITH-MATCH re-entry residual, not a row-pipeline unwind gap. Correctness gate: differential fuzz vs pandas (2000 queries, 0 disagreements) plus parity/pin/decline tests. ruff clean; no new mypy errors; new lines fully covered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
... WITH collect(x) AS xs UNWIND xs AS y RETURN ...— exploding a list-valued column produced bycollect()— previously declined on polars (unwind_polarsaccepted only a scalar literal list). Now explodes aList-dtype column reference (with_columnscopy →filter(list.len() > 0)→explode), matching the pandas oracle exactly: empty-list/null cells → 0 rows, nulls within a list survive, source column retained; the pre-filter also makes it independent of the polars-2.0empty_as_nulldefault.Verification (independent — re-run by the reviewing agent)
g.gfql()A/B vs pandas, match:UNWIND [1,2,3] AS x WITH collect(x) AS xs UNWIND xs AS y RETURN y→ rows {1,2,3}; withcollect(DISTINCT ...); andRETURN count(y), sum(y)→ (4, 10).test_engine_polars_row_pipeline.py: 20 unwind/collect tests pass (independently re-run). ruff clean; mypy master-baseline; added lines covered.Scope
Non-list columns, name collisions, unknown identifiers, nested-list literals → honest NIE. The
MATCH ... WITH collect(..) UNWIND .. MATCHform (IC6 with a trailing re-traversal) is compile-rewritten into WITH→MATCH reentry — a separate path (#1751), not this PR.Off master (touches
row_pipeline.py::unwind_polars— overlaps #1750/#1751's file; trivial rebase). Refs the IC6/IC11 coverage push.🤖 Generated with Claude Code