feat(gfql/cypher): EXISTS { pattern } subqueries — declarative prune-isolated, native on 4 engines

CHANGELOG.md

@@ -9,6 +9,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 <!-- Do Not Erase This Section - Used for tracking unreleased changes -->
 
 ### Added
+- **GFQL Cypher `EXISTS { <pattern> }` pattern-existence subqueries (openCypher-standard), native on all four engines**: `WHERE EXISTS { (n)-[:R]->() }` and `WHERE NOT EXISTS { (n)--() }` now parse and run — the declarative prune-isolated building blocks for the streamgl-viz filter pipeline. An `EXISTS` body reuses the existing pattern-predicate lowering wholesale (`semi_apply_mark` / `anti_semi_apply` row ops), so pandas/cuDF worked immediately; the polars engine gains NATIVE lowerings for the semi-apply family (correlated key sets computed by the polars chain executor's named-flag columns; order-preserving `is_in` joins) plus `rows(binding_ops=...)` for the single-entity row table — previously all honest-NIE. Aliases introduced inside the braces are existentially scoped (`EXISTS { (n)--(m) }` allowed with `m` unbound outside — bare pattern predicates keep the conservative guard), inline property maps work, and the one supported inner `WHERE` form is endpoint inequality — `EXISTS { (n)--(m) WHERE m <> n }`, the drop-self-loop prune-isolated flavor (pandas/cuDF filter the correlated bindings; polars excludes self-loop edges, which is exactly the `m <> n` witness). Both prune flavors are oracle-pinned in the conformance matrix on a self-loop discriminator graph, 4-engine parity-or-NIE. Honest declines with clear errors: `EXISTS` in RETURN/WITH projections, general inner `WHERE`, multi-pattern bodies, full `MATCH..RETURN` subquery bodies, multi-alias correlation on polars.
 - **GFQL polars execution config is Python-settable and live**: `set_cpu_streaming(bool)` and `set_gpu_executor('in-memory'|'streaming')` in `graphistry.compute.gfql.lazy` (plus the public `GPU_EXECUTORS` options and `GpuExecutor` type) set the CPU-streaming / GPU-executor knobs from Python. They resolve **Python override > environment variable > default**, read **live** per collect — previously these were env-only (`GFQL_POLARS_CPU_STREAMING` / `GFQL_POLARS_GPU_EXECUTOR`) and frozen at import, so neither a Python setting nor a post-import env change took effect. `None` resets a setter to env/default.
 - **GFQL engine conversion honors the `validate`/`warn` convention**: `Engine.df_to_engine(df, engine, *, validate=, warn=)` threads the repo-wide `validate` (`'strict'`/`'strict-fast'`/`'autofix'`; `True`→strict, `False`→autofix) + `warn` protocol into the pandas→polars and pandas→cuDF converters. On a mixed-type object column that Arrow/polars/cuDF cannot represent, `strict` raises (`NotImplementedError` for polars, `ArrowConversionError` for cuDF) and `autofix` coerces the column to string and warns — the same convention as `plot()`/`upload()`. Each engine keeps its established default (polars `strict` = parity-or-raise; cuDF `autofix` = its shipped best-effort coercion, now `warn`-suppressible).
 - **GFQL Cypher numeric functions + `toLower`/`toUpper`/`lower`/`upper` (openCypher/neo4j/GQL-standard)**: added the standard scalar functions `floor`, `ceil` (alias `ceiling`, per Cypher 25 GQL conformance and the GQL grammar's `CEIL|CEILING`), `round(x)` / `round(x, precision)`, and `toLower` / `toUpper` plus their GQL-conformance aliases `lower` / `upper` (ISO GQL §20.24 character-string functions; neo4j accepts both spellings) — the idiomatic case-insensitive compare `WHERE toLower(n.name) = 'bob'` — across the Cypher `WHERE`/`RETURN` expression surface. Evaluated natively on pandas/cuDF and polars (differential-parity tested). **`round` follows neo4j's documented tie-breaking** (standards-vetted against the neo4j manual): precision 0 (and the 1-arg form) rounds ties toward **positive infinity** (`round(-1.5)` → `-1.0`, `round(2.5)` → `3.0`), and precision > 0 rounds ties **away from zero** (HALF_UP: `round(-1.55, 1)` → `-1.6`) — not the numpy/polars half-to-even defaults, which give wrong answers on ties. The `round(x, precision, mode)` 3-arg form is not yet supported. Complements the already-supported `abs`/`sqrt`/`sign` and chained comparisons (`WHERE 1 < n.age < 65`). The `^` exponentiation operator is deferred: standards vetting settled it as **left-associative** (the openCypher TCK pins left, and neo4j's shipped Cypher 5/25 parser left-folds `Pow` — the manual's "right to left" row is a docs bug), but the current conformance corpus marks it reject-expected, so re-adding it is a coordinated corpus change. `LIKE`/`ILIKE`/`BETWEEN` remain intentionally unimplemented (verified absent from both Cypher and ISO GQL — GQL's only `LIKE` is the unrelated `CREATE GRAPH TYPE … LIKE g` DDL).

docs/source/gfql/cypher.rst

@@ -271,6 +271,21 @@ WHERE Forms
 - Positive relationship-existence pattern predicates such as
   ``WHERE (n)-[:R]->()`` and variable-length existence checks such as
   ``WHERE (n)-[*]-()`` and ``WHERE (n)-[:R*2]->()``.
+- ``EXISTS { <pattern> }`` pattern-existence subqueries (openCypher-standard) in
+  WHERE position, e.g. ``WHERE EXISTS { (n)-[:R]->() }`` and ``WHERE NOT EXISTS
+  { (n)--() }`` — the declarative prune-isolated building blocks. Aliases
+  introduced inside the braces are existentially scoped (``EXISTS { (n)--(m) }``
+  is fine even when ``m`` is unbound outside), inline property maps work
+  (``EXISTS { (n)-[:R {w: 1}]->() }``), and the one supported inner ``WHERE``
+  form is endpoint inequality: ``EXISTS { (n)--(m) WHERE m <> n }`` — "has a
+  neighbor other than itself", i.e. the drop-self-loop prune-isolated flavor.
+  Runs natively on all four engines. Not yet supported (clear errors):
+  ``EXISTS`` in RETURN/WITH projections, general inner ``WHERE`` clauses,
+  multi-pattern bodies, full ``MATCH .. RETURN`` subquery bodies, and ``EXISTS``
+  inside ``GRAPH { }`` pipelines. For prune-isolated in GRAPH STATE (nodes AND
+  edges back), use edge patterns instead: ``GRAPH { MATCH (a)-[e]-(b) }`` keeps
+  every edge-touching node with ALL its edges (self-loops included); add
+  ``WHERE a.id <> b.id`` for the drop-self-loop variant.
 - Pattern predicates can be combined with row predicates in the current
   boolean subset, including ``AND`` / ``OR`` / ``XOR`` and ``NOT`` forms.
 

graphistry/compute/ast.py

@@ -1662,41 +1662,45 @@ def anti_semi_apply(
     *,
     binding_ops: List[Dict[str, Any]],
     join_aliases: Sequence[str],
+    neq: Optional[Sequence[str]] = None,
 ) -> ASTCall:
     """Filter active rows by removing rows matching a correlated pattern.
 
     ``binding_ops`` encodes the pattern to evaluate as bindings rows.
     ``join_aliases`` names shared aliases used as anti-join keys.
+    ``neq`` optionally names two pattern aliases whose bindings must DIFFER
+    (``EXISTS { (n)--(m) WHERE m <> n }`` — the viz drop-self prune flavor).
     """
-    return ASTCall(
-        "anti_semi_apply",
-        {
-            "binding_ops": binding_ops,
-            "join_aliases": list(join_aliases),
-        },
-    )
+    params: Dict[str, Any] = {
+        "binding_ops": binding_ops,
+        "join_aliases": list(join_aliases),
+    }
+    if neq is not None:
+        params["neq"] = list(neq)
+    return ASTCall("anti_semi_apply", params)
 
 
 def semi_apply_mark(
     *,
     binding_ops: List[Dict[str, Any]],
     join_aliases: Sequence[str],
     out_col: str,
+    neq: Optional[Sequence[str]] = None,
 ) -> ASTCall:
     """Annotate active rows with a correlated pattern-existence boolean.
 
     ``binding_ops`` encodes the pattern to evaluate as bindings rows.
     ``join_aliases`` names shared aliases used as join keys.
     ``out_col`` receives a bool marker where True means the pattern matched.
     """
-    return ASTCall(
-        "semi_apply_mark",
-        {
-            "binding_ops": binding_ops,
-            "join_aliases": list(join_aliases),
-            "out_col": out_col,
-        },
-    )
+    params: Dict[str, Any] = {
+        "binding_ops": binding_ops,
+        "join_aliases": list(join_aliases),
+        "out_col": out_col,
+    }
+    if neq is not None:
+        params["neq"] = list(neq)
+    return ASTCall("semi_apply_mark", params)
 
 
 def join_apply(

graphistry/compute/gfql/call/support.py

@@ -83,6 +83,17 @@ def is_projection_items(v: object) -> bool:
     return True
 
 
+def is_string_pair(v: object) -> bool:
+    """Exactly two non-empty strings — the semi-apply family's ``neq`` endpoint pair
+    (wave-1: a 1-item list crashed late with IndexError; a 3-item list silently
+    filtered on the first pair)."""
+    return (
+        isinstance(v, list)
+        and len(v) == 2
+        and all(isinstance(item, str) and item != "" for item in v)
+    )
+
+
 def is_non_empty_list_of_strings(v: object) -> bool:
     return isinstance(v, list) and len(v) > 0 and all(isinstance(item, str) for item in v)
 

graphistry/compute/gfql/call/validation.py

@@ -39,6 +39,7 @@
     is_list_of_strings,
     is_list_or_dict,
     is_non_empty_list_of_strings,
+    is_string_pair,
     is_non_empty_string,
     is_non_negative_int_like,
     is_string,
@@ -395,22 +396,24 @@ def _semi_apply_mark_added_node_cols(params: Dict[str, object]) -> Set[str]:
     ),
 
     'anti_semi_apply': _safelist_entry(
-        {'binding_ops', 'join_aliases'},
+        {'binding_ops', 'join_aliases', 'neq'},
         required_params={'binding_ops', 'join_aliases'},
         param_validators={
             'binding_ops': is_list_of_dicts,
             'join_aliases': is_non_empty_list_of_strings,
+            'neq': is_string_pair,
         },
         description='Filter active rows by anti-semi joining against correlated binding rows',
     ),
 
     'semi_apply_mark': _safelist_entry(
-        {'binding_ops', 'join_aliases', 'out_col'},
+        {'binding_ops', 'join_aliases', 'out_col', 'neq'},
         required_params={'binding_ops', 'join_aliases', 'out_col'},
         param_validators={
             'binding_ops': is_list_of_dicts,
             'join_aliases': is_non_empty_list_of_strings,
             'out_col': is_non_empty_string,
+            'neq': is_string_pair,
         },
         description='Annotate active rows with correlated pattern-existence booleans',
         schema_effects=_schema_effects(adds_node_cols=_semi_apply_mark_added_node_cols),

graphistry/compute/gfql/cypher/ast.py

@@ -147,6 +147,13 @@ class WherePatternPredicate:
     # anti-semi-join lowering instead of intersect-MATCH.  Default False keeps
     # all existing single-positive / multi-positive callers unchanged.
     negated: bool = False
+    # viz-filter L1: "exists" when built from an EXISTS { } subquery — its pattern
+    # aliases are EXISTENTIALLY quantified (locals allowed); "bare" keeps the
+    # conservative no-new-aliases guard for bare pattern predicates.
+    pattern_origin: str = "bare"
+    # viz-filter L1: endpoint-inequality constraint from `EXISTS { (n)--(m) WHERE
+    # m <> n }` — the viz drop-self prune-isolated flavor. (alias_a, alias_b).
+    pattern_neq: Optional[Tuple[str, str]] = None
 
 
 WhereTerm = Union[WherePredicate, WherePatternPredicate]
@@ -183,6 +190,10 @@ class BooleanExpr:
     atom_text: Optional[str] = None
     atom_span: Optional[SourceSpan] = None
     pattern: Optional[Tuple[PatternElement, ...]] = None
+    # viz-filter L1 (EXISTS { } leaves): existential-origin marker + optional
+    # endpoint-inequality; both flow into WherePatternPredicate at lift time.
+    pattern_origin: str = "bare"
+    pattern_neq: Optional[Tuple[str, str]] = None
 
 
 @dataclass(frozen=True)

graphistry/compute/gfql/cypher/lowering.py

@@ -5763,7 +5763,9 @@ def _where_expr_tree_pattern_predicates(expr: BooleanExpr) -> List[WherePatternP
                     line=cur.span.line,
                     column=cur.span.column,
                 )
-            out.append(WherePatternPredicate(pattern=cur.pattern, span=cur.span, negated=False))
+            out.append(WherePatternPredicate(
+                pattern=cur.pattern, span=cur.span, negated=False,
+                pattern_origin=cur.pattern_origin, pattern_neq=cur.pattern_neq))
             continue
         if cur.left is not None:
             stack.append(cur.left)
@@ -5799,7 +5801,10 @@ def _lower_pattern_predicate_to_row_marker(
         )
 
     introduced_aliases = sorted(alias for alias in predicate_aliases if alias not in alias_targets)
-    if introduced_aliases:
+    if introduced_aliases and predicate.pattern_origin != "exists":
+        # EXISTS { } subquery aliases are EXISTENTIALLY quantified (locals) — the
+        # bindings table projects them away; bare pattern predicates keep the
+        # conservative guard (viz-filter L1).
         raise _unsupported(
             "Cypher WHERE pattern predicates cannot introduce new aliases in this phase",
             field="where",
@@ -5830,6 +5835,7 @@ def _lower_pattern_predicate_to_row_marker(
         binding_ops=serialize_binding_ops(pattern_ops),
         join_aliases=shared_aliases,
         out_col=out_col,
+        neq=predicate.pattern_neq,
     )
 
 
@@ -5870,7 +5876,9 @@ def _rewrite(expr: BooleanExpr) -> BooleanExpr:
             marker_col = _fresh_marker_col(expr.span)
             marker_ops.append(
                 _lower_pattern_predicate_to_row_marker(
-                    WherePatternPredicate(pattern=expr.pattern, span=expr.span, negated=False),
+                    WherePatternPredicate(
+                        pattern=expr.pattern, span=expr.span, negated=False,
+                        pattern_origin=expr.pattern_origin, pattern_neq=expr.pattern_neq),
                     alias_targets=alias_targets,
                     params=params,
                     out_col=marker_col,
@@ -5936,7 +5944,10 @@ def _lower_negated_pattern_predicate_to_row_filter(
         )
 
     introduced_aliases = sorted(alias for alias in predicate_aliases if alias not in alias_targets)
-    if introduced_aliases:
+    if introduced_aliases and predicate.pattern_origin != "exists":
+        # EXISTS { } subquery aliases are EXISTENTIALLY quantified (locals) — the
+        # bindings table projects them away; bare pattern predicates keep the
+        # conservative guard (viz-filter L1).
         raise _unsupported(
             "Cypher WHERE pattern predicates cannot introduce new aliases in this phase",
             field="where",
@@ -5966,6 +5977,7 @@ def _lower_negated_pattern_predicate_to_row_filter(
     return anti_semi_apply(
         binding_ops=serialize_binding_ops(pattern_ops),
         join_aliases=shared_aliases,
+        neq=predicate.pattern_neq,
     )