perf(gfql): general optimizations base (parse memoization + temporal dtype-gate) [PR0]

CHANGELOG.md

@@ -8,6 +8,20 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 ## [Development]
 <!-- Do Not Erase This Section - Used for tracking unreleased changes -->
 
+### Changed
+- **GFQL Cypher parse memoization (perf)**: `parse_cypher` now memoizes its result (LRU over the deterministic lark parse+transform → immutable frozen AST). Repeated identical Cypher queries skip the ~15 ms parse — the dominant per-call cost of small queries (~50% of a Cypher call at 100k rows) — making end-to-end query latency ~1.3–1.7× faster at small/interactive sizes across pandas/polars/cuDF. Safe to share the cached AST: every Cypher AST node is `@dataclass(frozen=True)` and `compile_cypher_query` does not mutate the parsed tree; validation errors still raise and are not cached.
+
+### Performance
+- **GFQL temporal-detection dtype gate (#1650)**: `order_detect_temporal_mode` now short-circuits for numeric/bool/complex columns, which can never hold temporal *text*, instead of running an `astype(str)` + multi-regex `fullmatch` scan on every comparison. Eliminates spurious row-wise stringification in `where_rows`/comparison paths whose output never contains entity-text. Byte-identical results; measured `where_rows` speedups ~3.1× (pandas) and ~4.4–13.3× (cuDF, scaling with row count). Does not address whole-entity `RETURN a` text rendering, which is tracked separately.
+- **GFQL generic single-hop fast path (perf, pandas + cuDF)**: a single `MATCH (n)` (node-only) or `MATCH (a {f})-[e]->(b)` (1-hop) — the dominant tabular/crossfilter + basic-graph-query shapes — now skip the forward/backward/combine BFS machinery in the generic engine: node-only returns the filtered node table; 1-hop returns the edges whose endpoints pass the node filters + those endpoint nodes. Same VALUES + node/edge sets as before (345-case adversarial golden: only dtype differs; hop/chain suites; gated to pandas/cuDF — dask/spark keep the full path). **~100× faster on pandas** (node filter 204→2 ms @10M; graph query similarly near-raw); cuDF stays on the resident frame (a couple semi-joins instead of the BFS + ~31 drop_duplicates), capturing the GPU semijoin win. **Minor behavior change:** the 1-hop now PRESERVES node-attribute dtypes (int stays int) instead of the full machinery's spurious int→float merge upcast — making pandas/cuDF consistent with the polars engine. The fast path is automatically skipped when a GFQL `policy` is installed, so the `prechain`/`postchain`/`postload` hooks (which can observe intermediate state and deny execution) always fire on the full path — keeping the optimization observationally transparent. Differential-verified equivalent to the full BFS path across 440 random well-formed graph×query cases plus targeted edge cases: it drops edges to nodes absent from the node table and dedups duplicate node ids (matching the full path's join semantics), and a NaN node id never validates a NaN edge endpoint.
+- **GFQL hop/chain redundant-dedup removal (perf)**: dropped the explicit `.unique()` dedup pass that fed only an `.isin()` membership test in the generic traversal — `_filter_edges_by_endpoint`, the undirected combine masks, and the per-hop wavefront filter (`compute/hop.py`, `compute/chain.py`). `isin(s) == isin(s.unique())` by set membership, so this is byte-identical (verified across 345 adversarial graph×query cases: dup/parallel edges, self-loops, isolated/dead-end nodes, cycles, undirected, multi-hop, fixed-point, min/max hops, names, filters, seeds). Each removed `.unique()` is one fewer kernel launch on GPU, where launch latency — not compute — dominates small/mid traversals: cuDF 1-hop chain ~126→103 ms @1M edges (~18% faster), pandas unaffected within noise.
+
+### Fixed
+- **GFQL single-hop fast path ignored `prune_to_endpoints`**: the generic single-hop fast path accepted edges with `prune_to_endpoints=True` (a public `e()/e_forward()/e_reverse()` kwarg) but returned BOTH endpoints, whereas the flag keeps only the arrival side (destinations for forward, sources for reverse). A query like `[n(), e_forward(prune_to_endpoints=True), n()]` silently returned the wrong node/edge set. The fast path now declines this shape and falls through to the full path, which honors the flag.
+- **GFQL whole-entity `RETURN a, a.val` emitted a duplicate column**: flattening a whole entity `a` into `a.id, a.val, …` (#1650) shares the `{alias}.{field}` namespace with an explicit property projection, so `RETURN a, a.val` produced two `a.val` columns — a duplicate-named column that breaks column selection and silently drops data on `to_dict`/serialization. The duplicate (always identical data, since dotted backtick aliases are rejected) is now collapsed to a single column, keeping first occurrence.
+- **GFQL chain on edges-only graphs (no node binding)**: A chain/Cypher query over a graph with edges but no node-id binding (`g._node is None`) that took the full traversal path (any fast-path-ineligible shape — multi-hop, named/queried/`prune_to_endpoints` edges — or any query with a policy attached) rebuilt the result from the *unbound* input graph, dropping the materialized node-id binding. The endpoint-reconciliation concat then synthesized a spurious `None`-named node column: a corrupt result on older pandas, and a hard `NotImplementedError` (void-block NA fill) on newer pandas (e.g. Python 3.14). The result now carries the materialized node-id binding and a single, correct node column, matching the fast-path output; the original edge binding (e.g. `None`) is still restored.
+- **GFQL chain fast path bypassed policy hooks**: the degenerate-shape chain fast path (node-only `MATCH (n)`, single-hop) short-circuited before the `prechain`/`postchain`/`postload` policy hook dispatch, so a policy-bearing query that hit the fast path never fired those hooks (or per-op policy inspection). The fast path is now skipped whenever a policy is attached — `prechain` is a pre-compute gate that must observe/block every load — so policy-bearing queries take the full path; the no-policy path keeps the optimization.
+
 ## [0.56.1 - 2026-05-27]
 
 ### Added

graphistry/compute/chain.py

@@ -28,11 +28,13 @@
 logger = setup_logger(__name__)
 
 
-def _filter_edges_by_endpoint(edges_df, nodes_df, node_id: str, edge_col: str):
+def _filter_edges_by_endpoint(
+    edges_df: DataFrameT, nodes_df: Optional[DataFrameT], node_id: str, edge_col: str
+) -> DataFrameT:
     if nodes_df is None or not node_id or not edge_col or edge_col not in edges_df.columns:
         return edges_df
-    ids = nodes_df[node_id].unique()
-    return edges_df[edges_df[edge_col].isin(ids)]
+    # isin() is set-membership, so the dropped .unique() is redundant (byte-identical).
+    return edges_df[edges_df[edge_col].isin(nodes_df[node_id])]
 
 
 class Chain(ASTSerializable):
@@ -219,8 +221,9 @@ def combine_steps(
                 direction = getattr(op, 'direction', 'forward') if isinstance(op, ASTEdge) else 'forward'
 
                 if direction == 'undirected' and prev_nodes is not None and next_nodes is not None and node_id:
-                    prev_ids = prev_nodes[node_id].unique()
-                    next_ids = next_nodes[node_id].unique()
+                    # isin() dedups internally -> the .unique() pass is redundant
+                    prev_ids = prev_nodes[node_id]
+                    next_ids = next_nodes[node_id]
                     fwd_mask = edges_df[src_col].isin(prev_ids) & edges_df[dst_col].isin(next_ids)
                     rev_mask = edges_df[dst_col].isin(prev_ids) & edges_df[src_col].isin(next_ids)
                     edges_df = edges_df[fwd_mask | rev_mask]
@@ -670,6 +673,91 @@ def _chain_otel_attrs(
     return attrs
 
 
+def _try_chain_fast_path(
+    g_in: Plottable,
+    ops: List[ASTObject],
+    engine_concrete: Engine,
+    start_nodes: Optional[DataFrameT] = None,
+) -> Optional[Plottable]:
+    """Degenerate-shape fast path (pandas/cuDF): node-only ``MATCH (n)`` or a plain
+    single-hop ``MATCH (a)-[e]->(b)`` skip the forward/backward/combine BFS machinery.
+    Returns the result Plottable, or ``None`` to fall through to the full path.
+
+    Same node/edge sets + VALUES as the full machinery (trackA_golden + hop/chain
+    suites); the 1-hop additionally preserves int node dtypes (the full path upcasts
+    int→float via merge). Gated to unnamed/unqueried nodes + a plain single-hop edge;
+    filtered-undirected and seeded chains fall through. polars/dask/spark also fall
+    through (own fast path / lazy semantics)."""
+    from graphistry.compute.filter_by_dict import filter_by_dict
+
+    if engine_concrete not in (Engine.PANDAS, Engine.CUDF):
+        return None
+    if start_nodes is not None:
+        return None  # seeded chains use the full path (fast path has no seed)
+    engine_abs = EngineAbstract(engine_concrete.value)
+
+    def _materialize_fast_path_graph() -> Plottable:
+        from graphistry.compute.ComputeMixin import _coerce_input_formats  # lazy — avoids circular import
+        g = g_in.materialize_nodes(engine=EngineAbstract(engine_concrete.value))
+        return _coerce_input_formats(g, engine_concrete)
+
+    if len(ops) == 1:
+        n0 = ops[0]
+        if not (isinstance(n0, ASTNode) and n0._name is None and n0.query is None):
+            return None
+        g = _materialize_fast_path_graph()
+        if g._nodes is None:
+            return None
+        nodes = filter_by_dict(g._nodes, n0.filter_dict, engine_abs) if n0.filter_dict else g._nodes
+        edges = g._edges.iloc[0:0] if g._edges is not None else None
+        return g.nodes(nodes).edges(edges) if edges is not None else g.nodes(nodes)
+
+    if len(ops) != 3:
+        return None
+    n0, e1, n2 = ops
+    if not (isinstance(n0, ASTNode) and n0._name is None and n0.query is None):
+        return None
+    if not (isinstance(n2, ASTNode) and n2._name is None and n2.query is None):
+        return None
+    if not (isinstance(e1, ASTEdge) and e1.is_simple_single_hop()
+            and e1.edge_match is None and e1.source_node_match is None
+            and e1.destination_node_match is None and e1._name is None
+            and e1.source_node_query is None and e1.destination_node_query is None
+            and e1.edge_query is None and not e1.include_zero_hop_seed
+            and not e1.prune_to_endpoints):  # prune keeps only the arrival side -> full path
+        return None
+    direction = e1.direction
+    unconstrained = not n0.filter_dict and not n2.filter_dict
+    if not unconstrained and direction == "undirected":
+        return None  # filtered-undirected (OR of both directions) -> full path
+    g = _materialize_fast_path_graph()
+    if g._nodes is None or g._edges is None:
+        return None
+    src, dst, node = g._source, g._destination, g._node
+    # Keep only edges with BOTH endpoints in the node table (the full path drops
+    # dangling edges via its joins). dropna so a NaN node id can't validate a NaN
+    # endpoint — .isin treats NaN as matchable but the BFS joins never match NaN<->NaN.
+    node_ids = g._nodes[node].dropna()
+    edges = g._edges[g._edges[src].isin(node_ids) & g._edges[dst].isin(node_ids)]
+    if not unconstrained:
+        from_col, to_col = (src, dst) if direction == "forward" else (dst, src)
+        if n0.filter_dict:
+            ids = filter_by_dict(g._nodes, n0.filter_dict, engine_abs)[node]
+            edges = edges[edges[from_col].isin(ids)]
+        if n2.filter_dict:
+            ids = filter_by_dict(g._nodes, n2.filter_dict, engine_abs)[node]
+            edges = edges[edges[to_col].isin(ids)]
+    concat = df_concat(engine_concrete)
+    endpoints = concat([
+        edges[[src]].rename(columns={src: node}),
+        edges[[dst]].rename(columns={dst: node}),
+    ]).drop_duplicates()
+    nodes = g._nodes[g._nodes[node].isin(endpoints[node])]
+    # match the full path's merge, which collapses duplicate node-id rows
+    nodes = nodes.drop_duplicates(subset=[node])
+    return g.nodes(nodes).edges(edges)
+
+
 @otel_traced("gfql.chain", attrs_fn=_chain_otel_attrs)
 def chain(
     self: Plottable,
@@ -794,6 +882,13 @@ def _chain_impl(
     if validate_schema:
         validate_chain_schema(self, ops, collect_all=False)
 
+    # The fast path skips the policy hook dispatch (prechain/postchain below), so only
+    # take it when no policy is attached — policy-bearing queries must keep hook firing.
+    if not policy:
+        _fast = _try_chain_fast_path(self, ops, engine_concrete, start_nodes)
+        if _fast is not None:
+            return _fast
+
     if isinstance(ops[0], ASTEdge):
         logger.debug('adding initial node to ensure initial link has needed reversals')
         ops = cast(List[ASTObject], [ ASTNode() ]) + ops
@@ -1002,7 +1097,14 @@ def _chain_impl(
             )
             if added_edge_index:
                 final_edges_df = final_edges_df.drop(columns=[g._edge])
-                g_out = self.nodes(final_nodes_df).edges(final_edges_df, edge=original_edge)
+                # Rebuild from `self` to restore the ORIGINAL edge binding (`self._edge`,
+                # often None — `g` carries the internal edge-index binding instead), but
+                # explicitly carry the materialized node-id binding `g._node`: for an
+                # edges-only input `self._node is None`, so rebuilding from `self` alone
+                # drops it, leaving the endpoint-reconciliation concat below to synthesize
+                # a `None`-named column (corrupt result + a void-block concat crash on
+                # newer pandas).
+                g_out = self.nodes(final_nodes_df, g._node).edges(final_edges_df, edge=original_edge)
             else:
                 g_out = g.nodes(final_nodes_df).edges(final_edges_df)
 

graphistry/compute/gfql/cypher/ast.py

@@ -2,6 +2,8 @@
 from typing import Literal, Optional, Tuple, Union
 
 
+# INVARIANT: keep every node deeply immutable (scalar/tuple fields only) — parse_cypher
+# shares results by reference via lru_cache, so a mutable field would poison cache hits.
 @dataclass(frozen=True)
 class SourceSpan:
     line: int

graphistry/compute/gfql/cypher/parser.py

@@ -1857,7 +1857,19 @@ def parse_cypher(query: str) -> Union[CypherQuery, CypherUnionQuery, CypherGraph
     """
     if not isinstance(query, str) or query.strip() == "":
         raise _to_syntax_error("Cypher query must be a non-empty string")
+    return _parse_cypher_cached(query)
 
+
+@lru_cache(maxsize=512)
+def _parse_cypher_cached(query: str) -> Union[CypherQuery, CypherUnionQuery, CypherGraphQuery]:
+    """Cached parse body: pure function of the query text -> immutable frozen AST.
+
+    Memoizing skips the ~15ms lark parse+transform on repeated identical queries
+    (the dominant per-call cost of small cypher queries). Safe to share the cached
+    result: every cypher AST node is ``@dataclass(frozen=True)`` and the downstream
+    ``compile_cypher_query`` does not mutate the parsed tree (verified). Validation
+    errors raise (and are not cached by ``lru_cache``), preserving error behavior.
+    """
     # Pre-parse detection of known-but-unsupported Cypher forms
     _check_unsupported_syntax_patterns(query)
 

graphistry/compute/gfql/cypher/result_postprocess.py

@@ -7,6 +7,7 @@
 
 from graphistry.Plottable import Plottable
 from graphistry.compute.typing import DataFrameT, SeriesT
+from graphistry.compute.gfql.series_str_compat import is_non_textual_scalar_dtype
 
 from .lowering import ResultProjectionColumn, ResultProjectionPlan
 from graphistry.compute.gfql.row.entity_props import (
@@ -123,6 +124,12 @@ def _project_property_column(
     if column.source_name is None or column.source_name not in rows_df.columns:
         raise ValueError(f"projection source column not found: {column.source_name!r}")
     series = cast(SeriesT, rows_df[column.source_name])
+    # Temporal-constructor normalization only applies to STRING values; numeric/bool/
+    # complex columns can never hold temporal text, so skip the (otherwise spurious)
+    # ``astype(str)`` + detection scan and return the column as-is — byte-identical,
+    # since the scan returns None for these dtypes. Mirrors the #1650/#1651 gate.
+    if is_non_textual_scalar_dtype(getattr(series, "dtype", None)):
+        return series
     if hasattr(series, "astype") and hasattr(cast(SeriesT, series.astype(str)), "str"):
         normalized = _normalize_temporal_constructor_series(
             rows_df,

graphistry/compute/gfql/expr_parser.py

@@ -21,6 +21,8 @@
 DEFAULT_ALLOWED_QUANTIFIERS: FrozenSet[str] = GFQL_ALLOWED_QUANTIFIERS
 
 
+# INVARIANT: keep every ExprNode deeply immutable (scalar/tuple fields only) — parse_expr
+# shares results by reference via lru_cache, so a mutable field would poison cache hits.
 @dataclass(frozen=True)
 class Identifier:
     name: str

graphistry/compute/gfql/row/ordering.py

@@ -12,6 +12,7 @@
 from graphistry.compute.typing import SeriesT
 
 from graphistry.compute.gfql.series_str_compat import (
+    is_non_textual_scalar_dtype,
     series_sequence_len,
     series_str_extract,
     series_str_fullmatch,
@@ -115,9 +116,19 @@ def _actual_string_mask(series: Any, text: Any, null_mask: Any) -> Any:
     return mask
 
 
+def _is_non_listable_dtype(series: Any) -> bool:
+    """True for numeric/bool/complex columns, which can never hold list values or
+    list-syntax *text*. Lets list/temporal detection skip the spurious
+    ``astype(str)`` + regex scan on these columns (byte-identical — the scan
+    returns False/None for them anyway). Mirrors the #1650/#1651 dtype gate."""
+    return is_non_textual_scalar_dtype(getattr(series, "dtype", None))
+
+
 def order_detect_list_series(series: Any) -> bool:
     if not hasattr(series, "isna") or not hasattr(series, "astype"):
         return False
+    if _is_non_listable_dtype(series):
+        return False
     null_mask = series.isna()
     non_null = ~null_mask
     if hasattr(non_null, "any") and not bool(non_null.any()):
@@ -143,6 +154,8 @@ def order_detect_stringified_list_series(series: Any) -> bool:
     """
     if not hasattr(series, "isna") or not hasattr(series, "astype"):
         return False
+    if _is_non_listable_dtype(series):
+        return False
     null_mask = series.isna()
     non_null = ~null_mask
     if hasattr(non_null, "any") and not bool(non_null.any()):
@@ -214,6 +227,10 @@ def parse_stringified_list_series(series: Any) -> Optional[SeriesT]:
 def order_detect_temporal_mode(series: Any) -> Optional[str]:
     if not hasattr(series, "dropna"):
         return None
+    # Temporal values are text (Cypher date/time literals/constructors); numeric/bool/
+    # complex can't match those regexes, so skip the astype(str)+scan for them (#1650).
+    if is_non_textual_scalar_dtype(getattr(series, "dtype", None)):
+        return None
     non_null = series.dropna()
     if len(non_null) == 0 or not hasattr(non_null, "astype"):
         return None

graphistry/compute/gfql/row/pipeline.py

@@ -59,7 +59,7 @@
     is_entity_text_scalar,
 )
 from graphistry.compute.gfql.same_path_types import NODE_IDENTITY_COLUMN
-from graphistry.compute.gfql.series_str_compat import series_sequence_len, series_str_match
+from graphistry.compute.gfql.series_str_compat import is_non_textual_scalar_dtype, series_sequence_len, series_str_match
 from graphistry.compute.gfql.row.ordering import (
     build_list_sort_columns,
     build_temporal_sort_columns,
@@ -1992,6 +1992,10 @@ def _coerce_duration_property(value_text: str) -> Any:
     def _gfql_series_is_list_like(series: Any) -> bool:
         if not hasattr(series, "isna") or not hasattr(series, "astype"):
             return False
+        # numeric/bool/complex columns can never be list-like — skip the spurious
+        # astype(str)+regex scan (byte-identical; the scan returns False anyway).
+        if is_non_textual_scalar_dtype(getattr(series, "dtype", None)):
+            return False
         null_mask = series.isna()
         non_null = ~null_mask
         if hasattr(non_null, "any") and not bool(non_null.any()):