diff --git a/CHANGELOG.md b/CHANGELOG.md index 04d31f5bb1..d69fa533c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm - **GFQL temporal-detection dtype gate (#1650)**: `order_detect_temporal_mode` now short-circuits for numeric/bool/complex columns, which can never hold temporal *text*, instead of running an `astype(str)` + multi-regex `fullmatch` scan on every comparison. Eliminates spurious row-wise stringification in `where_rows`/comparison paths whose output never contains entity-text. Byte-identical results; measured `where_rows` speedups ~3.1× (pandas) and ~4.4–13.3× (cuDF, scaling with row count). Does not address whole-entity `RETURN a` text rendering, which is tracked separately. - **GFQL generic single-hop fast path (perf, pandas + cuDF)**: a single `MATCH (n)` (node-only) or `MATCH (a {f})-[e]->(b)` (1-hop) — the dominant tabular/crossfilter + basic-graph-query shapes — now skip the forward/backward/combine BFS machinery in the generic engine: node-only returns the filtered node table; 1-hop returns the edges whose endpoints pass the node filters + those endpoint nodes. Same VALUES + node/edge sets as before (345-case adversarial golden: only dtype differs; hop/chain suites; gated to pandas/cuDF — dask/spark keep the full path). **~100× faster on pandas** (node filter 204→2 ms @10M; graph query similarly near-raw); cuDF stays on the resident frame (a couple semi-joins instead of the BFS + ~31 drop_duplicates), capturing the GPU semijoin win. **Minor behavior change:** the 1-hop now PRESERVES node-attribute dtypes (int stays int) instead of the full machinery's spurious int→float merge upcast — making pandas/cuDF consistent with the polars engine. The fast path is automatically skipped when a GFQL `policy` is installed, so the `prechain`/`postchain`/`postload` hooks (which can observe intermediate state and deny execution) always fire on the full path — keeping the optimization observationally transparent. Differential-verified equivalent to the full BFS path across 440 random well-formed graph×query cases plus targeted edge cases: it drops edges to nodes absent from the node table and dedups duplicate node ids (matching the full path's join semantics), and a NaN node id never validates a NaN edge endpoint. - **GFQL hop/chain redundant-dedup removal (perf)**: dropped the explicit `.unique()` dedup pass that fed only an `.isin()` membership test in the generic traversal — `_filter_edges_by_endpoint`, the undirected combine masks, and the per-hop wavefront filter (`compute/hop.py`, `compute/chain.py`). `isin(s) == isin(s.unique())` by set membership, so this is byte-identical (verified across 345 adversarial graph×query cases: dup/parallel edges, self-loops, isolated/dead-end nodes, cycles, undirected, multi-hop, fixed-point, min/max hops, names, filters, seeds). Each removed `.unique()` is one fewer kernel launch on GPU, where launch latency — not compute — dominates small/mid traversals: cuDF 1-hop chain ~126→103 ms @1M edges (~18% faster), pandas unaffected within noise. +- **GFQL row-expression parse memoization**: `parse_expr()` (the GFQL row-expression parser) now memoizes its result per expression string. Complements the Cypher-query parse memo (PR #1652); profiling showed that once whole-query parsing is cached, the residual fixed per-call compile cost is dominated by `parse_expr`, which is invoked ~4× per query compile (each RETURN/WHERE/WITH expression) and rebuilt a Lark transformer on every call. `parse_expr` is a pure function of the expression text (no params/schema) returning a tree of frozen (immutable) dataclasses, so identical expressions — re-parsed on every compile and recurring across queries (e.g. `a.val > 50`) — are served from an `lru_cache(maxsize=1024)`. Measured (dgx-spark): stacked on the query-parse memo, the fixed per-call cost of a repeated string Cypher query drops a further ~4.5 ms → ~1.8 ms (≈ parity with the equivalent native chain). The non-str/empty guard stays outside the cache; only successful parses are cached (invalid input re-raises every call). ### Fixed - **GFQL single-hop fast path ignored `prune_to_endpoints`**: the generic single-hop fast path accepted edges with `prune_to_endpoints=True` (a public `e()/e_forward()/e_reverse()` kwarg) but returned BOTH endpoints, whereas the flag keeps only the arrival side (destinations for forward, sources for reverse). A query like `[n(), e_forward(prune_to_endpoints=True), n()]` silently returned the wrong node/edge set. The fast path now declines this shape and falls through to the full path, which honors the flag. diff --git a/graphistry/compute/gfql/expr_parser.py b/graphistry/compute/gfql/expr_parser.py index 0dcd30c55d..1914d54d5b 100644 --- a/graphistry/compute/gfql/expr_parser.py +++ b/graphistry/compute/gfql/expr_parser.py @@ -767,9 +767,21 @@ def _normalize_dotted_identifiers(node: ExprNode) -> ExprNode: def parse_expr(expr: str) -> ExprNode: + """Parse a GFQL row-expression string into its frozen-dataclass AST. + + Memoized per expression string: parsing is a pure function of the text and the + returned ``ExprNode`` is immutable, so identical expressions — re-run on every + query compile, and recurring across queries (e.g. ``a.val > 50``) — skip the + Lark parse and the per-call transformer rebuild. The non-str/empty guard stays + outside the cache so invalid input still raises (and is never cached). + """ if not isinstance(expr, str) or expr.strip() == "": raise GFQLExprParseError("Expression must be a non-empty string") + return _parse_expr_cached(expr) + +@lru_cache(maxsize=1024) +def _parse_expr_cached(expr: str) -> ExprNode: parser = _parser() transformer = _build_transformer() try: diff --git a/graphistry/tests/compute/gfql/test_expr_parser.py b/graphistry/tests/compute/gfql/test_expr_parser.py index f427963243..7be42281dd 100644 --- a/graphistry/tests/compute/gfql/test_expr_parser.py +++ b/graphistry/tests/compute/gfql/test_expr_parser.py @@ -358,3 +358,33 @@ def test_validate_expr_capabilities_rejects_wildcard_function_arg() -> None: errors = validate_expr_capabilities(node) assert "unsupported function: count" in errors assert "unsupported wildcard: *" in errors + + +def test_parse_expr_memoizes_identical_expressions() -> None: + from graphistry.compute.gfql import expr_parser as _ep + + expr = "a.val > 50 AND a.kind = 'x'" + first = _ep.parse_expr(expr) + second = _ep.parse_expr(expr) + # Same object identity => cache hit (no re-parse / transformer rebuild). + assert first is second + + +def test_parse_expr_cache_distinguishes_and_registers_hits() -> None: + from graphistry.compute.gfql import expr_parser as _ep + + a = _ep.parse_expr("zzz_expr_probe.k + 1") + b = _ep.parse_expr("zzz_expr_probe.k + 2") + assert a is not b + before = _ep._parse_expr_cached.cache_info().hits + _ep.parse_expr("zzz_expr_probe.k + 1") + after = _ep._parse_expr_cached.cache_info().hits + assert after == before + 1 + + +def test_parse_expr_does_not_cache_invalid_expressions() -> None: + from graphistry.compute.gfql.expr_parser import GFQLExprParseError, parse_expr + + for _ in range(2): + with pytest.raises(GFQLExprParseError): + parse_expr("")