Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- **GFQL temporal-detection dtype gate (#1650)**: `order_detect_temporal_mode` now short-circuits for numeric/bool/complex columns, which can never hold temporal *text*, instead of running an `astype(str)` + multi-regex `fullmatch` scan on every comparison. Eliminates spurious row-wise stringification in `where_rows`/comparison paths whose output never contains entity-text. Byte-identical results; measured `where_rows` speedups ~3.1× (pandas) and ~4.4–13.3× (cuDF, scaling with row count). Does not address whole-entity `RETURN a` text rendering, which is tracked separately.
- **GFQL generic single-hop fast path (perf, pandas + cuDF)**: a single `MATCH (n)` (node-only) or `MATCH (a {f})-[e]->(b)` (1-hop) — the dominant tabular/crossfilter + basic-graph-query shapes — now skip the forward/backward/combine BFS machinery in the generic engine: node-only returns the filtered node table; 1-hop returns the edges whose endpoints pass the node filters + those endpoint nodes. Same VALUES + node/edge sets as before (345-case adversarial golden: only dtype differs; hop/chain suites; gated to pandas/cuDF — dask/spark keep the full path). **~100× faster on pandas** (node filter 204→2 ms @10M; graph query similarly near-raw); cuDF stays on the resident frame (a couple semi-joins instead of the BFS + ~31 drop_duplicates), capturing the GPU semijoin win. **Minor behavior change:** the 1-hop now PRESERVES node-attribute dtypes (int stays int) instead of the full machinery's spurious int→float merge upcast — making pandas/cuDF consistent with the polars engine. The fast path is automatically skipped when a GFQL `policy` is installed, so the `prechain`/`postchain`/`postload` hooks (which can observe intermediate state and deny execution) always fire on the full path — keeping the optimization observationally transparent. Differential-verified equivalent to the full BFS path across 440 random well-formed graph×query cases plus targeted edge cases: it drops edges to nodes absent from the node table and dedups duplicate node ids (matching the full path's join semantics), and a NaN node id never validates a NaN edge endpoint.
- **GFQL hop/chain redundant-dedup removal (perf)**: dropped the explicit `.unique()` dedup pass that fed only an `.isin()` membership test in the generic traversal — `_filter_edges_by_endpoint`, the undirected combine masks, and the per-hop wavefront filter (`compute/hop.py`, `compute/chain.py`). `isin(s) == isin(s.unique())` by set membership, so this is byte-identical (verified across 345 adversarial graph×query cases: dup/parallel edges, self-loops, isolated/dead-end nodes, cycles, undirected, multi-hop, fixed-point, min/max hops, names, filters, seeds). Each removed `.unique()` is one fewer kernel launch on GPU, where launch latency — not compute — dominates small/mid traversals: cuDF 1-hop chain ~126→103 ms @1M edges (~18% faster), pandas unaffected within noise.
- **GFQL row-expression parse memoization**: `parse_expr()` (the GFQL row-expression parser) now memoizes its result per expression string. Complements the Cypher-query parse memo (PR #1652); profiling showed that once whole-query parsing is cached, the residual fixed per-call compile cost is dominated by `parse_expr`, which is invoked ~4× per query compile (each RETURN/WHERE/WITH expression) and rebuilt a Lark transformer on every call. `parse_expr` is a pure function of the expression text (no params/schema) returning a tree of frozen (immutable) dataclasses, so identical expressions — re-parsed on every compile and recurring across queries (e.g. `a.val > 50`) — are served from an `lru_cache(maxsize=1024)`. Measured (dgx-spark): stacked on the query-parse memo, the fixed per-call cost of a repeated string Cypher query drops a further ~4.5 ms → ~1.8 ms (≈ parity with the equivalent native chain). The non-str/empty guard stays outside the cache; only successful parses are cached (invalid input re-raises every call).

### Fixed
- **GFQL single-hop fast path ignored `prune_to_endpoints`**: the generic single-hop fast path accepted edges with `prune_to_endpoints=True` (a public `e()/e_forward()/e_reverse()` kwarg) but returned BOTH endpoints, whereas the flag keeps only the arrival side (destinations for forward, sources for reverse). A query like `[n(), e_forward(prune_to_endpoints=True), n()]` silently returned the wrong node/edge set. The fast path now declines this shape and falls through to the full path, which honors the flag.
Expand Down
12 changes: 12 additions & 0 deletions graphistry/compute/gfql/expr_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -767,9 +767,21 @@ def _normalize_dotted_identifiers(node: ExprNode) -> ExprNode:


def parse_expr(expr: str) -> ExprNode:
"""Parse a GFQL row-expression string into its frozen-dataclass AST.

Memoized per expression string: parsing is a pure function of the text and the
returned ``ExprNode`` is immutable, so identical expressions — re-run on every
query compile, and recurring across queries (e.g. ``a.val > 50``) — skip the
Lark parse and the per-call transformer rebuild. The non-str/empty guard stays
outside the cache so invalid input still raises (and is never cached).
"""
if not isinstance(expr, str) or expr.strip() == "":
raise GFQLExprParseError("Expression must be a non-empty string")
return _parse_expr_cached(expr)


@lru_cache(maxsize=1024)
def _parse_expr_cached(expr: str) -> ExprNode:
parser = _parser()
transformer = _build_transformer()
try:
Expand Down
30 changes: 30 additions & 0 deletions graphistry/tests/compute/gfql/test_expr_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -358,3 +358,33 @@ def test_validate_expr_capabilities_rejects_wildcard_function_arg() -> None:
errors = validate_expr_capabilities(node)
assert "unsupported function: count" in errors
assert "unsupported wildcard: *" in errors


def test_parse_expr_memoizes_identical_expressions() -> None:
from graphistry.compute.gfql import expr_parser as _ep

expr = "a.val > 50 AND a.kind = 'x'"
first = _ep.parse_expr(expr)
second = _ep.parse_expr(expr)
# Same object identity => cache hit (no re-parse / transformer rebuild).
assert first is second


def test_parse_expr_cache_distinguishes_and_registers_hits() -> None:
from graphistry.compute.gfql import expr_parser as _ep

a = _ep.parse_expr("zzz_expr_probe.k + 1")
b = _ep.parse_expr("zzz_expr_probe.k + 2")
assert a is not b
before = _ep._parse_expr_cached.cache_info().hits
_ep.parse_expr("zzz_expr_probe.k + 1")
after = _ep._parse_expr_cached.cache_info().hits
assert after == before + 1


def test_parse_expr_does_not_cache_invalid_expressions() -> None:
from graphistry.compute.gfql.expr_parser import GFQLExprParseError, parse_expr

for _ in range(2):
with pytest.raises(GFQLExprParseError):
parse_expr("")
Loading