diff --git a/CHANGELOG.md b/CHANGELOG.md index 449a2713bd..346f1d8214 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ### Documentation - **GFQL pay-as-you-go resident indexing user guide**: New :doc:`Pay-As-You-Go Resident Indexing ` page — the lifecycle guide to resident indexes (`gfql_index_all()` / `gfql_index_edges()` / `create_index()` / `show_indexes()` / `drop_index()`): what the node-id + CSR in/out adjacency sidecars are, what engages them on 0.58.0 (seeded typed-hop fast paths incl. property RETURNs and property-seeded lookups per #1768/#1770, direct `g.hop()`; the general polars chain traversal honestly noted as not yet covered), the staleness/validity contract (identity + fingerprint; rebind invalidates; declines are safe — identical results either way), engine notes (polars needs `gfql_index_all(engine='polars')` until #1767), 0.58.0-tag measured numbers, and a runnable end-to-end example. Wired into the GFQL toctree + recommended paths alongside :doc:`Seeded Traversal Indexes `. +- **GFQL performance docs: 0.58.0 release-tag-verified numbers, siloed in one page**: `gfql/performance.rst` is now the canonical benchmark-numbers page (alongside `gfql/index_adjacency.rst` for the index benchmarks) — a benchmark rerun updates it alone. It carries the 0.58.0 tag sweep (DGX Spark GB10, warm medians N=30; four-engine numbers cross-engine parity-verified, competitor pairs validated against expected result rows): seeded typed-hop fast path across all four engines (e.g. pandas 29.9→2.46ms, 12.1×), native chain form, resident-index covered-shape lookups (with the `gfql_index_all(engine='polars')` caveat / PR #1767), flat seeded-hop scaling on pandas (0.159–0.164ms from 0.25M to 32M edges), the one-keyword `engine='polars'` LDBC SNB SF1 seed-lookup win (1,299.6→106.1ms, 12.3×), LDBC SNB interactive SF1 vs Neo4j 5.26 same-box pairs (GFQL 4 of 5; Neo4j wins recent-replies — reported as-is), OLAP multi-join vs embedded Kuzu (q8 200×, q9 14.2×) with the honest inverse (Kuzu wins single-table aggregates 2–4×, seeded property-projection lookups 2.4–64×), plus the prior Orkut/LiveJournal bulk sweep (moved from `engines.rst`, dated once) and its methodology. All other pages — `engines.rst`, `quick.rst`, `about.rst`, `overview.rst`, `index.rst` — now carry stable qualitative claims (e.g. "often an order of magnitude faster on query-heavy workloads") that link into `performance.rst` instead of inline figures, replacing the stale "up to ~38×" headlines and avoiding scattered per-claim version labels. ## [0.58.0 - 2026-07-22] diff --git a/docs/source/gfql/about.rst b/docs/source/gfql/about.rst index bfb32d6c24..2eb79623ee 100644 --- a/docs/source/gfql/about.rst +++ b/docs/source/gfql/about.rst @@ -382,7 +382,7 @@ results on every engine — see :doc:`Choosing an Engine `. :: - g_result = g.gfql([ ... ], engine='polars') # up to ~38x over pandas on real graphs + g_result = g.gfql([ ... ], engine='polars') # often much faster on query-heavy workloads **Example: Force GFQL to use a GPU engine** diff --git a/docs/source/gfql/engines.rst b/docs/source/gfql/engines.rst index 4b7c169724..ae2510aa66 100644 --- a/docs/source/gfql/engines.rst +++ b/docs/source/gfql/engines.rst @@ -31,7 +31,11 @@ engine is a one-keyword change — no GPU, same results: query = "MATCH (a)-[e]->(b) RETURN b" # any GFQL / Cypher query g.gfql(query) # engine='pandas' (default) - g.gfql(query, engine='polars') # up to ~38x faster on real graphs, no GPU, identical results + g.gfql(query, engine='polars') # often much faster on query-heavy workloads, identical results + +Switching is often an order-of-magnitude speedup on query-heavy workloads — the current +measured numbers (release-verified engine sweeps, cross-database pairs, OLAP) live in +:ref:`gfql-0580-numbers` on the :doc:`performance` page. Your existing pandas, Polars, or cuDF graph works as-is: the input frames are accepted and coerced once; the only change is the keyword. The catch: a few exotic Cypher features still @@ -96,72 +100,31 @@ The four engines ``engine='auto'`` resolves to ``cudf`` for cuDF input and ``pandas`` otherwise. **AUTO never selects Polars or Polars-GPU** — they are explicit opt-in (see *Why opt-in?* below). -Motivating comparison (real graphs) ------------------------------------ - -Same query, same answers, four engines. Warm-median latency on **Orkut** (3.1M nodes / -**117M edges**, SNAP), measured on a single machine: - -.. list-table:: - :header-rows: 1 - :widths: 34 16 16 16 16 - - * - Workload (Orkut, 117M edges) - - ``pandas`` - - ``polars`` - - ``cudf`` - - ``polars-gpu`` - * - 1-hop from 10K seeds - - 2613 ms - - **68 ms** - - 1005 ms - - 63 ms - * - 2-hop from 10K seeds - - 18161 ms - - 2695 ms - - 2774 ms - - **1518 ms** - * - Full out-degree aggregation - - 799 ms - - 205 ms - - 314 ms - - **167 ms** - * - 2-hop from 100K seeds (~85M output rows) - - 28822 ms - - 8215 ms - - **6002 ms** - - 8559 ms - -*Warm median, identical result rows across all four engines. Reproducer:* -``benchmarks/gfql/index_bulk_olap_bench.py``. *See Methodology below.* - -Reading the table: - -- **Polars-CPU beats pandas up to ~38x** on bulk traversal and ~4x on aggregation — **with no - GPU**. On the 1-hop workload it is ~38x faster than pandas (68 ms vs 2613 ms). -- **Polars-CPU also beats cuDF** on these shapes (68 ms vs 1005 ms on 1-hop). cuDF runs - GFQL *eagerly*, op by op (a kernel launch + a materialized intermediate per hop), while - Polars builds **one fused lazy plan and collects once**. The fused plan wins until the - work is large enough to amortize GPU launch costs. -- **Polars-GPU is fastest on heavy multi-hop** (2-hop from 10K seeds: 1518 ms) and on - aggregation — the same fused plan, executed on the GPU. -- **cuDF wins the one extreme case** — a 2-hop from 100K seeds materializing ~85M output rows - (6.0 s) — where raw GPU throughput on a single massive join overtakes everything and - Polars-GPU comes under memory pressure (footnote F3). -- On a smaller graph (**LiveJournal**, 35M edges) the pattern holds: 1-hop from 10K seeds is - pandas 1129 ms → polars **37 ms** (~30x). Filter- and lookup-heavy workloads favor Polars - even more strongly — a separate **LDBC SNB sf1** benchmark shows order-of-magnitude gains - (tens of × over pandas; see ``benchmarks/gfql/`` and the GFQL benchmark notes). - -.. note:: - Route by workload shape and size (next section). **CPU Polars wins the common graph-query - shapes from ~10K edges up** — on LiveJournal subsampled (CPU, warm-median): 1-hop traversal - 2.7× / 4.5× / 7.6× and ``WHERE``+``ORDER`` 3.0× / 3.0× / 18× over pandas at 10K / 100K / 1M. - The **GPU** engines (cuDF / Polars-GPU) are the ones with a real small-size floor — they need - enough work to amortize kernel-launch cost (work-bound, [F2]). The only case pandas edges out - is a trivial sub-millisecond operation (e.g. a bare node-equality filter), where its boolean - mask beats Polars' plan overhead — but at <1 ms the difference is immaterial. Reproducer: - ``benchmarks/gfql/index_crossover_bench.py``. +How the engines compare +----------------------- + +The measured tables live on the :doc:`performance` page — the release-verified engine +sweeps plus the large-graph bulk comparison — and in :doc:`index_adjacency` for the +resident-index benchmarks. The stable pattern they show: + +- **Polars-CPU is the everyday win**: typically much faster than pandas on query-heavy + workloads (traversal, ``WHERE``/``ORDER``, aggregation) from roughly 10K edges up — + often by an order of magnitude on bulk traversal — with **no GPU**. It also frequently + beats eager cuDF on bulk work, because it builds **one fused lazy plan and collects + once** while cuDF pays a kernel launch and a materialized intermediate per op. +- **Polars-GPU leads heavy multi-hop and full-graph aggregation**: the same fused plan, + executed on the GPU, once there is enough work to amortize kernel launches. +- **cuDF wins the extreme single materialization**: one very large join/output where raw + GPU throughput dominates and the in-memory Polars-GPU executor comes under memory + pressure. +- **pandas keeps trivial sub-millisecond operations**: a bare equality filter's boolean + mask beats Polars' plan overhead — immaterial at <1 ms. +- **Seeded / selective lookups are an indexing problem**, not an engine race: with the + opt-in resident index they run in low single-digit milliseconds on every engine and stay + flat in graph size — see [F5] below and :doc:`index_adjacency`. + +Current measured numbers: :ref:`gfql-0580-numbers` (release-verified sweep) and the bulk +engine comparison on the same page. .. _gfql-vs-external-tools: @@ -171,9 +134,10 @@ GFQL vs external graph tools GFQL is **dataframe-native**: ``pip install``, then query your existing pandas / Polars / cuDF frame in-process — no separate database to stand up, no ETL to load, no cluster. Graph databases (Neo4j, Kuzu) are a **system-of-record** you provision and ingest into first. The -table below is deliberately conservative: every speedup is stated with its condition, ``>`` -and did-not-finish markers are kept, and where we have no head-to-head we say **not -benchmarked** rather than guess. +table below is deliberately conservative: wins are stated with their conditions, losses are +reported as-is, and where we have no head-to-head we say **not benchmarked** rather than +guess. The measured pairs behind every claim live in :doc:`performance` (and +:doc:`index_adjacency` for the index benchmarks). .. list-table:: :header-rows: 1 @@ -185,37 +149,42 @@ benchmarked** rather than guess. - Where it complements / GFQL doesn't claim * - **Neo4j + GDS** - Server + GDS library; stand up a DB and ETL your data in. - - **Filter→PageRank→filter pipeline**, dgx-spark GB10, warm median: Twitter 2.4M — - 13.83 s Neo4j vs 2.55 s GFQL-CPU / **0.30 s GFQL-GPU (46×)**; GPlus 30M — - **>187 s (did-not-finish)** vs 75.78 s CPU / **3.33 s GPU (>56×)**. - - Neo4j remains the transactional system-of-record; run the read-heavy analytics in - GFQL. See :doc:`benchmark_filter_pagerank`. + - **LDBC SNB interactive pairs** (same box, warm): GFQL wins most clean pairs — + seeded lookups, message content/creator projections, one-hop expands. **End-to-end + filter→PageRank→filter pipelines**: GFQL wins by large margins on CPU and + especially GPU, including a case where Neo4j did not finish. Measured pairs in + :ref:`gfql-0580-numbers`; pipeline benchmark in :doc:`benchmark_filter_pagerank`. + - **Neo4j wins some interactive shapes** (e.g. the recent-replies pair) — reported + as-is in :doc:`performance`. Neo4j remains the transactional system-of-record; run + the read-heavy analytics in GFQL. * - **Kuzu** - Embedded graph DB; still a separate store to load + index. - - **Seeded index lookup** (0.8M nodes / 6.4M edges): 1-hop **0.123 ms vs 1.15 ms - (9.4×)**, 2-hop **0.150 ms vs 4.25 ms (28×)**; prepared-Kuzu LiveJournal 35M ~ **17×** - typical seed, 6× hub. **Bulk frontier expansion** (LiveJournal 35M, 1-hop, many - seeds): **22× Kuzu**, up to **87× at k=100k**. See :doc:`index_adjacency`. - - **Not claimed:** cyclic / multi-way-join patterns (triangles, cliques) where Kuzu's - worst-case-optimal joins can win. Use Kuzu as the store; GFQL for bulk read analytics. + - GFQL typically wins **multi-join OLAP** (by one to two orders of magnitude with + ``engine='polars'``), **seeded / selective index lookups**, and **bulk frontier + expansion**. Measured pairs in :ref:`gfql-0580-numbers`; index benchmarks in + :doc:`index_adjacency`. + - **Embedded Kuzu typically wins single-table aggregates and seeded + property-projection lookups** (shapes GFQL's covered fast paths don't yet reach) — + exact ratios in :doc:`performance`. GFQL's strengths are traversals, multi-join + OLAP, and covered seeded shapes. Also **not claimed:** cyclic / multi-way-join + patterns (triangles, cliques) where Kuzu's worst-case-optimal joins can win. Use + Kuzu as the store; GFQL for bulk read analytics. * - **LadybugDB** - Actively-maintained **Kuzu fork** (Kuzu is archived); embedded C++, strongly-typed Cypher, opt-in ART *or* hash indexing, zero-copy Arrow/CSR scans, and **out-of-core billion-scale** (query a 1.8B-edge graph in <8 GB RAM). - Against **LadybugDB's published numbers** for their own 5M-node / 20M-edge suite - (their figures, their hardware; GFQL measured separately on an NVIDIA DGX Spark - GB10 running the identical Cypher ``MATCH … RETURN`` row pipeline, each engine on - its **native** frames — a cross-machine comparison, so read the ratios as - indicative): GFQL **wins the scan-shaped ops** — full node scan **~65×** (polars - 58 ms vs 3789 ms), id **range ~1.2×** (polars 6.1 ms vs 7.5 ms), relationship - property/rowid scans **~3.5–3.7×** (cuDF 4.2 s vs ~15 s). **Point lookup** (single - id) is ~4 ms vs Ladybug's ~0.3 ms — a full columnar scan vs a B-tree/hash **index - seek**; close in absolute terms, and a resident GFQL node-id index (tracked in - issue #1676) should close it. Ladybug still wins the two ops backed by - persistent structure: point lookups and a relationship ``COUNT(*)`` (an O(1) cached - count vs GFQL's O(E) endpoint-validated scan — a dataframe has no referential - integrity). GFQL's angle is dataframe-native, in-process, and GPU-accelerated with - no separate store to load/index. + (their figures, their hardware; GFQL measured separately on the same Cypher row + pipeline, each engine on its **native** frames — a cross-machine comparison, so + read it as indicative): GFQL **wins the scan-shaped ops** — full node scans and + relationship property/rowid scans — by large margins. **Point lookups** are close + in absolute terms but favor Ladybug's B-tree/hash **index seek** over a columnar + scan; a resident GFQL node-id index (tracked in issue #1676) should close that. + Ladybug wins the two ops backed by persistent structure: point lookups and a + cached relationship ``COUNT(*)`` (an O(1) count vs GFQL's O(E) endpoint-validated + scan — a dataframe has no referential integrity). GFQL's angle is + dataframe-native, in-process, and GPU-accelerated with no separate store to + load/index. Details and reproducer in the :doc:`performance` methodology notes. - **Complement:** Ladybug is a durable embedded store with an out-of-core mode (billion-scale in <8 GB RAM); GFQL is a query engine over your dataframes. GFQL's *default* is in-memory, but it is **not limited to it** — Polars streaming @@ -238,10 +207,9 @@ benchmarked** rather than guess. - *Distributed* graph engine on a Spark cluster; provision + tune the cluster. - GFQL is *single-node* (CPU or one GPU): 100M+ edges in-process on **one machine**, no cluster to stand up, interactive latency — and a single node often matches or beats - Spark on read-heavy traversal and, with the GPU engine, PageRank at a fraction of the cost. - Head-to-head on LiveJournal (35M) and Orkut (117M): GFQL wins filter/traversal 1.3–43× - even on CPU, and the GPU engine wins PageRank ~10–15×; on CPU, PageRank via igraph is - *slower* than GraphFrames — see :doc:`benchmark_graphframes`. + Spark on read-heavy filter/traversal even on CPU, while the GPU engine wins PageRank + by an order of magnitude; on CPU, PageRank via igraph is *slower* than GraphFrames. + Measured head-to-head in :doc:`benchmark_graphframes`. - Reach for GraphFrames when the graph genuinely exceeds one machine's memory. Motif / triangle / multi-way-join queries **run** in GFQL but are not yet perf-benchmarked. * - **PuppyGraph** @@ -254,9 +222,11 @@ benchmarked** rather than guess. GFQL **complements** a graph database more than it replaces one: keep Neo4j or Kuzu as the system-of-record, and do the read-heavy search + analytics in GFQL so ETL, traversal, and scoring stay in one in-process dataframe pipeline. Route by shape — **selective** seeded -lookups favor the GFQL index (up to 28× Kuzu, 16.9× Neo4j on 2-hop), **bulk** frontier -expansion and full pipelines favor Polars / GPU (22–87× Kuzu; **46–56× Neo4j** on the -filter→PageRank→filter pipeline). Against the **distributed** engines the axis is different: +lookups favor the GFQL index, **multi-join OLAP** favors Polars, and **bulk** frontier +expansion and full pipelines favor Polars / GPU (measured pairs in :doc:`performance` and +:doc:`index_adjacency`). The inverse holds too: embedded engines win single-table +aggregates and seeded property-projection lookups. +Against the **distributed** engines the axis is different: GFQL trades horizontal scale-out for zero cluster/warehouse setup and interactive latency — choose it below the single-machine ceiling (100M+ edges fit in-process; a cluster is only needed once the graph genuinely exceeds one node's memory), and complement PuppyGraph's @@ -285,7 +255,7 @@ Decision matrix - > ~10K - CPU - ``polars`` - - wins from ~10K (2.7x); up to ~38x pandas, ~15x cuDF at 100M [F1] + - wins from ~10K; gap grows to order-of-magnitude at scale [F1] * - Heavy multi-hop (2-hop+) - large - GPU @@ -313,18 +283,20 @@ Decision matrix - O(degree), not an engine choice [F5] **[F1] CPU crossover is ~10K, not ~1M.** For the common graph-query shapes (traversal, -``WHERE``/``ORDER``, aggregation) CPU Polars beats pandas from ~10K edges up (2.7-18× in our -runs). Pandas only edges out on a trivial sub-millisecond operation (a bare equality mask), +``WHERE``/``ORDER``, aggregation) CPU Polars beats pandas from ~10K edges up (measured +crossover in :doc:`performance`). +Pandas only edges out on a trivial sub-millisecond operation (a bare equality mask), where the absolute difference is immaterial. The real small-size floor is **GPU-only** — cuDF / Polars-GPU need enough work to amortize kernel launch ([F2]). **[F2] GPU is work-bound, not size-bound.** A GPU wins when there is enough work to amortize -its ~3 ms kernel-launch floor: big frontiers, dense joins, full-graph aggregation. Tiny or -seeded work finishes faster on CPU. +its millisecond-scale kernel-launch floor: big frontiers, dense joins, full-graph +aggregation. Tiny or seeded work finishes faster on CPU. -**[F3] Polars-GPU memory pressure.** On an extreme single materialization (~85M output rows, -2-hop from 100K seeds on Orkut) raw ``cudf`` leads (6.0 s) and ``polars-gpu`` slips (8.6 s) -as its in-memory GPU executor comes under memory pressure. Prefer ``cudf`` for that regime. +**[F3] Polars-GPU memory pressure.** On an extreme single materialization (tens of millions +of output rows from one join) raw ``cudf`` leads and ``polars-gpu`` slips as its in-memory +GPU executor comes under memory pressure (measured in :doc:`performance`). Prefer ``cudf`` +for that regime. **[F4] Polars-GPU is GPU-or-error.** It never silently falls back to CPU and reports the result as a GPU run (see *Honesty* below). @@ -332,12 +304,12 @@ result as a GPU run (see *Honesty* below). **[F5] Selective traversal is an indexing problem, not an engine choice.** A seeded ``hop`` from a few nodes is fastest with the opt-in **CSR adjacency index** (``g.gfql_index_all()`` / ``g.create_index(...)``, ``index_policy=``), which turns the O(E) scan into an O(degree) -gather — flat in graph size, and 9–28× faster than Kuzu / Neo4j on selective lookups. It works -on all four engines, but seeded work is so small that **CPU wins**: on LiveJournal 35M a -typical-seed 1-hop is ~0.13 ms on pandas and ~0.16 ms on Polars (numpy ``searchsorted``) vs -~3 ms on cuDF (GPU kernel-launch floor) — the clean inverse of bulk, where the GPU pulls -ahead. So pick the index for selective traversal and a CPU engine to drive it. See -:doc:`index_adjacency` for the full guide. +gather — flat in graph size (measured flat-scaling numbers in :ref:`gfql-0580-numbers`; +index routing for the native seeded ``g.hop()`` currently engages on pandas, not yet the +Polars hop path). It works on all four engines, but seeded work is so small that **CPU +wins** — sub-millisecond on pandas/Polars vs the GPU kernel-launch floor on cuDF — the +clean inverse of bulk, where the GPU pulls ahead. So pick the index for selective +traversal and a CPU engine to drive it. See :doc:`index_adjacency` for the full guide. Switching engines ----------------- @@ -386,7 +358,10 @@ never selects Polars or Polars-GPU**, so those two are always an explicit opt-in .. tip:: For selective, seeded traversal, build the CSR adjacency index once with ``g.gfql_index_all()`` (or ``index_policy=``) — it works on all four engines - and turns the O(E) scan into an O(degree) gather. See :doc:`index_adjacency`. + and turns the O(E) scan into an O(degree) gather. **Polars frames currently need + the engine passed explicitly** — ``g.gfql_index_all(engine='polars')`` — because an + AUTO build swaps Polars frames to pandas (fix tracked in PR #1767). + See :doc:`index_adjacency`. .. _gfql-offengine-calls: @@ -480,9 +455,9 @@ little latency for a much larger working set: results **larger than device memory** (the default in-memory executor would OOM). -Both are **off by default** on purpose: they add overhead that *regresses* -small/interactive work (~0.86× at 100K edges), and for the in-memory regime this -page measures, the default is faster and more stable. Results are +Both are **off by default** on purpose: they add overhead that mildly *regresses* +small/interactive work, and for the in-memory regime the benchmarks measure, the +default is faster and more stable. Results are **parity-identical** to the default — streaming changes *how* the plan runs, not *what* it returns. @@ -549,11 +524,14 @@ Honesty matters more than a bigger number: misreport pandas performance as Polars (see *Honesty*). - **One extreme materialization (80M+ output rows):** prefer ``cudf`` over ``polars-gpu`` (footnote F3). -- **vs graph databases:** GFQL-Polars beats embedded kuzu on frontier expansion (up to ~87x - on LiveJournal 1-hop in our runs — reproducer ``benchmarks/gfql/index_vs_kuzu_prepared.py``), - and separately beats Neo4j+GDS end-to-end (:doc:`benchmark_filter_pagerank`). The honest - boundary: kuzu's worst-case-optimal joins target **cyclic / multi-way join** patterns - (triangles, cliques) that we have **not** yet benchmarked, and kuzu may lead there. +- **vs graph databases:** GFQL-Polars typically beats embedded kuzu on multi-join OLAP and + bulk frontier expansion by one to two orders of magnitude (measured pairs in + :doc:`performance`; reproducer ``benchmarks/gfql/index_vs_kuzu_prepared.py``); it + separately beats Neo4j+GDS end-to-end (:doc:`benchmark_filter_pagerank`). The honest + boundary: embedded kuzu **wins single-table aggregates and seeded property-projection + lookups** (exact ratios in :doc:`performance`), and its worst-case-optimal joins + target **cyclic / multi-way join** patterns (triangles, cliques) that we have **not** + yet benchmarked, and kuzu may lead there. Parity and honesty ------------------ @@ -580,22 +558,10 @@ Parity and honesty Methodology ----------- -- Host: NVIDIA DGX Spark (GB10 Grace-Blackwell, unified memory — the F3 memory-pressure - boundary is partly a property of this box), RAPIDS container - ``graphistry/test-rapids-official:26.02-gfql-polars``. -- Datasets: `SNAP `_ **com-LiveJournal** (35M edges), - **com-Orkut** (117M edges). The order-of-magnitude filter/lookup figure is from a separate - **LDBC SNB sf1** benchmark, not the table above. -- Measurement: **warm median** after 2 warmups (5 timed runs on Orkut, 8 on LiveJournal); - every reported cell is **guarded** — the result rows are verified identical across engines - before any timing is kept. -- Reproduce: ``benchmarks/gfql/index_bulk_olap_bench.py`` (engine comparison), - ``benchmarks/gfql/pandas_vs_polars.py``, and ``benchmarks/gfql/index_vs_kuzu_prepared.py`` - (vs kuzu). Numbers on this page are rendered from saved runs; the page does not re-run them. -- **LadybugDB row**: the Ladybug figures are **their published results on their hardware**; - the GFQL side ran on the host above via ``benchmarks/gfql/bench_ladybug_cypher.py`` - (5M/20M synthetic per their suite shape, native frames per engine, warm medians) — a - cross-machine comparison, disclosed as such in the row. +The measured tables and their full methodology — hosts, datasets, warm-median protocol, +cross-engine parity guards, and reproducer scripts — live with the numbers on the +:doc:`performance` page (release-verified tag sweeps plus the prior large-graph bulk +sweep) and in :doc:`index_adjacency` for the resident-index benchmarks. Install ------- diff --git a/docs/source/gfql/index.rst b/docs/source/gfql/index.rst index 3f53b064fd..2fc8d99574 100644 --- a/docs/source/gfql/index.rst +++ b/docs/source/gfql/index.rst @@ -36,7 +36,7 @@ Recommended paths: - New to GFQL: :doc:`overview` -> :doc:`quick` -> :doc:`where` -> :doc:`return` - Running Cypher syntax in GFQL: :doc:`cypher` -> :doc:`quick` -> :doc:`return` -> :doc:`spec/cypher_mapping` -- Faster on CPU (no GPU): :doc:`engines` -> :doc:`performance` (one keyword, ``engine='polars'``, up to ~38x over pandas) +- Faster on CPU (no GPU): :doc:`engines` -> :doc:`performance` (one keyword, ``engine='polars'``, often an order of magnitude faster on query-heavy workloads) - Performance path (intro -> engine choice -> GPU -> remote GPU): :doc:`about` -> :doc:`engines` -> :doc:`performance` -> :doc:`remote` - Fast seeded lookups (start from known nodes, like a DB index): :doc:`indexing` (build once with ``gfql_index_all()``, reused automatically) -> :doc:`index_adjacency` (O(degree), flat in graph size, 9-28x vs Kuzu/Neo4j) - Translating existing Cypher to native GFQL: :doc:`spec/cypher_mapping` diff --git a/docs/source/gfql/overview.rst b/docs/source/gfql/overview.rst index b3442cd5ac..2bca4bc5b8 100644 --- a/docs/source/gfql/overview.rst +++ b/docs/source/gfql/overview.rst @@ -316,7 +316,7 @@ Key advantages of GFQL Let: Leveraging GPU Acceleration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -GFQL runs the same query on four interchangeable engines, all returning identical results: ``pandas`` (CPU, default), ``polars`` (CPU columnar — up to ~38x over pandas, **no GPU**), ``cudf`` (NVIDIA GPU), and ``polars-gpu`` (NVIDIA GPU). ``engine='auto'`` resolves to ``cudf`` for cuDF input and ``pandas`` otherwise; ``polars`` / ``polars-gpu`` are explicit opt-in (``auto`` never selects them — **so a Polars-frame graph run with the default is coerced to pandas; pass** ``engine='polars'`` **to stay native**). Neither silently bridges: ``polars-gpu`` is GPU-or-error, and unsupported Polars/Cypher shapes are declined during validation, compilation, or planning before execution rather than falling back to pandas. See :doc:`Choosing an Engine ` for the decision matrix and benchmarks. +GFQL runs the same query on four interchangeable engines, all returning identical results: ``pandas`` (CPU, default), ``polars`` (CPU columnar — often an order of magnitude faster on query-heavy workloads, **no GPU**), ``cudf`` (NVIDIA GPU), and ``polars-gpu`` (NVIDIA GPU). ``engine='auto'`` resolves to ``cudf`` for cuDF input and ``pandas`` otherwise; ``polars`` / ``polars-gpu`` are explicit opt-in (``auto`` never selects them — **so a Polars-frame graph run with the default is coerced to pandas; pass** ``engine='polars'`` **to stay native**). Neither silently bridges: ``polars-gpu`` is GPU-or-error, and unsupported Polars/Cypher shapes are declined during validation, compilation, or planning before execution rather than falling back to pandas. See :doc:`Choosing an Engine ` for the decision matrix and benchmarks. When you use cuDF (GPU) dataframes with ``engine='auto'``, GFQL executes queries on the GPU for massive speedups. diff --git a/docs/source/gfql/performance.rst b/docs/source/gfql/performance.rst index 6b4e06a181..ee535f9599 100644 --- a/docs/source/gfql/performance.rst +++ b/docs/source/gfql/performance.rst @@ -3,30 +3,168 @@ GFQL Performance: Vectorization and GPU Acceleration ==================================================== +This page is the **canonical home for GFQL benchmark numbers** — the measured tables live +here (and, for the resident-index benchmarks, in :doc:`index_adjacency`), while the rest of +the docs make stable qualitative claims and link back here. + Engine speedups at a glance --------------------------- GFQL runs the **same query** on four interchangeable engines — ``pandas`` (default), ``polars`` (CPU, columnar), ``cudf`` (NVIDIA GPU), and ``polars-gpu`` (GPU) — and returns -**identical results** on each (differential parity is a release gate). Unsupported -engine/query combinations are declined before execution during validation, compilation, -or planning rather than silently falling back. The biggest, easiest win is one keyword, -**no GPU required**: +**identical results** on each (differential parity is a release gate; every four-engine +number on this page was kept only after the result rows were verified identical across +engines, and the cross-database pairs were validated against expected result rows). +Unsupported engine/query combinations are declined before execution during validation, +compilation, or planning rather than silently falling back. The biggest, easiest win is one +keyword, **no GPU required**: .. doc-test: skip .. code-block:: python g.gfql(query) # engine='pandas' (default) - g.gfql(query, engine='polars') # up to ~38x faster on real graphs, same results + g.gfql(query, engine='polars') # often much faster on query-heavy workloads, same results + +For example, in the release-verified sweep below, the LDBC SNB SF1 seed-lookup drops from +**1,299.6 ms** on eager pandas to **106.1 ms** with ``engine='polars'`` — a **12.3×** +one-keyword speedup, no GPU, results identical. + +.. _gfql-0580-numbers: + +Release-verified numbers (0.58.0) +--------------------------------- + +All numbers in this section were measured on the **0.58.0 release tag** on an NVIDIA DGX +Spark (GB10), warm medians over N=30 runs. The four-engine numbers (seeded fast paths, +resident index, scaling) were kept only after the result rows were asserted identical +across engines; the competitor pairs (vs Neo4j, vs Kuzu) were validated against expected +result rows and cross-database value/row-count checks. + +Seeded typed-hop fast path +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A seeded typed hop — Cypher ``MATCH (m {id: ...})-[:T]->(p) RETURN p`` on a 50k-node / +200k-edge graph — is the bread-and-butter selective lookup. The release's fast path speeds +it up on every engine (before → after within the sweep): + +.. list-table:: + :header-rows: 1 + :widths: 25 20 20 15 + + * - Engine + - Before + - After (fast path) + - Speedup + * - ``pandas`` + - 29.9 ms + - **2.46 ms** + - 12.1× + * - ``polars`` + - 13.8 ms + - **2.28 ms** + - 6.1× + * - ``cudf`` + - 30.1 ms + - **4.89 ms** + - 6.1× + * - ``polars-gpu`` + - 25.2 ms + - **2.49 ms** + - 10.1× + +The native chain form of the same query is faster still: pandas 21.1 → **1.65 ms** +(12.8×), cuDF 23.2 → **3.84 ms** (6.0×). + +With a resident index +~~~~~~~~~~~~~~~~~~~~~ + +Building the opt-in resident index once (``g.gfql_index_all()``) makes the covered-shape +seeded lookup faster again — pandas **1.74 ms**, polars **1.59 ms**, polars-gpu +**1.91 ms**, cudf **5.78 ms**. + +.. warning:: + **Polars + index: pass** ``engine='polars'`` **when building.** Polars frames currently + need ``g.gfql_index_all(engine='polars')`` explicitly — an AUTO build swaps Polars + frames to pandas. The fix is tracked in PR #1767. + +Scaling: flat in graph size +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +With the resident adjacency index, a native seeded 1-hop ``g.hop()`` on pandas stays +**flat at 0.159–0.164 ms from 0.25M to 32M edges** (constant average degree 4): the index +turns the O(E) scan into an O(degree) gather, so seeded latency does not grow with the +graph. (Pandas-only today — the Polars hop path is not yet index-routed.) + +vs Neo4j (LDBC SNB interactive SF1) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Warm-median latency, same query, identical result rows (**Orkut**, 117M edges, SNAP): +Same box, warm, against Neo4j 5.26 — GFQL wins **4 of 5** clean pairs: .. list-table:: :header-rows: 1 - :widths: 40 15 15 15 15 + :widths: 30 20 20 15 - * - Workload (117M edges) + * - Query + - GFQL + - Neo4j 5.26 + - Winner + * - seed-lookup + - **106.1 ms** + - 143.7 ms + - GFQL + * - message-content + - **7.1 ms** + - 23.0 ms + - GFQL + * - message-creator + - **6.8 ms** + - 27.7 ms + - GFQL + * - one-hop-expand + - **111.9 ms** + - 180.7 ms + - GFQL + * - recent-replies + - 209.6 ms + - **104.0 ms** + - Neo4j + +The message-creator flip shipped in this release via property-seeded resident-index +gathers (PR #1770). Neo4j still wins recent-replies — reported as-is. + +OLAP multi-join +~~~~~~~~~~~~~~~ + +On the graph-benchmark OLAP multi-join queries at 100k-node scale with +``engine='polars'``: **q8 runs in 5.0 ms vs 1,004 ms for embedded Kuzu (200×)**; q9 is +**14.2×**. + +When not to use GFQL +~~~~~~~~~~~~~~~~~~~~ + +Honesty matters more than a bigger number. In the same cross-DB sweep, **embedded Kuzu +wins single-table aggregates (2–4×) and seeded property-projection lookups (2.4–64×)**. +GFQL's strengths are **traversals, multi-join OLAP, and covered seeded shapes** — route by +workload, and keep a database as the system-of-record where one fits. + +.. _gfql-bulk-sweep: + +Bulk engine comparison (prior sweep) +------------------------------------ + +The numbers in this section are from an earlier, pre-0.58.0 bulk sweep on SNAP +**com-LiveJournal** (35M edges) and **com-Orkut** (117M edges) — retained as the +bulk-workload reference until rerun on a current tag. + +Same query, same answers, four engines — warm-median latency on Orkut (3.1M nodes / +117M edges), measured on a single machine: + +.. list-table:: + :header-rows: 1 + :widths: 34 16 16 16 16 + + * - Workload (Orkut, 117M edges) - ``pandas`` - ``polars`` - ``cudf`` @@ -36,18 +174,73 @@ Warm-median latency, same query, identical result rows (**Orkut**, 117M edges, S - **68 ms** - 1005 ms - 63 ms + * - 2-hop from 10K seeds + - 18161 ms + - 2695 ms + - 2774 ms + - **1518 ms** * - Full out-degree aggregation - 799 ms - 205 ms - 314 ms - **167 ms** + * - 2-hop from 100K seeds (~85M output rows) + - 28822 ms + - 8215 ms + - **6002 ms** + - 8559 ms + +Reading the table: + +- **Polars-CPU beat pandas up to ~38x** on bulk traversal and ~4x on aggregation — **with + no GPU**. On the 1-hop workload it was ~38x faster than pandas (68 ms vs 2613 ms). +- **Polars-CPU also beat cuDF** on these shapes (68 ms vs 1005 ms on 1-hop). cuDF runs + GFQL *eagerly*, op by op (a kernel launch + a materialized intermediate per hop), while + Polars builds **one fused lazy plan and collects once**. The fused plan wins until the + work is large enough to amortize GPU launch costs. +- **Polars-GPU was fastest on heavy multi-hop** (2-hop from 10K seeds: 1518 ms) and on + aggregation — the same fused plan, executed on the GPU. +- **cuDF won the one extreme case** — a 2-hop from 100K seeds materializing ~85M output + rows (6.0 s) — where raw GPU throughput on a single massive join overtakes everything + and Polars-GPU comes under memory pressure. +- On LiveJournal (35M edges) the pattern held: 1-hop from 10K seeds was pandas 1129 ms → + polars **37 ms** (~30x). +- The CPU crossover is early: on LiveJournal subsampled (CPU, warm-median), 1-hop + traversal was 2.7× / 4.5× / 7.6× and ``WHERE``+``ORDER`` 3.0× / 3.0× / 18× over pandas + at 10K / 100K / 1M edges. The only case pandas edged out was a trivial sub-millisecond + operation (a bare node-equality filter), where its boolean mask beats Polars' plan + overhead — immaterial at <1 ms. Reproducer: ``benchmarks/gfql/index_crossover_bench.py``. + +Methodology (prior sweep) +~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Host: NVIDIA DGX Spark (GB10 Grace-Blackwell, unified memory — the memory-pressure + boundary above is partly a property of this box), RAPIDS container + ``graphistry/test-rapids-official:26.02-gfql-polars``. +- Datasets: `SNAP `_ **com-LiveJournal** (35M edges), + **com-Orkut** (117M edges). +- Measurement: **warm median** after 2 warmups (5 timed runs on Orkut, 8 on LiveJournal); + every reported cell is **guarded** — the result rows are verified identical across + engines before any timing is kept. +- Reproduce: ``benchmarks/gfql/index_bulk_olap_bench.py`` (engine comparison), + ``benchmarks/gfql/pandas_vs_polars.py``, and ``benchmarks/gfql/index_vs_kuzu_prepared.py`` + (vs kuzu). Numbers on this page are rendered from saved runs; the page does not re-run + them. +- **LadybugDB comparison** (referenced qualitatively in :doc:`engines`): the Ladybug + figures are **their published results on their hardware**; the GFQL side ran on the host + above via ``benchmarks/gfql/bench_ladybug_cypher.py`` (5M/20M synthetic per their suite + shape, native frames per engine, warm medians) — a cross-machine comparison, disclosed + as such. GFQL won the scan-shaped ops by large margins (full node scan ~65×, + relationship property/rowid scans ~3.5–3.7×); Ladybug won the two ops backed by + persistent structure — point lookups (index seek vs columnar scan) and a cached + relationship ``COUNT(*)``. There is **no universal winner**: ``polars`` typically takes over from ~10K edges up (``pandas`` still wins trivial sub-millisecond operations), and the right GPU engine depends on the workload. See :doc:`engines` for the full decision matrix, the honest -"when *not* to use Polars", the cuDF-vs-Polars-GPU comparison, and the methodology + reproducer -scripts behind these numbers. The end-to-end CPU/GPU-vs-Neo4j benchmark is in -:doc:`benchmark_filter_pagerank`. +"when *not* to use Polars", and the cuDF-vs-Polars-GPU comparison. The end-to-end +CPU/GPU-vs-Neo4j pipeline benchmark is in :doc:`benchmark_filter_pagerank`, and the +Spark GraphFrames head-to-head is in :doc:`benchmark_graphframes`. How GFQL is fast ---------------- @@ -77,7 +270,8 @@ your workload grows into GPU territory. See :doc:`engines` for exactly when each Next Steps ---------- -- **Choose an engine**: :doc:`engines` — the full decision matrix, methodology, and reproducers. +- **Choose an engine**: :doc:`engines` — the full decision matrix and qualitative guidance. +- **Selective lookups**: :doc:`index_adjacency` — the resident index behind the flat-scaling numbers. - **End-to-end benchmark**: :doc:`benchmark_filter_pagerank` — CPU/GPU vs Neo4j+GDS. - **Explore GFQL**: :ref:`10min-gfql`. **Get started**: :ref:`10min-pygraphistry`. - **Ecosystem**: `Apache Arrow `_ and `NVIDIA RAPIDS `_. diff --git a/docs/source/gfql/quick.rst b/docs/source/gfql/quick.rst index f1f87f3f9d..bc94fa9933 100644 --- a/docs/source/gfql/quick.rst +++ b/docs/source/gfql/quick.rst @@ -27,7 +27,7 @@ Basic Usage :meth:`gfql ` sequences multiple matchers for more complex patterns of paths and subgraphs - **query**: Sequence of graph node/edge matchers and optional row-pipeline call steps (for example, `rows()`, `where_rows()`, `return_()`, `order_by()`, `limit()`), or an equivalent GFQL chain object. -- **engine**: Optional execution engine. Engine is typically not set, defaulting to `'auto'`. Use `'polars'` for a CPU columnar speedup (up to ~38x over pandas, no GPU), `'cudf'` or `'polars-gpu'` for NVIDIA GPU acceleration, and `'pandas'` for the default CPU path. See :doc:`Choosing an Engine `. +- **engine**: Optional execution engine. Engine is typically not set, defaulting to `'auto'`. Use `'polars'` for a CPU columnar speedup (often an order of magnitude faster on query-heavy workloads, no GPU — see :doc:`Performance `), `'cudf'` or `'polars-gpu'` for NVIDIA GPU acceleration, and `'pandas'` for the default CPU path. See :doc:`Choosing an Engine `. Native GFQL chains are typed Python inputs. Pass the list, dict envelope, or ``Chain`` object itself; strings passed to ``g.gfql(...)`` are interpreted as @@ -408,7 +408,7 @@ engine/query combinations are rejected before execution during validation, compi planning rather than silently falling back. Pick one with ``engine=``. See :doc:`Choosing an Engine ` for the full decision matrix. -- **CPU columnar speedup (no GPU):** ``'polars'`` — up to ~38x over pandas on real graphs. +- **CPU columnar speedup (no GPU):** ``'polars'`` — often an order of magnitude faster on query-heavy workloads, identical results (measured numbers in :doc:`Performance `). .. code-block:: python