docs(gfql): refresh perf docs with 0.58.0 release-tag-verified numbers#1772
Merged
Conversation
lmeyerov
added a commit
that referenced
this pull request
Jul 22, 2026
Review follow-ups on PR #1772: - engines.rst Methodology: the prior-sweep bullet no longer dangles with a bare colon over sibling bullets; reworded to state the remaining bullets describe that sweep's setup. - performance.rst + engines.rst: scope the verification claims — cross-engine result-row identity applies to the four-engine fast-path/index/scaling numbers; the vs-Neo4j and vs-Kuzu pairs were validated against expected result rows and cross-database value/row-count checks, not cross-engine identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
Contributor
Author
|
Review-skill pass (session): 2 minor findings, both FIXED in 3e2208b — (1) dangling methodology grouping bullet reworded self-contained; (2) parity claim scoped accurately (cross-engine identity for 4-engine numbers; expected-rows + cross-DB value checks for Neo4j/Kuzu pairs). Sphinx rebuild clean on touched files after the fix. All quoted numbers are 0.58.0-tag-sweep values (DGX Spark, warm N=30). Note: merge-order note added re #1771 overlap (gfql/index.rst + CHANGELOG). Ready for human review. 🤖 Generated with Claude Code |
Refresh the GFQL performance and engine-selection docs to lead with numbers measured on the 0.58.0 release tag (DGX Spark GB10, warm medians N=30, results verified identical across engines): - Seeded typed-hop Cypher fast path, all four engines (pandas 29.9->2.46ms 12.1x, polars 13.8->2.28ms 6.1x, cuDF 30.1->4.89ms 6.1x, polars-gpu 25.2->2.49ms 10.1x; native chain pandas 21.1->1.65ms 12.8x, cuDF 23.2->3.84ms 6.0x) - Resident-index covered-shape lookups (pandas 1.74 / polars 1.59 / polars-gpu 1.91 / cudf 5.78 ms) with the 0.58.0 caveat that Polars frames need gfql_index_all(engine='polars') explicitly (fix in PR #1767) - Flat seeded-hop scaling on pandas: 0.159-0.164ms from 0.25M to 32M edges (pandas-only; polars hop not yet index-routed) - One-keyword engine='polars' LDBC SNB SF1 seed-lookup: 1,299.6->106.1ms (12.3x, same build) — replaces the stale "up to ~38x" headline in engines/performance/quick/about/overview/index - LDBC SNB interactive SF1 vs Neo4j 5.26, same box, warm: GFQL wins 4 of 5 clean pairs; Neo4j wins recent-replies (reported as-is) - OLAP multi-join vs embedded Kuzu (100k scale, engine='polars'): q8 5.0ms vs 1,004ms (200x), q9 14.2x — with the honest inverse (Kuzu wins single-table aggregates 2-4x, seeded property-projection lookups 2.4-64x) Prior Orkut/LiveJournal bulk-sweep tables are retained but relabeled as prior-release measurements; methodology disclosures kept and extended with the 0.58.0 tag-sweep provenance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
Review follow-ups on PR #1772: - engines.rst Methodology: the prior-sweep bullet no longer dangles with a bare colon over sibling bullets; reworded to state the remaining bullets describe that sweep's setup. - performance.rst + engines.rst: scope the verification claims — cross-engine result-row identity applies to the four-engine fast-path/index/scaling numbers; the vs-Neo4j and vs-Kuzu pairs were validated against expected result rows and cross-database value/row-count checks, not cross-engine identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
…elsewhere Restructure per review: performance.rst is now the canonical benchmark-numbers page (alongside index_adjacency for the index benchmarks) so a benchmark rerun touches one page, not six. - performance.rst: keeps the 0.58.0 release-verified section with provenance stated once at the top; absorbs the full Orkut/LiveJournal bulk-sweep table, reading notes, crossover figures, and methodology from engines.rst with a single dating sentence instead of scattered "prior sweep" tags. - engines.rst: number tables and inline figures removed; new qualitative "How the engines compare" section; competitor rows (Neo4j, Kuzu, LadybugDB, GraphFrames), decision-matrix notes, and footnotes now make stable qualitative claims and link to performance.rst / index_adjacency; methodology section is a pointer to the numbers page. - quick.rst / about.rst / overview.rst / index.rst: one-liners reworded to stable "often an order of magnitude faster on query-heavy workloads" phrasing with links — no datasets, versions, or ratios. - Rebased on master (PR #1771 merged); CHANGELOG entries placed alongside, entry rewritten to describe the siloed architecture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
lmeyerov
force-pushed
the
docs/gfql-0.58.0-perf-refresh
branch
from
July 22, 2026 05:38
3e2208b to
7f13f98
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refresh the GFQL performance docs with 0.58.0 release-tag-verified numbers, and — per review feedback — silo all benchmark numbers into
performance.rstso a benchmark rerun touches one page (plusindex_adjacency.rst, which already silos the index benchmarks). Every other page now carries stable qualitative claims that link into the numbers page.Provenance of the new numbers: 0.58.0 tag sweep, DGX Spark (GB10), warm medians N=30. Four-engine numbers were kept only after result rows were verified identical across engines; the competitor pairs (vs Neo4j 5.26, vs embedded Kuzu) were validated against expected result rows and cross-database value/row-count checks.
Architecture (who holds what)
docs/source/gfql/performance.rst— the canonical benchmark-numbers page: the 0.58.0 release-verified section (provenance stated once at the top: seeded typed-hop fast-path table across all four engines, native chain form, resident-index covered-shape lookups with thegfql_index_all(engine='polars')caveat / PR fix(gfql): index AUTO preserves resident polars frames (kills the polars-hop O(E) tax) #1767, flat pandas seeded-hop scaling 0.25M→32M edges, the one-keyword LDBC SNB SF1 polars win 1,299.6→106.1ms / 12.3×, the 5-pair LDBC vs Neo4j table with Neo4j's recent-replies win reported as-is, OLAP q8 200× / q9 14.2× vs Kuzu, and the honest "when not to use GFQL" ratios), plus the prior Orkut/LiveJournal bulk sweep moved in fromengines.rstwith one dating sentence and its full methodology (incl. the LadybugDB cross-machine disclosure).docs/source/gfql/engines.rst— decision matrix, qualitative footnotes, honesty/parity contract, cuDF-vs-polars-gpu, streaming, and a new qualitative "How the engines compare" section; competitor rows (Neo4j, Kuzu, LadybugDB, GraphFrames) state stable win/loss shapes ("GFQL typically wins seeded traversals, covered seeded lookups, multi-join OLAP; embedded engines typically win single-table aggregates and uncovered projection shapes") and link toperformance.rst/index_adjacency.rstfor the measured pairs. No inline figures or per-claim version labels remain.quick.rst/about.rst/overview.rst/index.rst— stable one-liners ("often an order of magnitude faster on query-heavy workloads") with links; no datasets, versions, or ratios.CHANGELOG.md— entry under## [Development]→### Documentationdescribing the siloed architecture; rebased on master with the merged docs(gfql): pay-as-you-go resident indexing user guide #1771 entry placed alongside.Validation
origin/master(post-docs(gfql): pay-as-you-go resident indexing user guide #1771); the only conflict was the CHANGELOG[Development]Documentation section, resolved keep-both.python -m sphinx -b html -n --keep-going, Sphinx 8.0.2, graphviz+pandoc via conda): build succeeded, exit 0; grepped the warning log — zero warnings referencing any touched file (repo-wide warnings pre-existing and untouched).:ref:gfql-0580-numbers`` target lives inperformance.rst; all 5 referring sites in `engines.rst` resolve; no dangling `:doc:`/`:ref:` from the moved content.gfql(...)on pandas +engine='polars'(identical row counts),gfql_index_all()andgfql_index_all(engine='polars')on polars frames all run clean; no new executable code blocks added.Notes for reviewers
performance.rst(andindex_adjacency.rst) only.🤖 Generated with Claude Code
https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL