Skip to content

docs(gfql): refresh perf docs with 0.58.0 release-tag-verified numbers#1772

Merged
lmeyerov merged 3 commits into
masterfrom
docs/gfql-0.58.0-perf-refresh
Jul 22, 2026
Merged

docs(gfql): refresh perf docs with 0.58.0 release-tag-verified numbers#1772
lmeyerov merged 3 commits into
masterfrom
docs/gfql-0.58.0-perf-refresh

Conversation

@lmeyerov

@lmeyerov lmeyerov commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Refresh the GFQL performance docs with 0.58.0 release-tag-verified numbers, and — per review feedback — silo all benchmark numbers into performance.rst so a benchmark rerun touches one page (plus index_adjacency.rst, which already silos the index benchmarks). Every other page now carries stable qualitative claims that link into the numbers page.

Provenance of the new numbers: 0.58.0 tag sweep, DGX Spark (GB10), warm medians N=30. Four-engine numbers were kept only after result rows were verified identical across engines; the competitor pairs (vs Neo4j 5.26, vs embedded Kuzu) were validated against expected result rows and cross-database value/row-count checks.

Architecture (who holds what)

  • docs/source/gfql/performance.rstthe canonical benchmark-numbers page: the 0.58.0 release-verified section (provenance stated once at the top: seeded typed-hop fast-path table across all four engines, native chain form, resident-index covered-shape lookups with the gfql_index_all(engine='polars') caveat / PR fix(gfql): index AUTO preserves resident polars frames (kills the polars-hop O(E) tax) #1767, flat pandas seeded-hop scaling 0.25M→32M edges, the one-keyword LDBC SNB SF1 polars win 1,299.6→106.1ms / 12.3×, the 5-pair LDBC vs Neo4j table with Neo4j's recent-replies win reported as-is, OLAP q8 200× / q9 14.2× vs Kuzu, and the honest "when not to use GFQL" ratios), plus the prior Orkut/LiveJournal bulk sweep moved in from engines.rst with one dating sentence and its full methodology (incl. the LadybugDB cross-machine disclosure).
  • docs/source/gfql/engines.rst — decision matrix, qualitative footnotes, honesty/parity contract, cuDF-vs-polars-gpu, streaming, and a new qualitative "How the engines compare" section; competitor rows (Neo4j, Kuzu, LadybugDB, GraphFrames) state stable win/loss shapes ("GFQL typically wins seeded traversals, covered seeded lookups, multi-join OLAP; embedded engines typically win single-table aggregates and uncovered projection shapes") and link to performance.rst / index_adjacency.rst for the measured pairs. No inline figures or per-claim version labels remain.
  • quick.rst / about.rst / overview.rst / index.rst — stable one-liners ("often an order of magnitude faster on query-heavy workloads") with links; no datasets, versions, or ratios.
  • CHANGELOG.md — entry under ## [Development]### Documentation describing the siloed architecture; rebased on master with the merged docs(gfql): pay-as-you-go resident indexing user guide #1771 entry placed alongside.

Validation

  • Rebased on origin/master (post-docs(gfql): pay-as-you-go resident indexing user guide #1771); the only conflict was the CHANGELOG [Development] Documentation section, resolved keep-both.
  • Full Sphinx HTML build (python -m sphinx -b html -n --keep-going, Sphinx 8.0.2, graphviz+pandoc via conda): build succeeded, exit 0; grepped the warning log — zero warnings referencing any touched file (repo-wide warnings pre-existing and untouched).
  • Cross-ref audit: :ref:gfql-0580-numbers`` target lives in performance.rst; all 5 referring sites in `engines.rst` resolve; no dangling `:doc:`/`:ref:` from the moved content.
  • Code claims smoke-tested against the worktree earlier in this PR: gfql(...) on pandas + engine='polars' (identical row counts), gfql_index_all() and gfql_index_all(engine='polars') on polars frames all run clean; no new executable code blocks added.
  • Parity language: "results identical" / "verified identical" throughout — never "byte-identical".

Notes for reviewers

  • Invariant established: rerunning benchmarks requires touching performance.rst (and index_adjacency.rst) only.
  • Old bulk-sweep figures (Orkut/LiveJournal, ~38×, crossover ratios) are retained on the numbers page under a single pre-0.58.0 dating sentence — methodology disclosures kept, nothing invented.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

lmeyerov added a commit that referenced this pull request Jul 22, 2026
Review follow-ups on PR #1772:
- engines.rst Methodology: the prior-sweep bullet no longer dangles with
  a bare colon over sibling bullets; reworded to state the remaining
  bullets describe that sweep's setup.
- performance.rst + engines.rst: scope the verification claims —
  cross-engine result-row identity applies to the four-engine
  fast-path/index/scaling numbers; the vs-Neo4j and vs-Kuzu pairs were
  validated against expected result rows and cross-database
  value/row-count checks, not cross-engine identity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
@lmeyerov

Copy link
Copy Markdown
Contributor Author

Review-skill pass (session): 2 minor findings, both FIXED in 3e2208b — (1) dangling methodology grouping bullet reworded self-contained; (2) parity claim scoped accurately (cross-engine identity for 4-engine numbers; expected-rows + cross-DB value checks for Neo4j/Kuzu pairs). Sphinx rebuild clean on touched files after the fix. All quoted numbers are 0.58.0-tag-sweep values (DGX Spark, warm N=30). Note: merge-order note added re #1771 overlap (gfql/index.rst + CHANGELOG). Ready for human review.

🤖 Generated with Claude Code

lmeyerov and others added 3 commits July 21, 2026 22:28
Refresh the GFQL performance and engine-selection docs to lead with
numbers measured on the 0.58.0 release tag (DGX Spark GB10, warm medians
N=30, results verified identical across engines):

- Seeded typed-hop Cypher fast path, all four engines (pandas 29.9->2.46ms
  12.1x, polars 13.8->2.28ms 6.1x, cuDF 30.1->4.89ms 6.1x, polars-gpu
  25.2->2.49ms 10.1x; native chain pandas 21.1->1.65ms 12.8x, cuDF
  23.2->3.84ms 6.0x)
- Resident-index covered-shape lookups (pandas 1.74 / polars 1.59 /
  polars-gpu 1.91 / cudf 5.78 ms) with the 0.58.0 caveat that Polars
  frames need gfql_index_all(engine='polars') explicitly (fix in PR #1767)
- Flat seeded-hop scaling on pandas: 0.159-0.164ms from 0.25M to 32M
  edges (pandas-only; polars hop not yet index-routed)
- One-keyword engine='polars' LDBC SNB SF1 seed-lookup: 1,299.6->106.1ms
  (12.3x, same build) — replaces the stale "up to ~38x" headline in
  engines/performance/quick/about/overview/index
- LDBC SNB interactive SF1 vs Neo4j 5.26, same box, warm: GFQL wins 4 of
  5 clean pairs; Neo4j wins recent-replies (reported as-is)
- OLAP multi-join vs embedded Kuzu (100k scale, engine='polars'): q8
  5.0ms vs 1,004ms (200x), q9 14.2x — with the honest inverse (Kuzu wins
  single-table aggregates 2-4x, seeded property-projection lookups
  2.4-64x)

Prior Orkut/LiveJournal bulk-sweep tables are retained but relabeled as
prior-release measurements; methodology disclosures kept and extended
with the 0.58.0 tag-sweep provenance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
Review follow-ups on PR #1772:
- engines.rst Methodology: the prior-sweep bullet no longer dangles with
  a bare colon over sibling bullets; reworded to state the remaining
  bullets describe that sweep's setup.
- performance.rst + engines.rst: scope the verification claims —
  cross-engine result-row identity applies to the four-engine
  fast-path/index/scaling numbers; the vs-Neo4j and vs-Kuzu pairs were
  validated against expected result rows and cross-database
  value/row-count checks, not cross-engine identity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
…elsewhere

Restructure per review: performance.rst is now the canonical
benchmark-numbers page (alongside index_adjacency for the index
benchmarks) so a benchmark rerun touches one page, not six.

- performance.rst: keeps the 0.58.0 release-verified section with
  provenance stated once at the top; absorbs the full Orkut/LiveJournal
  bulk-sweep table, reading notes, crossover figures, and methodology
  from engines.rst with a single dating sentence instead of scattered
  "prior sweep" tags.
- engines.rst: number tables and inline figures removed; new qualitative
  "How the engines compare" section; competitor rows (Neo4j, Kuzu,
  LadybugDB, GraphFrames), decision-matrix notes, and footnotes now make
  stable qualitative claims and link to performance.rst /
  index_adjacency; methodology section is a pointer to the numbers page.
- quick.rst / about.rst / overview.rst / index.rst: one-liners reworded
  to stable "often an order of magnitude faster on query-heavy
  workloads" phrasing with links — no datasets, versions, or ratios.
- Rebased on master (PR #1771 merged); CHANGELOG entries placed
  alongside, entry rewritten to describe the siloed architecture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
@lmeyerov
lmeyerov force-pushed the docs/gfql-0.58.0-perf-refresh branch from 3e2208b to 7f13f98 Compare July 22, 2026 05:38
@lmeyerov
lmeyerov merged commit 5e18967 into master Jul 22, 2026
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant