Skip to content

docs(gfql): pay-as-you-go resident indexing user guide#1771

Merged
lmeyerov merged 2 commits into
masterfrom
docs/gfql-indexing-guide
Jul 22, 2026
Merged

docs(gfql): pay-as-you-go resident indexing user guide#1771
lmeyerov merged 2 commits into
masterfrom
docs/gfql-indexing-guide

Conversation

@lmeyerov

Copy link
Copy Markdown
Contributor

Summary

New user guide docs/source/gfql/indexing.rstPay-As-You-Go Resident Indexing — the lifecycle-focused companion to the existing index_adjacency.rst (planner/benchmark) page:

  • What resident indexes are: node-id index + CSR in/out adjacency sidecars over row positions; build once with gfql_index_all(), reused automatically by seeded GFQL/Cypher lookups and g.hop().
  • What engages them on 0.58.0: seeded typed-hop fast paths (whole-row and property RETURNs), including property-seeded lookups when the graph is bound on a different key column (perf(gfql): resident-index gathers engage for property-seeded lookups (SF1-vs-Neo4j lane) #1770), and direct g.hop(). Honestly notes the general polars chain traversal (multi-hop/multi-alias) is not yet covered.
  • Staleness/validity contract: identity + fingerprint validation; rebinding .edges()/.nodes() invalidates (edge rebind leaves node_id valid — verified by execution); rebuild via gfql_index_all(); declines are safe — identical results either way.
  • Engine notes: pandas/cuDF via AUTO; polars requires gfql_index_all(engine='polars') on 0.58.0 (AUTO swaps polars frames to pandas; fix tracked in fix(gfql): index AUTO preserves resident polars frames (kills the polars-hop O(E) tax) #1767); polars-gpu rides the polars-tagged index.
  • Numbers (0.58.0 tag sweep, DGX Spark, warm medians N=30 only): seeded lookup 1.74/1.59 ms (pandas/polars) with the index vs 2.46/2.28 fast-path-only vs 29.9/13.8 general; g.hop() flat 0.159–0.164 ms from 0.25M→32M edges; ~5.7 s build on 30.6M edges.
  • Runnable end-to-end example — executed from the worktree; printed outputs in the doc match actual output.

Also: wired into the GFQL toctree + recommended paths (docs/source/gfql/index.rst), CHANGELOG entry under [Development] > Documentation.

Validation

  • Full Sphinx HTML build (sphinx-build -b html docs/source ...): build succeeded, zero warnings/errors on the touched files (pre-existing warnings elsewhere unchanged). rstcheck clean on both touched .rst files.
  • Example code executed successfully against this worktree (0.58.0 = origin/master), including the decline-safety assertion (index vs no-index results equal) and the staleness demo.

Notes for reviewers

  • API surface documented exactly as it exists on master (graphistry/compute/gfql/index/api.py): gfql_index_all, gfql_index_edges, create_index, drop_index, show_indexes — all present; gfql_index_all() skips the node_id index on non-unique ids while explicit create_index("node_id") raises, both documented.

DO NOT MERGE without review sign-off.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

New docs/source/gfql/indexing.rst: lifecycle guide to GFQL resident
indexes (gfql_index_all / gfql_index_edges / create_index / show_indexes
/ drop_index) — node-id + CSR in/out adjacency sidecars, what engages
them on 0.58.0 (seeded typed-hop fast paths incl. property RETURNs and
property-seeded lookups per #1768/#1770, direct g.hop(); general polars
chain traversal honestly noted as not yet covered), the identity +
fingerprint staleness contract with safe declines, engine notes (polars
needs gfql_index_all(engine='polars') until #1767), 0.58.0-tag measured
numbers, and a runnable end-to-end example (executed).

Wired into the GFQL toctree + recommended paths next to
index_adjacency; CHANGELOG entry under Development > Documentation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
@lmeyerov

Copy link
Copy Markdown
Contributor Author

Review-skill pass (session): no findings. Verified against source: show_indexes() columns (name/kind/key_col/engine/backend/n_keys/n_rows/nbytes/valid) match the doc's usage; AUTO-engine cross-reference to engines.rst holds; runnable example executes with the decline-safety parity assert; all quoted numbers are 0.58.0-tag-sweep values (DGX Spark, warm N=30). Sphinx build clean on touched files. Ready for human review.

🤖 Generated with Claude Code

…lifecycle + decline-safety

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
@lmeyerov
lmeyerov merged commit 48e73fc into master Jul 22, 2026
35 checks passed
lmeyerov added a commit that referenced this pull request Jul 22, 2026
…elsewhere

Restructure per review: performance.rst is now the canonical
benchmark-numbers page (alongside index_adjacency for the index
benchmarks) so a benchmark rerun touches one page, not six.

- performance.rst: keeps the 0.58.0 release-verified section with
  provenance stated once at the top; absorbs the full Orkut/LiveJournal
  bulk-sweep table, reading notes, crossover figures, and methodology
  from engines.rst with a single dating sentence instead of scattered
  "prior sweep" tags.
- engines.rst: number tables and inline figures removed; new qualitative
  "How the engines compare" section; competitor rows (Neo4j, Kuzu,
  LadybugDB, GraphFrames), decision-matrix notes, and footnotes now make
  stable qualitative claims and link to performance.rst /
  index_adjacency; methodology section is a pointer to the numbers page.
- quick.rst / about.rst / overview.rst / index.rst: one-liners reworded
  to stable "often an order of magnitude faster on query-heavy
  workloads" phrasing with links — no datasets, versions, or ratios.
- Rebased on master (PR #1771 merged); CHANGELOG entries placed
  alongside, entry rewritten to describe the siloed architecture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant