Skip to content

docs: annbatch contrast + read-once/sample-once shuffle contract#19

Open
emfdavid wants to merge 5 commits into
mainfrom
docs/annbatch-contrast-shuffle-invariants
Open

docs: annbatch contrast + read-once/sample-once shuffle contract#19
emfdavid wants to merge 5 commits into
mainfrom
docs/annbatch-contrast-shuffle-invariants

Conversation

@emfdavid

Copy link
Copy Markdown
Owner

What

Documents where insitubatch sits relative to annbatch (scverse, Ilan Gold et al., arXiv:2604.01949), and pins down the shuffle/gather contract that comparison turns on.

DESIGN.md — the evolution ledger (why / alternatives)

  • New annbatch row in What it is, by contrast: same premise, opposite bet — annbatch pre-shuffles to disk and dissolves chunks into rows; insitubatch keeps chunks whole and resident and pays with an approximate shuffle. Both carry one bounded-randomness knob (their buffer m ≈ our block_chunks); the difference is the currency — a one-time rewrite vs. read locality.
  • Shuffle section trimmed to the thesis (why approximate at all) + the contrast; mechanism deferred to architecture.
  • Ragged-batch flagged as a possible wart in Known limitations, with options, pending user feedback.

docs/architecture.md — the live contract (how it works)

  • New Read-once and sample-once section: the two orthogonal invariants (read-once = read plan + ChunkPool; sample-once = the order ledger, fancy index recomputed at gather, never stored).
  • An archive → batch figure (mermaid) staged like annbatch's Fig 1C — colour tracks the bytes (a chunk keeps its colour archive→pool), the read plan is metadata (dashed), and each batch is colour-coded slices drawn across resident chunks that stay whole.
  • The ragged per-block tail as a user caveat: batches don't span shuffle blocks, so a short final batch appears per block (steps-per-epoch = Σ ⌈block/bs⌉, not ⌈N/bs⌉); no drop_last today.

Why

DESIGN.md's own header already declares the division (evolution ledger vs. live contract in architecture.md). This enforces it: contract + diagram live in architecture, why/alternatives in DESIGN, cross-linked both ways. Net −9 lines — less sprawl, one home per topic.

Notes for review

  • The mermaid figure renders on GitHub and in the mkdocs site; worth an eyeball on the nested batch subgraphs and the #known-limitations--defects / #read-once-and-sample-once cross-link anchors.
  • No code touched; contract claims are backed by existing tests (test_order_covers_every_sample_exactly_once, test_order_handles_partial_final_chunk, the decode-once suite).

🤖 Generated with Claude Code

emfdavid and others added 5 commits July 23, 2026 05:00
…riants

Add annbatch (scverse) to the "what it is, by contrast" table as the
sharpest mirror: same premise, opposite bet (reshard-for-randomness vs
train-in-place). Add a mermaid figure staged against their Fig 1C showing
the bounded block-shuffle — chunks along the sample axis, permutation in
the read plan (metadata) kept distinct from byte movement into the pool.

Document the two orthogonal invariants (read-once = plan + ChunkPool;
sample-once = the `order` ledger; fancy index recomputed in gather, never
stored), and flag per-block ragged batches as a possible wart under
evaluation (no drop_last; short final batch per block, not per epoch).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Flip the figure to flowchart TB (stages ①→②→③→④ top-down) with each
subgraph laid out LR internally, so the stages render wide instead of
squished into a horizontal strip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…Fig 1C)

Stage ④ now renders each batch as a row of colour-coded slice nodes, one
per source chunk (same colours as the resident pool in ③) — mirroring
annbatch Fig 1C's striped mini-batches. The contrast: our chunk boxes
stay whole in ③ and feed many batches, rather than ceasing to exist.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enforce the doc division DESIGN.md's own header already declares (evolution
ledger vs live contract in architecture.md):

- architecture.md gains "Read-once and sample-once": the two invariants, the
  archive→batch figure (moved here, reframed neutral with one contrast line),
  and the ragged-per-block tail as a user caveat (step-count, BatchNorm).
- DESIGN.md's "Two invariants" subsection and the shuffle figure/mechanism
  collapse to short pointers; the Shuffle section keeps only the thesis
  (why approximate) + the annbatch contrast. Ragged-batch Known-limitations
  entry trimmed to the open decision, linking to architecture for behaviour.
- Fix a stale "gather map" phrase that implied a stored index.

Net: less sprawl, one home per topic, cross-linked both ways.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t vectorized

- architecture.md: pipeline Properties gains a line that batch buffers are not
  pooled (DLPack aliasing) and host memory is not pinned yet, with the unified
  pinned-ring-buffer fix; the whole-field-crop scope note flags that the shipped
  random-crop example uses a per-sample Python loop (illustrative, not vectorized).
- DESIGN.md: matching Known-limitations entry for the fresh/unpinned batch buffers
  — one pinned ring buffer (depth ~prefetch_depth) fixes reuse + non_blocking H2D
  together; profile the H2D ceiling first (payload-dependent, ties to M2 GPU stage).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
emfdavid added a commit that referenced this pull request Jul 23, 2026
Fold last night's Ilan Gold / annbatch work (PR #19) into the Pangeo
Showcase draft. Adds the annbatch "sharp mirror" framing (same premise,
opposite bet on where batch randomness comes from) and the honest
block-local shuffle compromise as backing material, then three new
abstract cuts: a maximal draft, a punchy ~200w cut, and a
lead-with-the-thesis cut that opens on the differentiators (O(chunks)
hot path, flat memory, in-place remote) before deferring to xbatcher
and annbatch. Links out to the hosted architecture/examples/benchmarks
pages and caveats xbatcher's GRIB-end sweet spot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant