Skip to content

feat(comm): add copy-engine selected-record exchange with bulk layer packets#3

Draft
FujitsuPolycom wants to merge 1 commit into
codex/sparse-ckv-p2p-transportfrom
codex/selected-record-ce-exchange
Draft

feat(comm): add copy-engine selected-record exchange with bulk layer packets#3
FujitsuPolycom wants to merge 1 commit into
codex/sparse-ckv-p2p-transportfrom
codex/selected-record-ce-exchange

Conversation

@FujitsuPolycom

@FujitsuPolycom FujitsuPolycom commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a generic copy-engine transport for sparse fixed-width records.

  • exchange() moves one compact record plane.
  • exchange_layers() packs three planes with one destination map, performs one
    rank-rotated peer DMA sequence and one arrival barrier, then unpacks into
    caller-owned outputs.
  • Empty, single-plane, and layered calls share a graph-safe deferred-release
    protocol and can alternate on one stream.
  • Pool-scaled offsets use 64-bit arithmetic.

The public API is sparkinfer.comm.pcie.SelectedRecordCopyExchange. This PR is
stacked on the direct selected-record transport so both implementations remain
independently reviewable.

Sparkinfer rebase

The branch is based on local-inference-lab/sparkinfer@ec2cd4d; all runtime,
JIT, packaging, and test paths use the new sparkinfer.comm.pcie namespace.

Validation

  • focused CPU/source suite: 33 passed, 2 GPU-gated skips
  • ai01 exact-image DCP4/432-byte GPU gate: 1 passed
  • the DCP8/656-byte case is retained and skipped on the four-GPU host
  • eager exchange, CUDA graph replay, overflow, mixed replay, and >2 GiB offsets passed
  • packaged-wheel import, Ruff, compileall, and vLLM policy tests passed
  • rebased v19 integration image: 120 passed, 18 skips in CKV/indexer policy tests

Performance context

The dependent vLLM integration groups GLM S1/S2/S3 selected-record prefetches
into one exchange. Against the same-image per-layer CE control:

  • 64K+ comparable decode geometric mean: +8.50%
  • coding median: 99.63 -> 103.79 tok/s
  • reported KV capacity: unchanged at 302,047 tokens

Operational contract

  • every rank calls exchanges in identical order
  • instances are stream-affine
  • eager warmup precedes CUDA graph capture
  • callers fall back to single-plane exchange above layered capacity

AI-assisted implementation under human direction; the diff and validation
results were reviewed before publication.

Design document

GLM-5.2 Sparse CKV under Decode Context Parallelism is the canonical architecture, transport-contract, validation, and upstream-decomposition reference for this PR stack.

Add single-plane and three-plane CUDA-IPC copy-engine exchange with rotated peer scheduling, bounded slab overlays, graph-safe deferred release, and large-offset coverage.

Co-authored-by: OpenAI Codex <codex@openai.com>

Signed-off-by: FujitsuPolycom <87842395+FujitsuPolycom@users.noreply.github.com>
@FujitsuPolycom
FujitsuPolycom force-pushed the codex/selected-record-ce-exchange branch from 22292a4 to 68a4502 Compare July 21, 2026 04:24
@FujitsuPolycom
FujitsuPolycom force-pushed the codex/sparse-ckv-p2p-transport branch from a7ebc4b to f3d5c14 Compare July 21, 2026 04:24
@FujitsuPolycom FujitsuPolycom changed the title distributed: add copy-engine selected-record exchange with three-layer packets feat(comm): add copy-engine selected-record exchange with bulk layer packets Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant