Skip to content

[Bugfix][ROCm] Use batch DMA for CPU KV cache loads#49836

Closed
AndreasKaratzas wants to merge 1 commit into
mainfrom
agent/rocm-kv-offload-dma
Closed

[Bugfix][ROCm] Use batch DMA for CPU KV cache loads#49836
AndreasKaratzas wants to merge 1 commit into
mainfrom
agent/rocm-kv-offload-dma

Conversation

@AndreasKaratzas

Copy link
Copy Markdown
Member

Summary

  • Add ROCm to the existing XPU batch-DMA fallback for CPU-to-GPU KV transfers.
  • Keep CUDA, XPU, and every other prior dispatch path unchanged.

The ROCm Triton path directly loaded a raw shared-mmap host pointer. The reproduced fault address matched the mmap base plus the selected block offset in the first case above the Triton descriptor threshold.

Validation

  • Complete KV offload file: 25 passed on MI355.
  • The former failure passed the full 16-case matrix on two GPUs and 100 same-process repeats.
  • Changed-file pre-commit passed.

Buildkite: V1 Core + KV + Metrics

Duplicate check: no open PR addresses ROCm host-mapped KV loads; #41790 only adds separate offload metrics.

AI assistance: Codex assisted with investigation, implementation, and validation; Andreas directed the minimal platform dispatch and reviewed the resulting changes.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@mergify mergify Bot added rocm Related to AMD ROCm v1 bug Something isn't working labels Jul 25, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Jul 25, 2026
@github-project-automation github-project-automation Bot moved this from Todo to Done in AMD Jul 25, 2026
@AndreasKaratzas
AndreasKaratzas deleted the agent/rocm-kv-offload-dma branch July 25, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working rocm Related to AMD ROCm v1

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant