Skip to content

Add optimized DeepSeek DSpark CUDA primitives#7

Merged
dino65-dev merged 1 commit into
mainfrom
agent/add-dspark-cuda
Jul 10, 2026
Merged

Add optimized DeepSeek DSpark CUDA primitives#7
dino65-dev merged 1 commit into
mainfrom
agent/add-dspark-cuda

Conversation

@dino65-dev

Copy link
Copy Markdown
Owner

What changed

  • Added a standalone PyTorch CUDA extension for DeepSeek DSpark.
  • Added a fused low-rank Markov-head kernel with shared latent staging, four coalesced vocabulary stripes per CTA, register FMA accumulation, and no materialized vocabulary bias.
  • Added a hardware-aware prefix scheduler using warp shuffle prefix products, sequential temperature scaling, CUB radix sort/scan, causal first-throughput-drop admission, and device-side result scatter.
  • Added Python modules for one-step correction, semi-autoregressive block sampling, official DeepSpec weight import, and reusable scheduling.
  • Added exact PyTorch/CPU/autograd fallbacks, correctness tests, a CUDA equivalence test, an event-based benchmark, packaging, usage examples, and documentation.

Why

DSpark's default Markov head and confidence scheduler are latency-sensitive operations in speculative decoding. This keeps the repository's CUDA-kernel focus while exposing the primitives through a normal PyTorch API. It consumes logits/confidences from a trained DSpark drafter rather than duplicating DeepSeek's complete training stack.

Algorithm and tensor contracts were checked against the DSpark paper and DeepSeek-AI/DeepSpec.

Validation

  • python -m pytest DSpark/tests -q: 6 passed, 2 CUDA-only tests skipped
  • python -m compileall -q DSpark Usage/DSpark: passed
  • git diff --check: passed
  • C++/PyBind translation unit syntax-checked against PyTorch headers: passed
  • Remote tree SHA matches the locally validated commit tree: 64367c9b7b17ae40cb74595dedf4e4ea46e9b4ff

This worker has no NVIDIA device or nvcc, so the included CUDA equivalence test and GPU benchmark still need to be run on CUDA hardware before marking the PR ready.

@dino65-dev dino65-dev marked this pull request as ready for review July 10, 2026 15:35
@dino65-dev dino65-dev merged commit cca906d into main Jul 10, 2026
@dino65-dev dino65-dev deleted the agent/add-dspark-cuda branch July 10, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant