Skip to content

feat: support glm5.2 dsa sharing, cache elision and mooncake pd.#2000

Merged
phantomlei3 merged 3 commits into
xLLM-AI:mainfrom
phantomlei3:feat/glm-5.2-mlu-part2
Jul 22, 2026
Merged

feat: support glm5.2 dsa sharing, cache elision and mooncake pd.#2000
phantomlei3 merged 3 commits into
xLLM-AI:mainfrom
phantomlei3:feat/glm-5.2-mlu-part2

Conversation

@phantomlei3

@phantomlei3 phantomlei3 commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Description

Add GLM-5.2 DSA cross-layer Top-K sharing with per-layer indexer cache elision and accurate cache estimation. Refactor Mooncake KV cache allocation and transfer to support heterogeneous layer layouts and safer RDMA registration

  - GLM-5.2 与 GLM-5.0/5.1 继续共用 glm_moe_dsa 类型;没有共享层时保持原 GLM-5 行为。
  - 检测 Prefill CP 与 DSA 跨层共享的配置冲突。
  - 按层生成 Indexer Cache 启用掩码,共享层不再分配 Indexer Cache 和 Scale。
  - 修正 KV Cache 容量估算,使其按照实际拥有 Indexer Cache 的层数计算显存。
  - Mooncake 支持每层不同数量的缓存 Buffer,并根据实际偏移生成传输 Buffer ID。
  - 补充缓存估算、混合层缓存分配和 Mooncake 变长布局测试。
  - 新增 KVCacheTensorAllocator 抽象,将缓存结构决策留在 KV Cache 工厂中,将物理内存分配策略独立出来。
  - MLU Mooncake 使用专用分配器,仅对 INDEX_SCALE 分配满足 RDMA 注册长度要求的内存。
  - Mooncake 不再自行构造 MLU KV Cache,而是通过 Worker/KV Cache 工厂统一创建。

Change Type

  • Bug fix
  • New feature
  • Performance improvement
  • Refactor
  • Documentation
  • Test
  • Build or CI

Pull Request Checklist

Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.

PR Title and Commit Messages

  • The PR title and each commit message follow the xLLM commit format: <type>: <subject>.

Allowed types: feat, bugfix, docs, test, refactor, chore, style, revert, perf, model, build, release.
The subject should use clear English, start with a verb, include at least 4 words, and end with ..

Pre-commit Checks

  • I have installed pre-commit by running pip install pre-commit or an equivalent command.
  • I have installed the hooks with pre-commit install.
  • I have run pre-commit run --all-files and fixed any reported issues.

If you are unsure how to set up pre-commit, see the pre-commit documentation.

Self Review

  • I have self-reviewed the code according to .agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.
  • I have rebased this PR onto the latest main branch.

Build and Test Coverage

  • Tests have been added or updated as needed.
  • CUDA: python setup.py build test has passed on a CUDA machine.
  • NPU: python setup.py build test has passed on an NPU machine.
  • MLU: python setup.py build test has passed on an MLU machine.

Reviewer Notes

bupt-ycq and others added 3 commits July 22, 2026 10:13
- preserve legacy GLM5 behavior when no layers share top-k state
- align partial indexer cache allocation, estimation, and transfer layouts
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@phantomlei3
phantomlei3 merged commit ca6a5fd into xLLM-AI:main Jul 22, 2026
13 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants