Vulkan MoE GEMV backend for integrated/AMD GPUs (draft, complements #418) - #729
Draft
minne100 wants to merge 1 commit into
Draft
Vulkan MoE GEMV backend for integrated/AMD GPUs (draft, complements #418)#729minne100 wants to merge 1 commit into
minne100 wants to merge 1 commit into
Conversation
…JustVugg#418) Self-contained Vulkan compute backend for the routed-expert GEMVs used by the qwen36 engine. int4 weights are unpacked in-shader with a plain float GEMV so it runs on drivers whose int8/dot-product compiler path is broken (AMD Radeon 780M); an int8 float path and an optional OpSDotKHR IDP shader (blacklisted on known-broken drivers) are also provided. Targets integrated/AMD GPUs (shared system memory, no CUDA) and is complementary to JustVugg#418 (steve-m, CUDA/NVIDIA-oriented), not a competing second backend. Status: DRAFT. Depends on JustVugg#712 (qwen36 engine) as the caller; not yet wired into dev's build. Integration point is vg_init()/vg_expert_ensure(). Fix: slot byte offsets were uint32_t, so a weights pool >= 4GB would wrap/alias (layers 32-39 -> 0-7 at cache>=256). The backend now auto-disables and falls back to the CPU MoE path instead of silently corrupting output. 64-bit shader offsets (buffer_device_address) are a follow-up.
|
multiple files are autogenerated from scripts that are not part of the pr, is that intentional?
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Vulkan MoE GEMV backend for integrated / AMD GPUs (DRAFT)
Isolates the Vulkan compute path for the qwen36 engine routed-expert GEMVs so it can be reviewed independently of the engine PR (#712).
What
Relation to #418
#418 (steve-m) is the project Vulkan backend, CUDA/NVIDIA-oriented (expert tier + dense + MLA). This one targets integrated/AMD GPUs (shared memory, no CUDA) — complementary, not competing. Happy to coordinate with steve-m / maintainers toward a single backend if preferred.
Status: DRAFT
Fix included here
This supersedes the Vulkan portion of the now-closed #602 (engine taken over by #712).