Enable better kernel for GroupedRMSNorm - #1
Conversation
When running cutlass FusedMoE FP8 the scaling factors that are passed are None. This PR passes the correct scaling factors and enables the relevant test. Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
98b3df9 to
ed78454
Compare
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
75c4b08 to
c17029c
Compare
ed78454 to
ef94e83
Compare
…iton-kernel Triton kernel, CUDA graphs, and start of prefix caching fixes
…-quant-for-mxfp8 support dynamic weight quant
2 participants
WIP