fix(w4a16): admit ultra-wide FC2 tile in forced re-pin validation#38
Closed
sjug wants to merge 1 commit into
Closed
fix(w4a16): admit ultra-wide FC2 tile in forced re-pin validation#38sjug wants to merge 1 commit into
sjug wants to merge 1 commit into
Conversation
The TC-decode FC2 ultra-wide override selects (tile_k=32, tile_n=512) for
wave balance and admits it via a direct footprint check, deliberately
bypassing the generic tile_k>=64 floor in _candidate_tile_fits. When the
selected tiles are re-pinned across the custom-op boundary
(force_tile_config), the validation there re-checks them with the generic
floor and rejects the exact shape the auto selection produced:
ValueError: force_tile_config fc2 tile (tile_k=32, tile_n=512) does
not fit problem N/K=4096/1024 at moe_block_size=8
The override's wave-balance gates compare FC1/FC2 mn-tile counts against
the device SM count, so the bug is invisible on 188-SM SM120 parts and
fires on 48-SM GB10 (DGX Spark), where DeepSeek-v4-Flash TP2 decode
crashes during CUDA graph capture on every boot.
Teach the forced re-pin path the same exception: accept fc2
(tile_k=32, tile_n=512) under the identical direct footprint check the
auto selection uses. Verified on 2x GB10 (SM121): DS4-Flash TP2 boots and
serves; the ultra tile compiles and runs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThe forced tile validation in ChangesW4A16 tile validation
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Author
|
Superseded by #39 — identical commit, renamed branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The W4A16 TC-decode auto selection can pick the ultra-wide FC2 tile
(tile_k=32, tile_n=512)(the wave-balance override incompile_w4a16_fused_moe), which is admitted via a direct shared-memory footprint check becausetile_k=32sits below the generictile_k>=64floor in_candidate_tile_fits— the override's comment notes this explicitly.However, when the selected tiles cross the
torch.ops.b12x.w4a16_fused_moe_launchcustom-op boundary, the launch side re-pins them viaforce_tile_config, and that validation path re-checks with the generic_candidate_tile_fits— which rejects the exact shape the auto selection just produced:Why upstream never sees it
The ultra override only fires when its wave-balance gates hold, and those compare FC1/FC2 mn-tile counts against the device SM count (
fc1_mn_after <= sms, etc.). On 188-SM SM120 parts (RTX PRO 6000) the gates don't fire with typical model dims. On 48-SM GB10 (DGX Spark) they do: DeepSeek-v4-Flash TP2 decode selects the ultra tile and crashes during CUDA graph capture on every boot.Fix
Teach the forced re-pin path the same exception the selection path makes: accept fc2
(tile_k=32, tile_n=512)under the identical direct footprint check (exact N/K coverage, scale-group alignment, smem fit). No behavior change for any config the generic floor already admits; the tiles still land in the GEMM cache keys as before.Testing
Summary by CodeRabbit