MODEL-NEMOTRON-H W3: NemotronH is KNOWN -- registered, parsed, enumerated, KV-shaped (#517)

.agents/specs/nemotron-h-model.md

@@ -221,6 +221,114 @@ Until the GGUF arm lands it is **refused by name** at load, naming the missing
 piece, never silently dequantized to a supported path — a silent fallback is
 exactly what a token gate cannot see. Tracked as W7.
 
+## 5c. W3 result — registered, parsed, enumerated, KV-shaped (2026-08-13)
+
+W3 landed on `row/MODEL-NEMOTRON-H-W3B` (base `fafa16f0`). It makes the
+architecture KNOWN; it runs nothing. The forward and GGUF arms REFUSE BY NAME.
+
+**Enumeration, the hard numbers.** `EnumerateNemotronHTensors` claims **18487 of
+18487** released tensors; **0 unaccounted, 0 invented, 0 refused** — every tensor
+has a named consumer, so no refusal was needed. Composition: 5 root (embeddings,
+`norm_f`, the NVFP4 `lm_head` triple), 23 mamba layers x 13, 6 attention x 7,
+23 MoE x 777 (2 router + 128x6 expert + 6 shared + 1 norm), and 270 MTP
+(`mtp.layers.0` 8, `mtp.layers.1` 262). Both directions are gated against a
+committed headers-only projection of the index, and a second case re-verifies
+that projection against the LIVE checkpoint when `VT_NEMOTRON35_SNAPSHOT` names
+it.
+
+**Four things the reconnaissance had wrong or unstated, settled against source
+and disk:**
+
+1. **`layers_block_type` really is the source of truth, but for a subtler
+   reason than "the config says so".** vLLM VENDORS its own `NemotronHConfig`
+   (`transformers_utils/configs/nemotron_h.py:277-287`) in which the polarity is
+   REVERSED — `hybrid_override_pattern` is the ctor argument and
+   `layers_block_type` the derived property. That class is imported by
+   `nemotron_h.py:83` for TYPE ANNOTATION only; the object that reaches the
+   model comes from transformers `AutoConfig`, where `num_hidden_layers` is a
+   property over `layers_block_type` whose setter discards the checkpoint's
+   value (`configuration_nemotron_h.py:225-238`). §5a's live oracle run is what
+   settles it on the real checkpoint. Both spellings are accepted here, the
+   modern one winning.
+2. **`moe_latent_size`: absent and `null` are the SAME state.** Upstream's
+   predicate is `getattr(config, "moe_latent_size", None) is not None`
+   (`nemotron_h.py:143`), so a missing key and an explicit `null` both mean "no
+   latent MoE". A three-state representation would have been inventing a
+   distinction upstream does not make. `std::optional` covers both; a real value
+   REFUSES (§0).
+3. **The shared `detail::ResolveMambaSsmCacheDType` is the WRONG reader here,
+   and using it silently halves the recurrent state.** It reads
+   `HfConfig::mamba_ssm_dtype`, which `hf_config.cpp:439` parses from the key
+   **`mamba_ssm_dtype`** — Qwen3.5/3.6's spelling. NemotronH ships
+   **`mamba_ssm_cache_dtype`** (`configuration_nemotron_h.py:121`), so the
+   shared helper saw an empty string and returned the CONVOLUTION dtype. Caught
+   by the KV gate as `page_size_bytes() == 1085440` against an expected
+   `2134016` — the SSM state at bf16 instead of f32. This is not a Qwen bug;
+   the two families genuinely use different config keys. Resolved locally by
+   `NemotronHSsmCacheDType`, with the reason recorded at the call site.
+4. **The conv-state layout discrepancy is real and deliberate.** Upstream's
+   default is `"SD"` = `(state_len, dim)` (`mamba_utils.py:27-48`,
+   `VLLM_SSM_CONV_STATE_LAYOUT` unset); our local convention across
+   `qwen3_5_common.cpp:85` and `kimi_linear_registry.cpp:156` is
+   `(dim, state_len)`. Same bytes, same page size; the local convention is kept
+   so the shared runner sees one orientation, and the divergence is commented
+   rather than left for W4.
+
+**KV topology.** Two groups carrying their REAL per-layer names — 6
+`backbone.layers.{5,12,19,26,33,42}.mixer` on a `FullAttentionSpec(2 kv heads,
+head_size 128)`, and 23 mamba layers on a `MambaSpec` with shapes
+`{{6144, 3}, {64, 64, 128}}` and dtypes `{bf16, f32}`. `conv_dim == 6144` is
+falsified straight off disk by `mixer.conv1d.weight` BF16 `[6144, 1, 4]`, and
+`in_proj` `[10304, 2688]` confirms `z + xBC + dt`. The names are load-bearing:
+`kv_cache_utils.cpp:979` multiplies a mamba group's page by
+`layer_names.size()`, and `kv_cache_interface.cpp:151-158` does the same for an
+attention group, so a one-element tag would under-count both by 23x and 6x.
+
+**Scope boundary held.** No per-module quant algorithm is resolved — that is W1,
+which is not on `main`. W3 reads four coarse, individually falsifiable keys
+(`quant_method`, `quant_algo`, `kv_cache_scheme`, and the `mtp*` entry in
+`ignore`) and derives the scale companions STRUCTURALLY; the enumeration gate is
+what proves that derivation against all 18487 tensors. A non-ModelOpt producer
+refuses by name. One piece of honest debt is recorded in the code: the quantized
+companion layout of a dense `mlp` block is DERIVED from the shared linear
+layout, because no in-scope released NemotronH checkpoint ships one.
+
+**Fixture.** `tests/vllm/models/fixtures/nemotron_h_35_lightning/` holds the
+released `config.json` minus exactly `quantization_config.{config_groups,
+quantized_layers}` (865 KB of 1.34 MB, the 5981-entry maps W1 owns) and a
+707-family projection of the index. `ignore` is KEPT, unlike the original plan:
+at 2.4 KB it is small, and its `mtp*` wildcard is what makes the MTP tower
+unquantized — eliding it would have forced a guess about 270 tensors.
+
+**Mutation proof (IMP-MUTATE).** Each defect applied alone to the restored tree,
+rebuilt, the gate run, then `git checkout` and `git status --porcelain` verified
+empty. All five turn it RED:
+
+| Mutation | Result |
+|---|---|
+| `layers_block_type` `"moe"` mapped to `kAttention` | 4 cases / 9 assertions FAILURE |
+| `LayerIndices` shifts the FIRST attention index by +1 | 2 cases / 2 assertions FAILURE |
+| `conv_dim` drops the `2*n_groups*state_size` term | 4 cases / 8 assertions FAILURE |
+| SSM cache dtype collapsed to the conv/activation dtype | 1 case / 2 assertions FAILURE |
+| mamba `dt_bias` left UNCLAIMED (23 tensors) | 1 case / 2 assertions FAILURE |
+
+**Gate evidence.** Release `-Werror` CPU: `test_nemotron_h_scaffold` 10/10 cases,
+38245/38245 assertions, `Status: SUCCESS!`; with `VT_NEMOTRON35_SNAPSHOT` set,
+10/10 and 39113/39113. Debug arm (asserts unmasked): identical. Full `ctest`:
+`100% tests passed, 0 tests failed out of 401` (`test_voxtral_e2e` skipped, no
+asset). `test_model_registry` 24/24 and `test_model_loader_gguf` 3/3 after their
+pinned 37-architecture ledgers were reconciled to 38.
+
+One defect was found this way rather than by inspection: the live re-verification
+first died with `[json.exception.type_error.304] cannot use at() with null` from
+inside a loop nowhere near its cause. `nlohmann::json::parse(x).items()` binds a
+range to a TEMPORARY that is destroyed before the body runs. It reads as a clean
+one-liner and it is undefined behaviour; the materialized form is what the
+muse-glimmer precedent already used.
+
+**Not done here:** the forward (W4), the MTP head (W5), the e2e token gate (W6),
+the GGUF arm (W7). The row stays `INVENTORIED`.
+
 ## 6. Risks / decisions
 
 - **Non-gated MoE must not become a parallel path.** If

CMakeLists.txt

@@ -787,6 +787,11 @@ add_library(vllm STATIC
   src/vllm/model_executor/models/kimi_linear_forward.cpp
   src/vllm/model_executor/models/kimi_linear.cpp
   src/vllm/model_executor/models/kimi_linear_device.cpp
+  # MODEL-NEMOTRON-H W3 (#517): the additive NemotronH structural bring-up —
+  # config descent, the on-disk name map, the registry entry and the het-KV
+  # topology. No forward: nemotron_h.cpp is W4.
+  src/vllm/model_executor/models/nemotron_h_registry.cpp
+  src/vllm/model_executor/models/nemotron_h_weights.cpp
   src/vllm/model_executor/models/glm4_registry.cpp
   src/vllm/model_executor/models/glm4_weights.cpp
   src/vllm/model_executor/models/glm4.cpp

docs/FEATURES.md

@@ -25,7 +25,7 @@ are our reading of their documented behavior, not measurements.
 | Embeddable behind a C ABI | ✅ | ☐ | ☐ | ✅ |
 | Weight formats | Safetensors + GGUF | Safetensors | Safetensors | GGUF |
 | Correctness gate | token-exact vs vLLM | reference | own | own |
-| Architectures | 37 registered, 27 gated | 130+ | 100+ | 100+ |
+| Architectures | 38 registered, 27 gated | 130+ | 100+ | 100+ |
 | Downloadable server binaries | ✅ v0.0.2: eight indexed archives with checksums, provenance, manifests, and SBOMs. Windows ZIP downloads do not exist; native CPU/Vulkan lanes await hosted runtime, dry-run, prerelease, and authenticated audit gates | ✅ wheels/containers | ✅ wheels/containers | ✅ host-specific binaries |
 | Native Windows builds | ◐ CPU/Vulkan: `/MT /W4 /WX`, central `NOMINMAX`, UTF-8, aligned allocation, runtime ISA dispatch. Local closure includes the float-domain DeepSeek probe; hosted compile/runtime/release pending | ✅ | ✅ | ✅ |
 
@@ -87,7 +87,7 @@ are our reading of their documented behavior, not measurements.
 The supported set is exactly what the C++ registry registers: every
 architecture self-registers via `REGISTER_VLLM_MODEL`, and
 `scripts/check-supported-models.py` gates this list against the source so it
-cannot drift. Today that is **37 registered architectures**. Each row names the
+cannot drift. Today that is **38 registered architectures**. Each row names the
 checkpoint it was gated against and the verdict; caveats are in
 [STATUS.md](STATUS.md), agent detail in `.agents/model-matrix.md`. A mergeable
 gate/up MLP routes through one shared merged-GEMM method, so a tuned arm added
@@ -136,6 +136,7 @@ speed-pending, which [BENCHMARKS.md](BENCHMARKS.md) tracks.
 | `LagunaForCausalLM` | poolside/Laguna-S-2.1-NVFP4, GGUF-Q4_K, Laguna-XS | byte-exact near-tie (distributional vs vLLM) | vLLM parity+ 1.03x, default on, via the `laguna-gen` CLI; the registered engine forward VT_CHECKs non-bf16 (`ARCH-ONE-SURFACE` fold) |
 | `KimiLinearForCausalLM` | Kimi-Linear-48B-A3B (KDA + NoPE-MLA + MoE) | **Folded onto the shared paged runner (ROW 7 §21, #122): engine==CLI 128/128 byte-identical; vs golden 122/128 (the intrinsic near-tie profile); FA2 paged MLA default-ON; SACRED post-fold green** | Served via `vllm_engine_load` + `vllm_complete_tokens` (ABI v13); server 19.0 tok/s wall vs vLLM ~21 (~0.90×), speed residual open |
 | `KimiK3ForConditionalGeneration` | Kimi-K3 (2.8T MoE) | scaffold: registry+config+enumeration gated, forward refuses | HW-infeasible (~1.56 TB); no run |
+| `NemotronHForCausalLM` | Nemotron-3.5-Lightning-30B-A3B-NVFP4 (`nvidia` @`29f2d174`) | scaffold: registry+config+enumeration+KV-shape gated, forward refuses. 18487/18487 released tensors claimed, 0 unaccounted; het-KV shapes match `mamba2_state_shape`. Nothing runs yet (spec #517, blocked on #496) | no run; GGUF k-quants refuse by name and are owed |
 | `MuseGlimmerForCausalLM` | real tensors, **bf16 depth 4/52 only**: 5 prefill argmax positions match a torch transcription of vllm#51655 and HF. GGUF full depth generates coherently (#347, #359) but is **NOT token-exact** | text forward + loader vs an fp32 reference, per-mechanism property tests, scaffold 11/11, GGUF gate 17/17. An ABSENT config key now takes the architecture's constant (#412): GGUF post-norms ran at 1e-5, not 1e-8 | no vLLM denominator (pin cannot load it); SECONDARY llama.cpp, same GGUF, GB10 CPU: prefill tie **0.997x**, decode 0.232x, RSS 1.92x (#333) |
 | `MuseGlimmerForConditionalGeneration` | vision: **no reference run of any kind**; enumeration gated vs the released 30B index (1436/1436). Image/video need bf16 safetensors: `mmproj-kquant.gguf` is refused by name | perception encoder loaded and wired, so an image or video prompt runs; `perception_emb_norm` now armed by default (#405). Reachability plus placeholder scatter only, no image or video correctness | not measurable; anchored to open vllm#51655 |
 | `LlamaModel` | landed tiny synthetic embedding fixture (engine path == direct pooler path, identical vectors; f64 LAST+normalize reference); real checkpoint (e5-mistral class) is a NAMED residual | pooling/embed only, text paths refuse by task; `vllm_embed` + `/v1/embeddings` | n/a (CPU correctness-grade embeddings) |
@@ -166,11 +167,10 @@ Enumerated in `.agents/model-matrix.md`, not registered, no runnable GB10 gate:
 | `DeepseekV3ForCausalLM`, `DeepseekV32ForCausalLM` | DeepSeek-V3 / V3.2 | 671B, ~642 GiB fp8 vs 119 GiB unified; V3.2 also DSA-indexer dep-blocked |
 | `GlmMoeDsaForCausalLM` | GLM-5 (DSA) | ~1404 GiB bf16; dep-blocked (GLM-5.x is DeepSeek-V3.2 verbatim) |
 | `MiniMaxM2ForCausalLM` | MiniMax-M2 | ~230B, ~428 GiB bf16, ~4x over the unified pool |
-| `NemotronHForCausalLM` | Nemotron-H / Nemotron-3.5-Lightning-30B-A3B | capability-blocked, not HW-blocked (20.1 GiB fits the pool): Mamba2 SSD unported ([#496](https://github.com/mudler/vllm.cpp/issues/496)), MoE and loader owed; spec [#517](https://github.com/mudler/vllm.cpp/issues/517) |
 
-27 of the 31 registered text-generation architectures carry a passing
+27 of the 32 registered text-generation architectures carry a passing
 correctness gate today; the rest are honestly marked scaffold or blocked above.
-(The 37 registered total also covers 3 Parakeet ASR entry points and the
+(The 38 registered total also covers 3 Parakeet ASR entry points and the
 `LlamaModel` embedding arch, which are not text generation.)
 vLLM registers 130+ text architectures, so this is a curated, gated subset, not
 a breadth claim. The first EMBEDDING architecture is registered and live

docs/USAGE.md

@@ -342,6 +342,7 @@ tokens quietly.
 | Architecture | Why it refuses |
 |---|---|
 | `KimiK3ForConditionalGeneration` | Needs ~1.56 TB (MXFP4); no host here can run it |
+| `NemotronHForCausalLM` | The Mamba2 forward is not ported yet (#517 W4, blocked on #496). Safetensors resolve and parse; a GGUF file is refused by name, since no GGUF arm exists for it |
 
 This is a deliberate state, not a bug: registering the architecture is what lets
 the config parse and weight-name mapping be tested before the forward exists.