From 50c922ff201ff790c8cb1f87ffbba908d51cad5f Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 12 Aug 2026 22:36:28 +0000 Subject: [PATCH 1/6] spec(nvfp4): the nibble order is a PRODUCER convention, and two of ours disagree (#435) `DequantNvfp4ToBf16` is low-nibble-first, which is right for torchao and ModelOpt and wrong for Lightricks' `nvfp4-prequant` and for the H3 community converter. Read the wrong way round, every adjacent fp4 pair is transposed: finite, correctly shaped, and wrong. The header is shared with the MiniMax-H3, Laguna, DeepSeek-V4 and Qwen3-32B NVFP4 arms, so this is not LTX-2.5 loader work and does not ride inside that row. Spec committed before implementation. Records four things the implementation then has to honour: * the convention per producer, each with the line that packs or unpacks it (torchao `kernels.py:160`, vLLM `nvfp4_emulation_utils.py:321-324`, ltx-kernels `docs/NVFP4.md:27-29`); * that ltx-2-5.md section 4.2's open question is ANSWERED -- torchao is low-first on three independent witnesses, so the shipped text-encoder arm is correct and there is no defect to chase; * why H3's existing nibble-swap-at-load is NOT reused here, and the exact condition under which that decision must be revisited -- an fp4-resident LTX path, which a host-side dequant parameter could not fix; * a correlation gate against the vonkaiser FP8 DiT as an independent oracle, WITH a control arm, because a gate that cannot separate a correct read from a wrong one is section 7.0(c) over again. The parameter defaults to today's behaviour, so existing callers are untouched by construction; the proof obligation is that all ten NVFP4 gates keep their exact case and assertion counts, and those baselines are recorded here. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/nvfp4-nibble-order.md | 209 ++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 .agents/specs/nvfp4-nibble-order.md diff --git a/.agents/specs/nvfp4-nibble-order.md b/.agents/specs/nvfp4-nibble-order.md new file mode 100644 index 000000000..56bbf8136 --- /dev/null +++ b/.agents/specs/nvfp4-nibble-order.md @@ -0,0 +1,209 @@ +# NVFP4 nibble order — two producers, two conventions, one shared dequant + +**Row:** `MODEL-DIFFUSION-LTX25` phase L9a (`.agents/specs/ltx-2-5.md` §4.1, §4.2). +**Issue:** [#435](https://github.com/mudler/vllm.cpp/issues/435). +**Branch:** `row/LTX25-L9A-NVFP4-LINEAR`. +**Authority:** operator, 2026-08-13, after L9a returned `NEEDS_DECISION` rather than +building the arm it was briefed to build. +**Status:** spec committed BEFORE implementation, per AGENTS.md §"Spec before code". + +--- + +## 0. Why this needs its own spec + +`DequantNvfp4ToBf16` (`nvfp4_dequant.h:59`) is shared by the MiniMax-H3, Laguna +(Qwen3.5), DeepSeek-V4 and Qwen3-32B NVFP4 arms and by the `vt` fp4 GEMM reference. +Phase L9a needs it to read a checkpoint packed the other way round. A change there is +not LTX-2.5 loader work, so it is not made inside the LTX row — it is specified, +gated, and reviewed on its own terms. + +## 1. The fact, measured and sourced + +E2M1 packs two 4-bit values per byte. **Which logical element lands in which nibble is +a producer convention, and the two producers we consume disagree.** + +| Producer | Element `2j` goes in | Source | +|---|---|---| +| **torchao** (`NVFP4Tensor`) | the **LOW** nibble | `torchao/prototype/mx_formats/kernels.py:160` — `(uint8_data[::2] \| uint8_data[1::2] << 4)`; its own inverse at `:137-139` stacks `first_elements = data & 0b1111` first | +| **vLLM** (reads torchao/ModelOpt) | the **LOW** nibble | `nvfp4_emulation_utils.py:321-324` — `low = a_flat & 0x0F` then `torch.stack((low, high))`; the Triton path agrees at `:101-108` (`tl.interleave(low_result, high_result)`) | +| **Lightricks `nvfp4-prequant`** | the **HIGH** nibble | `ltx-kernels/docs/NVFP4.md:27-29` — "`hi_first=True` (default) puts element `2j` in the **high** nibble … Pre-quantized checkpoints used with `nvfp4-prequant` are expected in the default order"; `ltx-core/quantization/nvfp4/linear.py:6` | +| **"Star Ultimate Model Converter Pro"** (`lilcheaty/MiniMax-H3-NVFP4`) | the **HIGH** nibble | already recorded at `minimax_h3.h:1500-1515` | + +Our `DequantNvfp4ToBf16` is low-first (`nvfp4_dequant.cpp:74-75`), which is correct for +torchao and ModelOpt and wrong for the other two. + +**Read low-first, a high-first file has every adjacent pair transposed.** It is finite, +correctly shaped, and wrong — the exact failure class this project keeps recording. L9a +measured it on the first-party LTX-2.5 NVFP4 DiT against the independent `vonkaiser` FP8 +oracle: correct reading corr **0.9956** / 9.46% rel rms; wrong nibble order corr **0.032**. +See `.agents/specs/ltx-2-5.md` §4.1 for the full table and the control. + +### 1.1 §4.2 is ANSWERED, and the answer is "no shipped defect" + +`ltx-2-5.md` §4.2 recorded an open question: our torchao text-encoder arm reads low-first, +but its gate compared against goldens made by our own low-first helper — consistency, not +correctness. **torchao's source settles it: low-first.** Three mutually consistent and +independent witnesses (torchao's packer, torchao's unpacker, vLLM's two readers). The +shipped TE arm is correct. This spec's test work converts that self-referential gate into +a source-anchored one so it cannot regress into a question again. + +## 2. Scope + +**In.** One parameter on `DequantNvfp4ToBf16` selecting nibble order, **defaulting to +today's low-first behaviour**; the LTX-2.5 loader's producer discrimination and its use of +that parameter; the shape assertion admitting the cuBLAS-padded framing of the swizzled +scale; a correlation gate that proves the DiT dequantizes to the right numbers. + +**Out.** H3's `MiniMaxH3Nvfp4SwapNibbles` (§3.1 says why it stays); the fp4-resident +device GEMM path (`cuda_matmul_nvfp4.cu`), which stays low-first-only; MXFP4 and GGUF +NVFP4, which have their own packing and are untouched; the TE's tower wiring, owned by +`row/LTX25-L10-TEXT-TOWER`. + +## 3. Design + +```cpp +enum class Nvfp4NibbleOrder { + kLowFirst, // element 2j in the LOW nibble — torchao, ModelOpt, vLLM. DEFAULT. + kHighFirst, // element 2j in the HIGH nibble — Lightricks nvfp4-prequant. +}; + +void DequantNvfp4ToBf16(const uint8_t* packed, const uint8_t* weight_scale_fp8, + float weight_scale_2, int64_t out_dim, int64_t in_dim, + uint16_t* out_bf16, + Nvfp4NibbleOrder order = Nvfp4NibbleOrder::kLowFirst); +``` + +A defaulted trailing parameter makes "every existing caller is untouched" true **by +construction** rather than by inspection. The proof obligation in §5 is that the gates +agree. + +### 3.1 Why NOT H3's nibble-swap-at-load, which is prior art + +`minimax_h3.h:1500-1517` already solved this once, by swapping the two nibbles of every +byte at load so the file becomes standard low-first. That is a genuinely different design +and it was right **for H3**, whose bytes also feed a Marlin fp4-**resident** path: one +transform fixes the bf16 dequant and the device GEMM together. + +LTX-2.5 has no fp4-resident arm — `Ltx2StreamDitToDevice` dequantizes to bf16 and uploads +— so a swap would allocate and rewrite a second copy of 9.4 GB of packed weights to feed a +consumer that could just as well have read them in place. + +Two mechanisms for one concept is a real smell and it is recorded rather than hidden: +**if LTX-2.5 ever grows an fp4-resident NVFP4 device path, this decision must be revisited +and the load-time normalization adopted**, because a host-side dequant parameter cannot fix +a device GEMM that reads the packed bytes itself. That is the tracked condition, written +where it will be found. + +### 3.2 The producer discriminator (operator-RATIFIED, 2026-08-13) + +The scale layout and the nibble order are decided together, from the **`torchao_nvfp4` +marker's presence or absence**. This is in-file evidence, not provenance guessing: torchao +always writes that marker, so its absence excludes torchao. + +| Marker | Required `weight_scale` shape | Resolves to | +|---|---|---| +| present, `is_swizzled_scales=true` | `[32*ceil(N/128), 16*ceil(G/4)]` (`to_blocked` framing) | torchao: unswizzle, **low**-first | +| absent | `[round_up(N,128), round_up(G,4)]` (cuBLAS-padded framing) | `nvfp4-prequant`: unswizzle, **high**-first | +| anything else | — | **REFUSE BY NAME** | + +`G = in_features / 16`. No "probably" branch: a marker with the wrong shape, an absent +marker with the wrong shape, or a marker declaring an unimplemented combination all refuse. + +**Both framings describe the SAME bytes.** `[32*ceil(N/128), 16*ceil(G/4)]` and +`[round_up(N,128), round_up(G,4)]` have equal element counts and the identical layout; +they are two 2-D dresses on one flat buffer. `Ltx2UnswizzleNvfp4BlockScale` is already +framing-agnostic — it keys off logical `rows`/`cols` and a byte count, and its index +formula matches `ltx-kernels/csrc/nvfp4/quantize.cu:26-31` term for term — so **only the +shape assertion changes**, not the permutation. + +**The assumption must be legible at the code site.** The marker-absent arm is an inference +from a producer's signature, not a fact read off the file, and the comment there says so +and cites the evidence: the 0.9956 correlation and `ltx-kernels/docs/NVFP4.md:27-29`. + +### 3.3 Why the shape cannot discriminate alone + +For every quantized layer in the first-party DiT, `N % 128 == 0` and `G % 4 == 0`, so the +cuBLAS-padded shape **numerically equals the linear `[N, G]` shape**. A shape test can +therefore never separate "swizzled, padded framing" from "linear". That is why the marker +is primary and the shape only corroborates, and it is why the original linear-arm brief was +unimplementable as written. + +## 4. Risks + +| Risk | Mitigation | +|---|---| +| A defaulted parameter silently changes an existing caller | Every NVFP4 gate re-run with **assertion counts compared**, not just Status (§5) | +| The high-first arm is wrong and output is merely finite | The correlation gate (§5.2) requires the 0.9956 / 9.46% signature; finiteness is not accepted as evidence | +| Marker-absent inference generalizes to a file it should not | Refuse by name on any other marker/shape combination; no fallback branch | +| Two nibble mechanisms drift apart | §3.1 records the condition under which H3's is adopted here | +| The correlation gate needs a 21 GB and an 18.7 GB file | It reads byte RANGES from the two headers, materializes ~128 rows, and is opt-in behind an env var like the other shipped-checkpoint cases | + +## 5. Tests and gates + +### 5.1 RED first +A case that dequantizes the real first-party DiT bytes and asserts the FP8-correlation +signature must FAIL before the change, for the right reason (the loader refuses the shape), +and pass after. + +### 5.2 The correlation gate — the one nobody had +The `vonkaiser` FP8 DiT and the first-party NVFP4 DiT quantize the same base weights, so +the FP8 file is an **independent oracle** for the NVFP4 read. The gate: + +- dequantizes the same module from both files (rows 0..127), +- asserts Pearson correlation **>= 0.99** and relative rms error **<= 0.15**, +- asserts a **control**: the same NVFP4 read against a DIFFERENT module's FP8 weights + correlates **< 0.2**, so the gate proves it can tell right from wrong rather than + passing on any two finite arrays, +- and emits the VALUES, never a boolean. + +Without the control the gate is `7.0(c)` again — a fixture that cannot separate a correct +implementation from a plausible wrong one. + +### 5.3 Nibble order, source-anchored +A unit case pinning both orders against hand-computed bytes, plus a generator-side anchor +on torchao's `pack_uint4` line and `ltx-kernels`' `hi_first` documentation, so a convention +change upstream is reported as a source change (§1.1's repair). + +### 5.4 The discriminator +Every row of §3.2's table, including each refusal, asserted by name. + +### 5.5 Unchanged-behaviour proof (operator's non-negotiable) +These must be green with **identical case and assertion counts**, reported. Baselines +captured on this branch at `f400413e`, Release, CUDA=OFF: + +| Gate | Baseline | +|---|---| +| `test_nvfp4_dequant` | 4 / 47 | +| `test_gguf_nvfp4` | 14 / 2352 | +| `test_qwen36_weights` | 7 / 45 | +| `test_qwen3_forward` | 7 / 1557 | +| `test_minimax_h3` | 79 / 57395 | +| `test_ops_nvfp4_matmul` | 4 / 1 | +| `test_ops_moe_grouped` | 6 / 3 | +| `test_ops_nvfp4_fp4` | 22 / 919 | +| `test_ltx2_loader` | 20 / 2363 | +| `test_ltx2_device` | 13 / 498 | + +Counts only RISE, and only in the two suites this change adds cases to. **If any other +gate moves, STOP.** `Status:` is grepped every run: a thrown case DROPS the assertion +count while still printing "passed". + +### 5.6 The real artifact +The first-party NVFP4 DiT stages onto the GPU and forwards, on dgx.casa under +`flock -w 2700 $HOME/gpu.lock`, naming the file, its tensor count and its derived geometry, +with finite non-degenerate output. + +## 6. Stop conditions + +- The high-first read does not reproduce the correlation signature → STOP; the encoding is + not merely a nibble order and the premise is wrong again. +- Any existing NVFP4 gate changes case or assertion count → STOP. +- A checkpoint appears that needs a third convention, or a marker/shape combination outside + §3.2 → refuse by name and report; do not add a branch. +- Any temptation to accept "the forward ran and the output was finite" as the correctness + result → forbidden; §5.2 is the result. + +## 7. Outcome + +To be written when this lands: what was measured, what was rejected, and why the default is +low-first. From 9f807feb29f7d8b5b8cb0425afbedec78591dc17 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 12 Aug 2026 23:26:51 +0000 Subject: [PATCH 2/6] feat(ltx-2.5): the first-party NVFP4 DiT loads -- swizzled, HIGH-nibble-first (#435) The file was refused, and the refusal carried a WRONG diagnosis: it said the group scale was stored LINEAR [N, K/16]. It is not. The bytes are SWIZZLED -- the permutation Ltx2UnswizzleNvfp4BlockScale already inverts -- merely declared in the cuBLAS-padded framing [round_up(N,128), round_up(G,4)] instead of the to_blocked [32*ceil(N/128), 16*ceil(G/4)] the loader knew. For every layer in that file N % 128 == 0 and G % 4 == 0, so the padded framing is NUMERICALLY IDENTICAL to the linear shape. That coincidence is why the wrong diagnosis looked confirmed, and why no shape test could ever have settled it. The file also packs element 2j in the HIGH nibble, which no shape encodes at all. Both facts came from correlating the dequantized weights against the vonkaiser FP8 DiT of the same base weights -- an oracle that is not ours -- and were then confirmed in Lightricks' own runtime. ltx-kernels/docs/NVFP4.md:27-29 hi_first=True is the default ltx-kernels/csrc/nvfp4/quantize.cu:26-31 swizzled_offset, padded framing ltx-core/quantization/nvfp4/linear.py:6-7 both statements together torchao kernels.py:160 low-first, the other convention vllm nvfp4_emulation_utils.py:321-324 vLLM reads low-first, agreeing What lands: * Nvfp4NibbleOrder on DequantNvfp4ToBf16, DEFAULTING to low-first, so the MiniMax-H3, Laguna, DeepSeek-V4 and Qwen3-32B callers are untouched by construction rather than by inspection; * Ltx2ResolveNvfp4Producer -- the torchao marker decides, the framing corroborates, and every other combination REFUSES BY NAME. torchao always writes that marker, so its absence excludes torchao; that inference is stated at the code site with the evidence behind it, because it is the one step here not read directly off the file; * a CORRELATION GATE against the FP8 checkpoint, with a CONTROL arm and a deliberately-wrong-order arm, both asserted to collapse. corr 0.995, rms 10.1%, control 0.004, wrong-order 0.005. It also asserts absmax is IDENTICAL under both nibble orders -- which is why "the output was finite and correctly scaled" was never evidence of anything here. The synthetic NVFP4 fixture declared the to_blocked framing with NO marker: a combination neither producer emits. The new discriminator refused it, correctly, and the fixture was what was wrong -- it now mirrors the shipped artifact. Ten NVFP4 gates re-run. Eight byte-identical to baseline; the two that rose are the two this change adds cases to (test_nvfp4_dequant 4/47 -> 5/69, test_ltx2_loader 20/2363 -> 23/2426). All nine LTX/video baselines unchanged. Also settles ltx-2-5.md section 4.2: torchao is low-first on three independent witnesses, so the shipped text-encoder arm is CORRECT and there is no defect to chase. Its gate is now source-anchored instead of comparing our helper to itself. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/porting-inventory.md | 60 ++-- docs/FEATURES.md | 2 +- docs/USAGE.md | 18 +- .../model_loader/nvfp4_dequant.h | 40 ++- .../vllm/model_executor/models/ltx2_loader.h | 114 ++++++- scripts/gen-ltx2-quant-goldens.py | 220 ++++++++++++- .../model_loader/nvfp4_dequant.cpp | 17 +- .../model_executor/models/ltx2_loader.cpp | 182 ++++++++--- tests/vllm/models/ltx2_quant_goldens.inc | 53 +++- tests/vllm/models/test_ltx2_device.cpp | 25 +- tests/vllm/models/test_ltx2_loader.cpp | 292 +++++++++++++++++- tests/vllm/test_nvfp4_dequant.cpp | 56 ++++ 12 files changed, 959 insertions(+), 120 deletions(-) diff --git a/.agents/porting-inventory.md b/.agents/porting-inventory.md index db65f3f9e..40ddb42f6 100644 --- a/.agents/porting-inventory.md +++ b/.agents/porting-inventory.md @@ -1253,17 +1253,35 @@ Examples: `examples/cli` ✅ (C-API client), `examples/server` ✅ (OpenAI serve tower module without materializing it. * **Spec:** [ltx-2.5 spec](specs/ltx-2-5.md) §1.4 and §6 (L6). Lifecycle: shipped (host + load-time device staging). Owner: the LTX-2.5 row. - * **OWED, FOUND 2026-08-12 by the phase-L8 GB10 run (entry 20 below).** The - NVFP4 dequant here assumes torchao's SWIZZLED block scales, and the - FIRST-PARTY `ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` is not - that: it carries **no `.torchao_nvfp4` marker at all** and stores - `weight_scale` as `[4096, 256]`, the LINEAR `[N, K/16]` layout, against the - SWIZZLED `[1024, 1024]` this path expects. The refusal is correct and fires - by name — the two shapes have the same element count, so reading one as the - other type-checks and permutes every scale within a 128x4 tile — but it - means the first-party NVFP4 DiT cannot be loaded at all today. The LINEAR - read is the missing piece. Phase L7's shipped-checkpoint test only parsed - the MANIFEST, so nothing had materialized a tensor from that file before. + * **CLOSED 2026-08-13 by phase L9a — and the diagnosis it closes was WRONG.** + This entry previously recorded that the FIRST-PARTY + `ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` stores `weight_scale` + in the LINEAR `[N, K/16]` layout, and that a linear read was the missing + piece. **It does not, and it was not.** The bytes are SWIZZLED — the same + permutation this entry already inverts — merely declared in the + cuBLAS-padded framing `[round_up(N,128), round_up(G,4)]` = `[4096, 256]` + instead of torchao's `to_blocked` `[32*ceil(N/128), 16*ceil(G/4)]` = + `[1024, 1024]`. For every layer in that file `N % 128 == 0` and + `G % 4 == 0`, so the padded framing is NUMERICALLY IDENTICAL to the linear + shape; that coincidence is what made the wrong diagnosis look right, and it + is why no shape test could have settled it. + + The file additionally packs element `2j` in the HIGH nibble, which no shape + encodes at all. Both facts were established by correlating the dequantized + weights against the `vonkaiser` FP8 DiT of the same base weights (0.9956 + with 9.46% relative rms, against 0.0004-0.26 for every other reading), and + independently confirmed in Lightricks' own runtime + (`ltx-kernels/docs/NVFP4.md:27-29`, `csrc/nvfp4/quantize.cu:26-31`, + `ltx-core/quantization/nvfp4/linear.py:6-7`). + * **Resolution.** `Ltx2ResolveNvfp4Producer` discriminates on the + `torchao_nvfp4` marker (torchao always writes one, so its absence excludes + torchao), corroborates with the framing, and REFUSES by name on any other + combination. `Ltx2UnswizzleNvfp4BlockScale` was already framing-agnostic and + is UNCHANGED. The nibble order became an `Nvfp4NibbleOrder` parameter on + `DequantNvfp4ToBf16`, defaulting to low-first so every pre-existing caller + is untouched by construction — see + [nvfp4-nibble-order spec](specs/nvfp4-nibble-order.md), which also records + why H3's nibble-swap-at-load was NOT reused and when it must be. 19. **LTX-2.5 phase L7 — the family behind `vllm::multimodal::VideoEngine`, and the driving loop.** @@ -1373,16 +1391,16 @@ Examples: `examples/cli` ✅ (C-API client), `examples/server` ✅ (OpenAI serve produced finite, non-degenerate output. The wall-clock is SIZING ONLY and is not a speed result — spec §0, no production-configuration denominator exists. The FIRST-PARTY `ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` - (18.72 GB, 7876 tensors) does NOT stage, and the refusal is L6's and is - correct: the file carries **no `.torchao_nvfp4` marker at all**, and its - `weight_scale` is `[4096, 256]` — the LINEAR `[N, K/16]` layout — where - `Ltx2DequantTorchaoNvfp4ToBf16` expects the SWIZZLED `[1024, 1024]` form. - The two have the same element count, so reading one as the other - type-checks and permutes every scale within a 128x4 tile. The LINEAR-scale - read is owed against the **L6 loader** surface (entry 18 above), not this - one; the device forward never sees it. Phase L7's shipped-checkpoint test - only parsed the MANIFEST, which is why this surfaced now: nothing had - materialized a tensor from that file before. + (18.72 GB, 7876 tensors) did NOT stage at that time. **The reason recorded + here was WRONG and is superseded by entry 18 above (phase L9a, + 2026-08-13):** the file does not store a LINEAR `[N, K/16]` scale. Its bytes + are SWIZZLED, declared in the cuBLAS-padded framing — which for these + geometries is numerically identical to the linear shape, which is precisely + why the wrong reading looked confirmed — and it packs the OPPOSITE nibble + order. The refusal was correct; its diagnosis was not. Loading it needed a + producer discriminator and a nibble-order arm, not a linear read. Phase L7's + shipped-checkpoint test only parsed the MANIFEST, which is why this surfaced + as late as it did: nothing had materialized a tensor from that file before. * **CORRECTED 2026-08-12 by an adversarial review of L7+L8 (row `LTX25-L8-FIX`). The headline held; five records did not.** 1. **The FP8 DiT this row RAN carries no `__metadata__` at all.** The engine diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 43682ee23..cfefbeb33 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -153,7 +153,7 @@ they sit outside the gated list above. | Voxtral audio (`VoxtralForConditionalGeneration`) | Voxtral-Mini-3B-2507 | near-tie-robust 16/16 vs vLLM 0.25.0 | decode 0.97x (beats vLLM); encoder FORWARD 15.90x of vLLM's whole TTFT (pin 46.02 ms), or 2.89x with opt-in `VT_WHISPER_ENC_FA2=1` (costs 3 near-tie divergences vs 0). Not a TTFT ratio. Pending | | Whisper audio encoder | openai/whisper-small; whisper-large-v3 (Voxtral cfg) | encoder tower 77/77; large-v3 tower 203/203 | pending | | MiniMax-H3 DiT (`MiniMaxH3DiTModel`, vllm-omni lane) | MiniMax-H3 (33.1B video+audio) | portable 79/79; all three modalities COHERENT on Q4_K_M (§8.20); PRUNED ckpts run, Q8_0 seam 0.9941 (§8.21); ref2va grid was NVFP4 quant error, §8.9 REFUTED; GGUF/NVFP4/bf16 shards stream | FP4/Marlin landed; speed pending; no bf16 render yet. Render from the Q4_K_M GGUF, not the NVFP4 arm. Krea 2 text-to-image (roadmap C11) is scoped to reuse these DiT seams | -| LTX-2.5 DiT (`LTX2VideoTransformer3DModel`, Lightricks lane) | LTX-2.5 (21.00B video+audio) | `SPIKE`. DiT, both VAE decoders, both VAE ENCODERS + mel front-end, the conditioning items, the pipeline and the quantized loaders gated vs upstream at reduced dims: max\|diff\| 1.2e-07 DiT, 1.8e-06 VAE decode, 6.0e-07 VAE encode, 2.8e-06 pipeline. STRUCTURAL e2e only | Fits ONE GB10 at ~29 GB NVFP4. FP8 + torchao-NVFP4 arms load. CPU f32, CUDA bf16 device-resident. Text tower, DiffVAE, LoRA, image conditioning all still refused AT THE ENGINE. FP4-resident arm OWED. Speed PENDING | +| LTX-2.5 DiT (`LTX2VideoTransformer3DModel`, Lightricks lane) | LTX-2.5 (21.00B video+audio) | `SPIKE`. DiT, both VAE decoders, both VAE ENCODERS + mel front-end, the conditioning items, the pipeline and the quantized loaders gated vs upstream at reduced dims: max\|diff\| 1.2e-07 DiT, 1.8e-06 VAE decode, 6.0e-07 VAE encode, 2.8e-06 pipeline. STRUCTURAL e2e only | Fits ONE GB10 at ~29 GB NVFP4. FP8 + both NVFP4 producers load. CPU f32, CUDA bf16 device-resident. Text tower, DiffVAE, LoRA, image conditioning all still refused AT THE ENGINE. FP4-resident arm OWED. Speed PENDING | | MTP speculator | Qwen3.6-27B, Qwen3.6-35B-A3B | token-identical to vLLM `mtp` at c1 | ~4% faster c1; +16% output tput (MoE) | | DFlash block-diffusion | Qwen3 (DFlash draft) | near-tie e2e 27/27 vs vLLM | 2.9x over spec-off, 1.003x vs vLLM DFlash-on | | DeepSeek-V4 MTP | DeepSeek-V4-Flash (nextn head) | lossless 5/5; real-model weight-blocked | pending | diff --git a/docs/USAGE.md b/docs/USAGE.md index 91566a501..94c588e71 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -1672,10 +1672,19 @@ catch it. ## LTX-2.5 quantized loaders (no render path yet) `include/vllm/model_executor/models/ltx2_loader.h` materializes the shipped -LTX-2.5 checkpoints: the FP8 DiT, an NVFP4 DiT, and the torchao-NVFP4 Gemma-4 +LTX-2.5 checkpoints: the FP8 DiT, both NVFP4 DiTs, and the torchao-NVFP4 Gemma-4 text encoder with its embedded tokenizer. There is still no render path, so these are library entry points and not a command. +The two NVFP4 checkpoints were written by different producers that disagree about +both the group-scale framing and which nibble holds which weight, so the loader +resolves the producer from the `torchao_nvfp4` marker: present means torchao +(`to_blocked` framing, low-nibble-first), absent means the Lightricks +`nvfp4-prequant` tool (cuBLAS-padded framing, high-nibble-first). Any other +combination is refused by name rather than guessed, because both readings +type-check and produce finite, correctly scaled, wrong weights. See +`.agents/specs/nvfp4-nibble-order.md`. + Two behaviours a caller has to know. `Ltx2LoadDitFromSafetensors` REFUSES the shipped DiT by default, because that file carries five module families phase L2 does not port (`prompt_adaln_single`, `audio_prompt_adaln_single`, @@ -1689,11 +1698,12 @@ and exists only for the f32 parity forward. a time so peak residency is the device copy plus one tensor, and it stages at load because host-resident weights measure 20 to 30 percent slower there. -The gate needs the two checkpoint headers and a vLLM checkout; it reads a few -hundred bytes at their own offsets and never a payload: +The gate needs the three checkpoint headers, a vLLM checkout and an LTX-2 +checkout (the two nibble-order authorities); it reads a few hundred bytes at +their own offsets and never a payload: ```sh -python3 scripts/gen-ltx2-quant-goldens.py --vllm ~/_git/vllm --checkpoint-root /mnt/nas_share/checkpoints --out tests/vllm/models/ltx2_quant_goldens.inc +python3 scripts/gen-ltx2-quant-goldens.py --vllm ~/_git/vllm --ltx2 ~/_git/LTX-2 --checkpoint-root /mnt/nas_share/checkpoints --out tests/vllm/models/ltx2_quant_goldens.inc cmake --build build --target test_ltx2_loader && ./build/tests/test_ltx2_loader ``` diff --git a/include/vllm/model_executor/model_loader/nvfp4_dequant.h b/include/vllm/model_executor/model_loader/nvfp4_dequant.h index 2c73b3bc2..e1f7ad3e5 100644 --- a/include/vllm/model_executor/model_loader/nvfp4_dequant.h +++ b/include/vllm/model_executor/model_loader/nvfp4_dequant.h @@ -37,6 +37,37 @@ inline constexpr int kNvfp4GroupSize = 16; inline constexpr float kE2M1Lut[8] = {0.0F, 0.5F, 1.0F, 1.5F, 2.0F, 3.0F, 4.0F, 6.0F}; +// WHICH LOGICAL ELEMENT SITS IN WHICH NIBBLE — a PRODUCER convention, and the two +// producers this project reads DISAGREE. See .agents/specs/nvfp4-nibble-order.md. +// +// Read the wrong way round, every adjacent fp4 pair is transposed. The result is +// finite, correctly shaped and correctly scaled, so nothing downstream notices: +// a decoder emits fluent tokens, a DiT renders a plausible-but-wrong frame. It is +// therefore never inferred and never defaulted per call site — the loader that +// knows the producer passes it. +enum class Nvfp4NibbleOrder { + // Element 2j in the LOW nibble, 2j+1 in the HIGH. torchao's `pack_uint4` + // (torchao/prototype/mx_formats/kernels.py:160, + // `uint8_data[::2] | uint8_data[1::2] << 4`), which is what NVIDIA ModelOpt and + // compressed-tensors checkpoints carry, and what vLLM's own reader assumes + // (`break_fp4_bytes`, nvfp4_emulation_utils.py:321-324: `low = a_flat & 0x0F` + // then `torch.stack((low, high))`). THE DEFAULT — every caller predating + // .agents/specs/nvfp4-nibble-order.md means this one. + kLowFirst, + // Element 2j in the HIGH nibble. Lightricks' `nvfp4-prequant`, which wrote the + // first-party LTX-2.5 NVFP4 DiT (ltx-kernels/docs/NVFP4.md:27-29: "`hi_first=True` + // (default) puts element `2j` in the **high** nibble of byte `j`"; the same + // statement at ltx-core/quantization/nvfp4/linear.py:6). + // + // MiniMax-H3's community NVFP4 checkpoints are also high-first and are handled a + // DIFFERENT way — `MiniMaxH3Nvfp4SwapNibbles` normalizes the bytes at load + // (minimax_h3.h:1500-1517) because H3 also feeds a Marlin fp4-RESIDENT path, and + // one byte transform fixes both arms where a host-dequant flag fixes only one. + // Both mechanisms are deliberate; nvfp4-nibble-order.md section 3.1 records which + // to use when, and the condition under which LTX-2.5 must switch to H3's. + kHighFirst, +}; + // Decode one IEEE fp8-e4m3fn byte (1 sign, 4 exp, 3 mantissa; bias 7; no inf; // NaN = 0x7F/0xFF; 0x00 = +0) to f32. Matches // torch.Tensor.view(torch.float8_e4m3fn).to(torch.float32). @@ -45,10 +76,14 @@ float F8E4M3ToF32(uint8_t byte); // Dequantize a modelopt W4A16_NVFP4 weight matrix to bf16 (row-major bit // patterns in out_bf16). // -// packed [out_dim, in_dim/2] U8, low-nibble-first packing +// packed [out_dim, in_dim/2] U8, packed per `order` // weight_scale_fp8 [out_dim, in_dim/16] fp8-e4m3fn bytes, linear layout // weight_scale_2 per-tensor f32 global scale (multiplied, not reciprocated) // out_bf16 [out_dim, in_dim] bf16 bit patterns (caller-owned) +// order which nibble holds element 2j; DEFAULTS to the torchao / +// ModelOpt convention, so every caller written before +// .agents/specs/nvfp4-nibble-order.md is unchanged BY +// CONSTRUCTION rather than by inspection // // Requires in_dim % 16 == 0. Aborts (VT_CHECK) otherwise. // @@ -58,7 +93,8 @@ float F8E4M3ToF32(uint8_t byte); // (the safetensors reader validates every span). void DequantNvfp4ToBf16(const uint8_t* packed, const uint8_t* weight_scale_fp8, float weight_scale_2, int64_t out_dim, int64_t in_dim, - uint16_t* out_bf16); + uint16_t* out_bf16, + Nvfp4NibbleOrder order = Nvfp4NibbleOrder::kLowFirst); // Dequantize a modelopt per-tensor FP8 (W8A16) weight to bf16. The 35B gate // checkpoint stores its attention/GDN projections this way (hf_quant_config diff --git a/include/vllm/model_executor/models/ltx2_loader.h b/include/vllm/model_executor/models/ltx2_loader.h index 5ec3064d9..9c4bc09af 100644 --- a/include/vllm/model_executor/models/ltx2_loader.h +++ b/include/vllm/model_executor/models/ltx2_loader.h @@ -38,6 +38,25 @@ // the same permutation written twice — and hands the result to the UNCHANGED // `DequantNvfp4ToBf16`. Nothing else about the scheme differs. // +// ─── AND A SECOND PRODUCER, WITH A SECOND DELTA ────────────────────────────── +// +// Recorded 2026-08-13 (phase L9a, .agents/specs/nvfp4-nibble-order.md). The above +// describes the torchao-quantized TEXT ENCODER. Lightricks' first-party NVFP4 +// **DiT** was written by their own `nvfp4-prequant`, and it differs twice: +// +// * the SAME swizzled bytes are declared in the cuBLAS-PADDED framing +// [round_up(N,128), round_up(G,4)] rather than torchao's `to_blocked` +// [32*ceil(N/128), 16*ceil(G/4)]. Same buffer, same permutation, different +// 2-D dress — `Ltx2UnswizzleNvfp4BlockScale` reads both unchanged; +// * element 2j is in the HIGH nibble, not the low one. +// +// The second is a different byte ENCODING and is why `DequantNvfp4ToBf16` grew an +// `Nvfp4NibbleOrder`. This file used to be REFUSED, with a message diagnosing its +// scale as the LINEAR layout; that diagnosis was wrong — the bytes were swizzled +// all along, and the linear shape is numerically identical to the padded framing +// for every layer here, which is exactly why a shape test could not tell. +// `Ltx2ResolveNvfp4Producer` below is what decides, and what refuses. +// // ─── WHAT THE STORED SHAPES MEAN, AND THE TRAP IN THEM ─────────────────────── // // NVFP4 packs TWO values per byte along the LAST dimension, so every U8 width @@ -112,6 +131,7 @@ #include +#include "vllm/model_executor/model_loader/nvfp4_dequant.h" // Nvfp4NibbleOrder #include "vllm/model_executor/models/ltx2.h" #include "vllm/model_executor/models/ltx2_audio_vae.h" #include "vllm/model_executor/models/ltx2_text_encoder.h" @@ -148,6 +168,66 @@ struct Ltx2TorchaoNvfp4Marker { // The tensor name this port is keyed on. inline constexpr const char* kLtx2TorchaoNvfp4MarkerSuffix = ".torchao_nvfp4"; +// --------------------------------------------------------------------------- +// WHICH PRODUCER WROTE THIS FILE — .agents/specs/nvfp4-nibble-order.md section 3.2 +// --------------------------------------------------------------------------- +// +// The two NVFP4 checkpoints this campaign loads were written by DIFFERENT +// producers that disagree about BOTH the group scale's 2-D framing AND the nibble +// order. Getting either wrong yields finite, correctly shaped, correctly scaled, +// WRONG weights, so the choice is made from evidence and refused when the +// evidence does not fit. +// +// torchao `.torchao_nvfp4` marker present, scale stored in the +// `to_blocked` framing [32*ceil(N/128), 16*ceil(G/4)], +// LOW-nibble-first. (the Gemma-4 text encoder) +// nvfp4-prequant NO marker anywhere, scale stored in the cuBLAS-PADDED framing +// [round_up(N,128), round_up(G,4)], HIGH-nibble-first. +// (Lightricks' first-party LTX-2.5 DiT) +// +// THE MARKER IS THE DISCRIMINANT, AND ITS ABSENCE IS EVIDENCE, NOT A DEFAULT: +// torchao always emits that sidecar, so a file without one was not written by +// torchao. That is an inference from a producer's signature and it is stated here +// rather than buried, because it is the one step in this path that is not read +// directly off the file. What supports it, measured rather than argued: reading +// the first-party DiT this way correlates 0.9956 (9.46% relative rms, which IS +// NVFP4 quantization error) against the independent vonkaiser FP8 DiT of the same +// base weights, where the alternative readings give 0.0004 to 0.26; and Lightricks' +// own runtime documents exactly this layout and order +// (ltx-kernels/docs/NVFP4.md:27-29, ltx-core/quantization/nvfp4/linear.py:6-7). +// +// THE SHAPE CANNOT DISCRIMINATE ALONE, which is why the marker leads. Every layer +// of the first-party DiT has N % 128 == 0 and G % 4 == 0, so its cuBLAS-padded +// shape is NUMERICALLY IDENTICAL to the linear [N, G] one. A shape test can never +// separate those two. It is used only to CORROBORATE what the marker decided, and +// any other combination is refused by name rather than resolved in either +// direction. +enum class Ltx2Nvfp4Producer { + kTorchao, // marker present: to_blocked framing, low-nibble-first + kNvfp4Prequant, // marker absent: cuBLAS-padded framing, high-nibble-first +}; + +// The two 2-D framings of the SAME swizzled byte buffer. Equal element counts, +// identical layout; `Ltx2UnswizzleNvfp4BlockScale` reads either because it keys +// off the LOGICAL rows/cols and a byte count, not off the declared shape. +std::vector Ltx2Nvfp4ToBlockedScaleShape(int64_t out_features, + int64_t in_features); +std::vector Ltx2Nvfp4PaddedScaleShape(int64_t out_features, + int64_t in_features); + +// Resolve the producer from the marker, corroborated by the stored scale shape. +// `marker` is null when the file declares none for this module. +// +// Throws std::runtime_error naming `module` when the marker and the shape +// disagree, or when the shape matches neither framing — never picking one. +Ltx2Nvfp4Producer Ltx2ResolveNvfp4Producer(const std::string& module, + const Ltx2TorchaoNvfp4Marker* marker, + const std::vector& scale_shape, + int64_t out_features, int64_t in_features); + +// The nibble order each producer packs with. +Nvfp4NibbleOrder Ltx2Nvfp4NibbleOrderFor(Ltx2Nvfp4Producer producer); + // Parses the marker's JSON payload. Throws std::runtime_error naming `module` // on: a non-JSON payload, a `format` other than "torchao_nvfp4", a // `block_size` other than kNvfp4GroupSize (16), or `is_swizzled_scales` false — @@ -174,20 +254,26 @@ Ltx2TorchaoNvfp4Marker ParseLtx2TorchaoNvfp4Marker(const std::string& module, void Ltx2UnswizzleNvfp4BlockScale(const uint8_t* swizzled, size_t swizzled_bytes, int64_t rows, int64_t cols, uint8_t* linear); -// One torchao-NVFP4 quantized module, dequantized to bf16 through the UNCHANGED -// modelopt path: unswizzle the group scale, then `DequantNvfp4ToBf16`. -// -// packed U8 [out, in/2] (in = stored cols * 2) -// scale F8_E4M3 [out/4, (in/16)*4] the SWIZZLED layout -// scale_2 F32 scalar MULTIPLIER -// -// Every one of those three shapes is asserted against `out`/`in` before a byte -// is read, and a mismatch throws naming `module`. `out_bf16` receives out*in -// bf16 bit patterns. -void Ltx2DequantTorchaoNvfp4ToBf16(const std::string& module, const StTensor& packed, - const StTensor& scale, const StTensor& scale_2, - int64_t out_features, int64_t in_features, - uint16_t* out_bf16); +// One NVFP4 quantized module, dequantized to bf16 through the UNCHANGED modelopt +// path: unswizzle the group scale, then `DequantNvfp4ToBf16`. +// +// packed U8 [out, in/2] (in = stored cols * 2), packed per `producer` +// scale F8_E4M3 the SWIZZLED group scale, in `producer`'s framing +// scale_2 F32 scalar MULTIPLIER +// +// Both scale layouts are the same permutation over the same bytes; only the +// declared 2-D shape and the nibble order differ, and `producer` names which. +// Every one of those three shapes is asserted against `out`/`in` before a byte is +// read, and a mismatch throws naming `module`. `out_bf16` receives out*in bf16 bit +// patterns. +// +// `producer` is NOT defaulted. A default here would let a caller that never +// thought about the question silently get the torchao reading for a +// nvfp4-prequant file, which is precisely the failure this seam exists to refuse. +void Ltx2DequantNvfp4ToBf16(const std::string& module, const StTensor& packed, + const StTensor& scale, const StTensor& scale_2, + int64_t out_features, int64_t in_features, + Ltx2Nvfp4Producer producer, uint16_t* out_bf16); // --------------------------------------------------------------------------- // The DiT diff --git a/scripts/gen-ltx2-quant-goldens.py b/scripts/gen-ltx2-quant-goldens.py index 29bff17cc..a54eeb766 100644 --- a/scripts/gen-ltx2-quant-goldens.py +++ b/scripts/gen-ltx2-quant-goldens.py @@ -139,6 +139,47 @@ def pinned_revision(root: Path, label: str, paths: list[str]) -> tuple[str, bool _PINNED_VLLM_PATHS = [ "vllm/model_executor/layers/quantization/qutlass_utils.py", "vllm/model_executor/layers/quantization/utils/nvfp4_utils.py", + "vllm/model_executor/layers/quantization/utils/nvfp4_emulation_utils.py", +] + +# ── THE NIBBLE ORDER, PINNED ON BOTH SIDES ────────────────────────────────── +# +# .agents/specs/nvfp4-nibble-order.md. E2M1 packs two values per byte and the two +# producers we read DISAGREE about which logical element gets which nibble, so +# both conventions are anchored against a pinned, clean checkout. A silent flip +# on either side transposes every adjacent weight pair: finite, correctly shaped, +# and wrong. + +# LOW-first, vLLM's reader — and therefore torchao's and ModelOpt's, which it +# reads. Matches torchao's own producer, `pack_uint4` +# (pytorch/ao torchao/prototype/mx_formats/kernels.py:160, +# `uint8_data[::2] | uint8_data[1::2] << 4`), which is NOT pinnable here: torchao +# is not installed on this host and not vendored. vLLM's reader is the local +# witness, and it is an independent implementation of the same convention. +_VLLM_LOW_NIBBLE_ANCHORS = [ + "high = (a_flat & 0xF0) >> 4 # Upper nibbles", + "low = a_flat & 0x0F # Lower nibbles", + "combined = torch.stack((low, high), dim=1).flatten()", +] + +# HIGH-first, Lightricks' own runtime, which is what wrote the first-party +# LTX-2.5 NVFP4 DiT. +_PINNED_LTX2_PATHS = [ + "packages/ltx-kernels/docs/NVFP4.md", + "packages/ltx-kernels/csrc/nvfp4/quantize.cu", + "packages/ltx-core/src/ltx_core/quantization/nvfp4/linear.py", +] +_LTX2_HIGH_NIBBLE_ANCHORS = [ + ("packages/ltx-kernels/docs/NVFP4.md", + "`hi_first=True` (default) puts element `2j` in the **high** nibble of byte `j`;"), + ("packages/ltx-core/src/ltx_core/quantization/nvfp4/linear.py", + "element ``2j`` in the high nibble)"), + ("packages/ltx-core/src/ltx_core/quantization/nvfp4/linear.py", + "``weight_scale`` — E4M3 block scales as ``uint8``, cuBLAS 128x4 tiled layout"), + # The permutation itself, which our Ltx2UnswizzleNvfp4BlockScale inverts, and + # the `padded_cols == roundup(K/16, 4)` framing that the DiT declares. + ("packages/ltx-kernels/csrc/nvfp4/quantize.cu", + "return static_cast(tile) * 512 + (r & 31) * 16 + (r >> 5) * 4 + (col & 3);"), ] @@ -169,6 +210,33 @@ def check_transcription(vllm_root: Path) -> None: f"{other}: `swizzle_blockscale` no longer contains {fragment!r}; " "the two vLLM producers may have diverged. Re-read both." ) + # LOW-nibble-first, vLLM's own reader. + emu = vllm_root / "vllm/model_executor/layers/quantization/utils/nvfp4_emulation_utils.py" + if not emu.is_file(): + raise SystemExit(f"not a vLLM checkout: {emu} is missing") + etext = emu.read_text(encoding="utf-8") + for fragment in _VLLM_LOW_NIBBLE_ANCHORS: + if fragment not in etext: + raise SystemExit( + f"{emu}: `break_fp4_bytes` no longer contains {fragment!r}.\n" + " That is the LOW-nibble-first convention DequantNvfp4ToBf16 defaults " + "to. If it moved, re-read it; do not relax this check." + ) + + +def check_ltx2_nibble_order(ltx2_root: Path) -> None: + """Fail if Lightricks' HIGH-first convention is no longer stated where we read it.""" + for rel, fragment in _LTX2_HIGH_NIBBLE_ANCHORS: + path = ltx2_root / rel + if not path.is_file(): + raise SystemExit(f"not an LTX-2 checkout: {path} is missing") + if fragment not in path.read_text(encoding="utf-8"): + raise SystemExit( + f"{path}: no longer contains {fragment!r}.\n" + " The first-party NVFP4 DiT is read HIGH-nibble-first on the strength of " + "that statement (.agents/specs/nvfp4-nibble-order.md section 1). Re-read " + "it; do not relax this check." + ) def load_upstream_to_blocked(vllm_root: Path): @@ -306,18 +374,28 @@ def emit_f32(out, name: str, data) -> None: def main() -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--vllm", required=True, type=Path, help="the pinned vLLM checkout") + parser.add_argument("--ltx2", required=True, type=Path, + help="the Lightricks LTX-2 checkout (the HIGH-nibble authority)") parser.add_argument("--checkpoint-root", required=True, type=Path) parser.add_argument("--out", required=True, type=Path) args = parser.parse_args() check_transcription(args.vllm) + check_ltx2_nibble_order(args.ltx2) to_blocked = load_upstream_to_blocked(args.vllm) vllm_sha, vllm_clean = pinned_revision(args.vllm, "vllm", _PINNED_VLLM_PATHS) + ltx2_sha, ltx2_clean = pinned_revision(args.ltx2, "ltx2", _PINNED_LTX2_PATHS) + root = args.checkpoint_root / "ltx-2.5/vonkaiser-fp8-nvfp4" dit_path = root / "transformer/ltx-2.5-22b-distilled-fp8.safetensors" te_path = root / "text_encoders/gemma4-12b-with-proj-nvfp4-torchao.safetensors" - for p in (dit_path, te_path): + nvfp4_dit_path = ( + args.checkpoint_root + / "ltx-2.5/lightricks-ltx-2.5/diffusion_models" + / "ltx-2.5-22b-distilled-transformer-nvfp4.safetensors" + ) + for p in (dit_path, te_path, nvfp4_dit_path): if not p.is_file(): raise SystemExit(f"missing shipped checkpoint: {p}") @@ -326,9 +404,18 @@ def main() -> int: out.write( "// GENERATED by scripts/gen-ltx2-quant-goldens.py — DO NOT EDIT BY HAND.\n" "//\n" - "// LTX-2.5 phase L6 (.agents/specs/ltx-2-5.md, issue #435): the torchao-NVFP4\n" - "// scale swizzle and the two shipped checkpoints' own bytes.\n" + "// LTX-2.5 phases L6 + L9a (.agents/specs/ltx-2-5.md, issue #435): the\n" + "// torchao-NVFP4 scale swizzle, the NIBBLE ORDER\n" + "// (.agents/specs/nvfp4-nibble-order.md), and the three shipped checkpoints'\n" + "// own bytes.\n" + "//\n" + f"// FP8 DiT {dit_path}\n" + f"// NVFP4 TE {te_path}\n" + f"// NVFP4 DiT {nvfp4_dit_path} ({nvfp4_dit_path.stat().st_size} bytes)\n" "//\n" + f"// LTX-2 revision (HIGH-nibble-first authority): {ltx2_sha}" + + ("" if ltx2_clean else " [worktree not fully clean; the pinned files are]") + + "\n" f"// vLLM revision (swizzle transcription pinned against it): {vllm_sha}\n" + ( "// Both pinned vLLM sources are clean at that revision; the rest of that\n" @@ -344,7 +431,7 @@ def main() -> int: "//\n" "// Regenerate (one line; a trailing backslash in a // comment is a\n" "// -Werror=comment line continuation):\n" - "// python3 scripts/gen-ltx2-quant-goldens.py --vllm " + "// python3 scripts/gen-ltx2-quant-goldens.py --vllm --ltx2 " " --checkpoint-root --out tests/vllm/models/ltx2_quant_goldens.inc\n" "#pragma once\n\n#include \n\nnamespace vllm_test {\n\n" ) @@ -440,6 +527,131 @@ def main() -> int: emit_bytes(out, "kLtx2RealDitFp8Head", w_raw) out.write(f"inline constexpr float kLtx2RealDitFp8Scale = {cxx_f32(dit_scale)};\n\n") emit_f32(out, "kLtx2RealDitFp8HeadF32", f8e4m3_to_f32(bytes(w_raw)) * dit_scale) + + # --- section 4: the FIRST-PARTY NVFP4 DiT, against the FP8 file as ORACLE --- + # + # Phase L9a / .agents/specs/nvfp4-nibble-order.md section 5.2. The two shipped + # DiTs quantize the SAME base weights, so the FP8 file is an oracle that is not + # ours for the NVFP4 read. What is emitted here lets the C++ suite run the + # WHOLE pipeline -- unswizzle, then hi-nibble-first dequant -- on real bytes and + # correlate the result against real FP8 weights, with a CONTROL arm. + n4_fh, n4_hdr, n4_base = open_header(nvfp4_dit_path) + n4_module = "model.diffusion_model.transformer_blocks.0.attn1.to_q" + # The control: a DIFFERENT module, so the gate must prove it can tell the right + # answer from a wrong one rather than passing on any two finite arrays. + n4_control = "model.diffusion_model.transformer_blocks.0.attn1.to_k" + + n4_w = n4_hdr[n4_module + ".weight"] + n4_out, n4_packed_in = n4_w["shape"] + n4_in = n4_packed_in * 2 + n4_groups = n4_in // 16 + if any(name.endswith(".torchao_nvfp4") for name in n4_hdr): + raise SystemExit( + f"{nvfp4_dit_path}: a torchao_nvfp4 marker appeared. Its ABSENCE is half of " + "how the loader discriminates the producer. Re-read the file." + ) + # The cuBLAS-PADDED framing of the swizzled scale, which is what this file + # declares, and which for these dims is numerically identical to the LINEAR + # [N, K/16] shape -- the ambiguity spec section 4.1 records. + padded_framing = [((n4_out + 127) // 128) * 128, ((n4_groups + 3) // 4) * 4] + to_blocked_framing = [32 * ((n4_out + 127) // 128), 16 * ((n4_groups + 3) // 4)] + if list(n4_hdr[n4_module + ".weight_scale"]["shape"]) != padded_framing: + raise SystemExit( + f"{n4_module}.weight_scale is " + f"{n4_hdr[n4_module + '.weight_scale']['shape']}, not the cuBLAS-padded " + f"{padded_framing} this section was written for." + ) + + # THE SCALE TILE, and why exactly 2048 bytes is a self-contained problem. + # Swizzled offset is tile*512 + (r%32)*16 + ((r%128)//32)*4 + (c%4), with + # tile = (r//128)*n_col_blocks + c//4. For r < 128 the row-tile index is 0, so + # n_col_blocks drops out entirely and the first 2048 bytes of the REAL scale + # are, byte for byte, a VALID STANDALONE swizzled [128, 16] grid. The C++ test + # therefore runs the real unswizzle on real bytes without the 4096x256 buffer. + n4_scale_tile = read_slice(n4_fh, n4_hdr, n4_base, n4_module + ".weight_scale", 2048) + n4_scale2 = struct.unpack( + " 128 packed bytes, 16 groups + out.write( + "// ---------------------------------------------------------------------\n" + "// Phase L9a: the FIRST-PARTY NVFP4 DiT vs the FP8 DiT as an ORACLE\n" + f"// {nvfp4_dit_path.name}\n" + f"// module {n4_module}\n" + f"// control {n4_control}\n" + f"// logical [{n4_out}, {n4_in}]; weight_scale stored " + f"{n4_hdr[n4_module + '.weight_scale']['shape']} = the cuBLAS-PADDED framing\n" + f"// {padded_framing}. torchao's to_blocked framing would be " + f"{to_blocked_framing};\n" + "// both dress the SAME bytes. This file carries NO torchao_nvfp4 marker.\n" + "//\n" + "// The first 2048 scale bytes are a valid standalone SWIZZLED [128, 16] grid\n" + "// (the row-tile index is 0 below row 128, so n_col_blocks drops out).\n" + f"// Rows {n4_rows} span all four 32-row quarters of that tile.\n" + ) + # The module's REAL geometry and the shape it REALLY declares, so the C++ gate + # can run the producer resolver on the artifact's own numbers rather than on a + # geometry invented in the test. + out.write(f"inline constexpr int64_t kLtx2RealDitNvfp4OutFeatures = {n4_out};\n") + out.write(f"inline constexpr int64_t kLtx2RealDitNvfp4InFeatures = {n4_in};\n") + out.write( + f"inline constexpr int64_t kLtx2RealDitNvfp4DeclaredScaleShape[] = " + f"{{{n4_hdr[n4_module + '.weight_scale']['shape'][0]}, " + f"{n4_hdr[n4_module + '.weight_scale']['shape'][1]}}};\n" + ) + out.write( + f'inline constexpr const char* kLtx2RealDitNvfp4Module =\n "{n4_module}";\n' + ) + out.write(f"inline constexpr int64_t kLtx2RealDitNvfp4ScaleTileRows = 128;\n") + out.write(f"inline constexpr int64_t kLtx2RealDitNvfp4ScaleTileCols = 16;\n") + out.write(f"inline constexpr int64_t kLtx2RealDitNvfp4RowElems = {n4_elems};\n") + out.write( + f"inline constexpr int64_t kLtx2RealDitNvfp4Rows[] = " + f"{{{', '.join(str(r) for r in n4_rows)}}};\n" + f"inline constexpr int64_t kLtx2RealDitNvfp4RowCount = {len(n4_rows)};\n" + ) + out.write(f"inline constexpr float kLtx2RealDitNvfp4Scale2 = {cxx_f32(n4_scale2)};\n\n") + emit_bytes(out, "kLtx2RealDitNvfp4ScaleTile", n4_scale_tile) + + # The packed NVFP4 weight rows, and the FP8 oracle's same rows. + def packed_rows(fh, hdr, base, module, rows, nbytes, stride): + begin = hdr[module + ".weight"]["data_offsets"][0] + acc = bytearray() + for r in rows: + fh.seek(base + begin + r * stride) + chunk = fh.read(nbytes) + if len(chunk) != nbytes: + raise SystemExit(f"{module} row {r}: short read") + acc += chunk + return bytes(acc) + + n4_packed = packed_rows(n4_fh, n4_hdr, n4_base, n4_module, n4_rows, + n4_elems // 2, n4_packed_in) + emit_bytes(out, "kLtx2RealDitNvfp4Packed", n4_packed) + n4_fh.close() + + # The ORACLE: the same rows of the same module out of the FP8 checkpoint, + # decoded by TORCH. Emitted as f32 VALUES, per section 7.0's "emit values". + dit_fh2, dit_hdr2, dit_base2 = open_header(dit_path) + for label, module in (("Oracle", n4_module), ("Control", n4_control)): + fw_shape = dit_hdr2[module + ".weight"]["shape"] + fw_scale = struct.unpack( + "> 4; // element 2i+1 + const uint8_t low = b & 0x0FU; + const uint8_t high = b >> 4; const float lo_val = kE2M1Lut[low & 0x7U] * ((low & 0x8U) ? -1.0F : 1.0F); const float hi_val = kE2M1Lut[high & 0x7U] * ((high & 0x8U) ? -1.0F : 1.0F); - out_row[base_elem + 2 * j] = vt::F32ToBF16(lo_val * group_scale); - out_row[base_elem + 2 * j + 1] = vt::F32ToBF16(hi_val * group_scale); + // Which of the two is element 2j is the PRODUCER's convention. Selected, + // never guessed: nvfp4-nibble-order.md section 1. Both arms do the + // identical f32 multiply and bf16 store, so this cannot perturb the + // low-first result — only which slot each value lands in. + const bool low_first = order == Nvfp4NibbleOrder::kLowFirst; + out_row[base_elem + 2 * j] = + vt::F32ToBF16((low_first ? lo_val : hi_val) * group_scale); + out_row[base_elem + 2 * j + 1] = + vt::F32ToBF16((low_first ? hi_val : lo_val) * group_scale); } } } diff --git a/src/vllm/model_executor/models/ltx2_loader.cpp b/src/vllm/model_executor/models/ltx2_loader.cpp index cc44f2a9c..d2591fc21 100644 --- a/src/vllm/model_executor/models/ltx2_loader.cpp +++ b/src/vllm/model_executor/models/ltx2_loader.cpp @@ -197,10 +197,76 @@ void Ltx2UnswizzleNvfp4BlockScale(const uint8_t* swizzled, size_t swizzled_bytes } } -void Ltx2DequantTorchaoNvfp4ToBf16(const std::string& module, const StTensor& packed, - const StTensor& scale, const StTensor& scale_2, - int64_t out_features, int64_t in_features, - uint16_t* out_bf16) { +std::vector Ltx2Nvfp4ToBlockedScaleShape(int64_t out_features, + int64_t in_features) { + const int64_t groups = in_features / kNvfp4GroupSize; + return {RoundUp(out_features, 128) / 4, RoundUp(groups, 4) * 4}; +} + +std::vector Ltx2Nvfp4PaddedScaleShape(int64_t out_features, + int64_t in_features) { + const int64_t groups = in_features / kNvfp4GroupSize; + return {RoundUp(out_features, 128), RoundUp(groups, 4)}; +} + +Nvfp4NibbleOrder Ltx2Nvfp4NibbleOrderFor(Ltx2Nvfp4Producer producer) { + return producer == Ltx2Nvfp4Producer::kTorchao ? Nvfp4NibbleOrder::kLowFirst + : Nvfp4NibbleOrder::kHighFirst; +} + +Ltx2Nvfp4Producer Ltx2ResolveNvfp4Producer(const std::string& module, + const Ltx2TorchaoNvfp4Marker* marker, + const std::vector& scale_shape, + int64_t out_features, int64_t in_features) { + if (in_features % kNvfp4GroupSize != 0) { + Fail("'" + module + "': in_features " + std::to_string(in_features) + + " is not a multiple of the group size " + std::to_string(kNvfp4GroupSize)); + } + const std::vector blocked = Ltx2Nvfp4ToBlockedScaleShape(out_features, in_features); + const std::vector padded = Ltx2Nvfp4PaddedScaleShape(out_features, in_features); + const bool is_blocked = scale_shape == blocked; + const bool is_padded = scale_shape == padded; + + // The MARKER decides; the shape only corroborates. A disagreement is refused + // rather than resolved in either direction, because both readings type-check + // and one of them permutes every scale in a 128x4 tile. + if (marker != nullptr) { + if (is_blocked) return Ltx2Nvfp4Producer::kTorchao; + Fail("'" + module + ".weight_scale' is " + ShapeText(scale_shape) + + " but the module carries a torchao_nvfp4 marker, and a torchao swizzled scale " + "for [" + std::to_string(out_features) + ", " + std::to_string(in_features) + + "] is stored " + ShapeText(blocked) + + (is_padded ? ". It is instead the cuBLAS-PADDED framing " + ShapeText(padded) + + ", which is what the Lightricks nvfp4-prequant tool writes -- and that " + "producer also packs the OPPOSITE nibble order. The marker says " + "torchao and the shape says nvfp4-prequant; refusing rather " + "than picking one, because each reading is finite and wrong " + "under the other's assumption." + : ". The declaration and the stored shape disagree.") + + " See .agents/specs/nvfp4-nibble-order.md section 3.2."); + } + + // No marker anywhere for this module. torchao ALWAYS writes one, so its absence + // excludes torchao — the inference ltx2_loader.h records, with its evidence. + if (is_padded) return Ltx2Nvfp4Producer::kNvfp4Prequant; + Fail("'" + module + ".weight_scale' is " + ShapeText(scale_shape) + + " and the module carries NO torchao_nvfp4 marker. Without a marker this port " + "reads the Lightricks nvfp4-prequant layout, whose swizzled scale for [" + + std::to_string(out_features) + ", " + std::to_string(in_features) + + "] is stored " + ShapeText(padded) + + (is_blocked ? " -- but this is the torchao to_blocked framing " + ShapeText(blocked) + + ", and a torchao file without its marker is a combination this " + "port does not recognise. The two producers pack DIFFERENT " + "nibble orders, so guessing picks a byte encoding as well as a " + "layout." + : ", which this is not.") + + " Refusing by name. See .agents/specs/nvfp4-nibble-order.md section 3.2."); +} + +void Ltx2DequantNvfp4ToBf16(const std::string& module, const StTensor& packed, + const StTensor& scale, const StTensor& scale_2, + int64_t out_features, int64_t in_features, + Ltx2Nvfp4Producer producer, uint16_t* out_bf16) { if (packed.dtype != "U8") { Fail("'" + module + ".weight' must be U8 (two E2M1 values per byte), not " + packed.dtype); @@ -228,41 +294,44 @@ void Ltx2DequantTorchaoNvfp4ToBf16(const std::string& module, const StTensor& pa Fail("'" + module + ".weight_scale' must be F8_E4M3, not " + scale.dtype); } const int64_t groups = in_features / kNvfp4GroupSize; - const int64_t padded_rows = RoundUp(out_features, 128); - const int64_t padded_cols = RoundUp(groups, 4); - const std::vector want_shape = {padded_rows / 4, padded_cols * 4}; + // The shape the RESOLVED producer stores. Both framings dress the same buffer, + // so this asserts the caller's resolution against the file rather than + // re-deciding it: a caller that resolved torchao and hands a padded-framing + // scale is refused here even though the bytes would unswizzle identically, + // because it would then decode them with the wrong NIBBLE ORDER. + const std::vector want_shape = + producer == Ltx2Nvfp4Producer::kTorchao + ? Ltx2Nvfp4ToBlockedScaleShape(out_features, in_features) + : Ltx2Nvfp4PaddedScaleShape(out_features, in_features); if (scale.shape != want_shape) { - // NAME WHAT WAS SEEN, and attribute "torchao" to what this port READS rather - // than to the file. The first-party `Lightricks/LTX-2.5` NVFP4 DiT lands - // here, and it carries no `.torchao_nvfp4` marker at all — so a message - // saying its scale is not "a SWIZZLED torchao scale" described the file as a - // broken torchao checkpoint when it is simply a different, valid layout this - // port does not read yet. That sends a reader looking for corruption instead - // of for the missing loader arm. - const std::vector linear_shape = {out_features, groups}; - const bool is_linear = scale.shape == linear_shape; + const std::vector other = + producer == Ltx2Nvfp4Producer::kTorchao + ? Ltx2Nvfp4PaddedScaleShape(out_features, in_features) + : Ltx2Nvfp4ToBlockedScaleShape(out_features, in_features); Fail("'" + module + ".weight_scale' is " + scale.dtype + " " + ShapeText(scale.shape) + - (is_linear ? ", which is exactly the LINEAR [N, K/16] group-scale layout" : "") + - ". This port reads the SWIZZLED torchao layout, which for [" + - std::to_string(out_features) + ", " + std::to_string(in_features) + "] is stored as " + - ShapeText(want_shape) + ". " + - (is_linear ? std::string("The two have the same element count, so reading this one as " - "swizzled type-checks and permutes every scale within a " - "128x4 tile. Reading the LINEAR layout is owed loader work; " - "refusing rather than silently permuting.") - : "The LINEAR shape " + ShapeText(linear_shape) + - " has the same element count, so reading one as the other " - "type-checks and permutes every scale within a 128x4 tile.")); + ", but this module resolved to the " + + (producer == Ltx2Nvfp4Producer::kTorchao ? "torchao" : "nvfp4-prequant") + + " producer, whose swizzled group scale for [" + std::to_string(out_features) + + ", " + std::to_string(in_features) + "] is stored " + ShapeText(want_shape) + + ". The other producer framing would be " + ShapeText(other) + + "; both have the same element count, so reading one as the other type-checks " + "and — because the two producers also pack OPPOSITE nibble orders — decodes " + "every weight pair transposed. See .agents/specs/nvfp4-nibble-order.md."); } const float global = ReadScalarF32(module + ".weight_scale_2", scale_2); + // ONE permutation, both framings. `Ltx2UnswizzleNvfp4BlockScale` keys off the + // LOGICAL rows/cols and the byte count, which are identical either way, so + // nothing about the swizzle inverse is producer-specific. std::vector linear(static_cast(out_features) * static_cast(groups)); Ltx2UnswizzleNvfp4BlockScale(static_cast(scale.data), scale.nbytes, out_features, groups, linear.data()); - // From here the path is the EXISTING modelopt one, byte for byte. + // From here the path is the EXISTING modelopt one, byte for byte, except for the + // producer's nibble order. DequantNvfp4ToBf16(static_cast(packed.data), linear.data(), global, - out_features, in_features, out_bf16); + out_features, in_features, out_bf16, + Ltx2Nvfp4NibbleOrderFor(producer)); } // --------------------------------------------------------------------------- @@ -281,6 +350,10 @@ struct DitPlan { std::map> logical; std::vector manifest; std::vector unported; // module families, header order, deduped + // module (bare) -> the `.torchao_nvfp4` FILE name, for every module that + // declares one. Absent from this map means the file carries NO marker for that + // module, which is what `Ltx2ResolveNvfp4Producer` reads as "not torchao". + std::map marker_name; }; bool IsScaleSidecar(const std::string& bare) { @@ -308,8 +381,12 @@ DitPlan PlanDit(const SafetensorsFile& file) { plan.prefix = prefixed != 0 ? prefix : std::string(); bool saw_u8 = false, saw_f8 = false; + const std::string marker_suffix = kLtx2TorchaoNvfp4MarkerSuffix; for (const std::string& n : names) { const std::string bare = n.substr(plan.prefix.size()); + if (EndsWith(bare, marker_suffix)) { + plan.marker_name[bare.substr(0, bare.size() - marker_suffix.size())] = n; + } if (IsScaleSidecar(bare)) continue; const StTensor& t = file.Get(n); std::vector shape = t.shape; @@ -398,14 +475,24 @@ vt::DType MaterializeDitTensor(const SafetensorsFile& file, const DitPlan& plan, const StTensor& scale = file.Get(fname + "_scale"); const StTensor& global = file.Get(fname + "_scale_2"); buffer.resize(static_cast(numel) * sizeof(uint16_t)); - // Ltx2DequantTorchaoNvfp4ToBf16 takes a MODULE prefix and appends `.weight`, + // Ltx2DequantNvfp4ToBf16 takes a MODULE prefix and appends `.weight`, // `.weight_scale` and `.weight_scale_2` itself. `spec.name` is already the // full tensor name, so passing it produced 'patchify_proj.weight.weight' — // a name that appears nowhere in the checkpoint, which defeats the whole // point of refusing BY NAME. Strip the suffix the callee will re-add. - Ltx2DequantTorchaoNvfp4ToBf16(ModulePrefixOfWeight(spec.name), t, scale, global, - spec.shape[0], spec.shape[1], - reinterpret_cast(buffer.data())); + const std::string module = ModulePrefixOfWeight(spec.name); + // Which producer wrote this file decides BOTH the scale framing and the nibble + // order, and it is resolved from the marker the file does or does not carry. + const auto marker_it = plan.marker_name.find(module); + Ltx2TorchaoNvfp4Marker marker; + const bool has_marker = marker_it != plan.marker_name.end(); + if (has_marker) { + marker = ParseLtx2TorchaoNvfp4Marker(module, file.Get(marker_it->second)); + } + const Ltx2Nvfp4Producer producer = Ltx2ResolveNvfp4Producer( + module, has_marker ? &marker : nullptr, scale.shape, spec.shape[0], spec.shape[1]); + Ltx2DequantNvfp4ToBf16(module, t, scale, global, spec.shape[0], spec.shape[1], + producer, reinterpret_cast(buffer.data())); return vt::DType::kBF16; } Fail("'" + spec.name + "' has dtype " + t.dtype + ", which this loader does not read"); @@ -585,10 +672,19 @@ Ltx2TextProjection LoadProjection(const SafetensorsFile& file, const std::string const StTensor* g = Find(file, module + ".weight_scale_2"); if (s == nullptr) Fail("the text encoder is missing '" + module + ".weight_scale'"); if (g == nullptr) Fail("the text encoder is missing '" + module + ".weight_scale_2'"); + // Resolved, not assumed — even though this file is torchao and its marker is + // present. Hard-coding kTorchao here would make the projections the one NVFP4 + // path in the loader that cannot notice a producer change. + const StTensor* m = Find(file, module + kLtx2TorchaoNvfp4MarkerSuffix); + Ltx2TorchaoNvfp4Marker marker; + if (m != nullptr) marker = ParseLtx2TorchaoNvfp4Marker(module, *m); + const Ltx2Nvfp4Producer producer = Ltx2ResolveNvfp4Producer( + module, m != nullptr ? &marker : nullptr, s->shape, proj.out_features, + proj.in_features); proj.weight_bf16.resize(static_cast(proj.out_features) * static_cast(proj.in_features)); - Ltx2DequantTorchaoNvfp4ToBf16(module, *w, *s, *g, proj.out_features, proj.in_features, - proj.weight_bf16.data()); + Ltx2DequantNvfp4ToBf16(module, *w, *s, *g, proj.out_features, proj.in_features, + producer, proj.weight_bf16.data()); const StTensor* b = Find(file, module + ".bias"); if (b != nullptr) { @@ -647,7 +743,7 @@ Ltx2TextEncoderCheckpoint Ltx2LoadTextEncoderFromSafetensors( if (!EndsWith(name, marker_suffix)) continue; const std::string module = name.substr(0, name.size() - marker_suffix.size()); out.quantized_modules.push_back(module); - ParseLtx2TorchaoNvfp4Marker(module, file.Get(name)); + const Ltx2TorchaoNvfp4Marker parsed = ParseLtx2TorchaoNvfp4Marker(module, file.Get(name)); const StTensor* w = Find(file, module + ".weight"); const StTensor* s = Find(file, module + ".weight_scale"); @@ -671,13 +767,15 @@ Ltx2TextEncoderCheckpoint Ltx2LoadTextEncoderFromSafetensors( Fail("'" + module + "' unpacks to in_features " + std::to_string(in_features) + ", not a multiple of " + std::to_string(kNvfp4GroupSize)); } - const std::vector want = {RoundUp(out_features, 128) / 4, - RoundUp(in_features / kNvfp4GroupSize, 4) * 4}; - if (s->dtype != "F8_E4M3" || s->shape != want) { - Fail("'" + module + ".weight_scale' is " + s->dtype + " " + ShapeText(s->shape) + - " but a swizzled torchao scale for [" + std::to_string(out_features) + ", " + - std::to_string(in_features) + "] is F8_E4M3 " + ShapeText(want)); + if (s->dtype != "F8_E4M3") { + Fail("'" + module + ".weight_scale' is " + s->dtype + + ", and an NVFP4 group scale is F8_E4M3"); } + // Every module in this loop carries a marker (the loop is keyed on them), so + // this validates the torchao framing — through the SAME resolver the + // materializing paths use, so a validation pass cannot accept a shape the + // load would then refuse. + (void)Ltx2ResolveNvfp4Producer(module, &parsed, s->shape, out_features, in_features); if (g->dtype != "F32") { Fail("'" + module + ".weight_scale_2' must be F32, not " + g->dtype); } diff --git a/tests/vllm/models/ltx2_quant_goldens.inc b/tests/vllm/models/ltx2_quant_goldens.inc index 4173eeaa6..5b956c251 100644 --- a/tests/vllm/models/ltx2_quant_goldens.inc +++ b/tests/vllm/models/ltx2_quant_goldens.inc @@ -1,8 +1,15 @@ // GENERATED by scripts/gen-ltx2-quant-goldens.py — DO NOT EDIT BY HAND. // -// LTX-2.5 phase L6 (.agents/specs/ltx-2-5.md, issue #435): the torchao-NVFP4 -// scale swizzle and the two shipped checkpoints' own bytes. +// LTX-2.5 phases L6 + L9a (.agents/specs/ltx-2-5.md, issue #435): the +// torchao-NVFP4 scale swizzle, the NIBBLE ORDER +// (.agents/specs/nvfp4-nibble-order.md), and the three shipped checkpoints' +// own bytes. // +// FP8 DiT /mnt/nas_share/checkpoints/ltx-2.5/vonkaiser-fp8-nvfp4/transformer/ltx-2.5-22b-distilled-fp8.safetensors +// NVFP4 TE /mnt/nas_share/checkpoints/ltx-2.5/vonkaiser-fp8-nvfp4/text_encoders/gemma4-12b-with-proj-nvfp4-torchao.safetensors +// NVFP4 DiT /mnt/nas_share/checkpoints/ltx-2.5/lightricks-ltx-2.5/diffusion_models/ltx-2.5-22b-distilled-transformer-nvfp4.safetensors (18721432024 bytes) +// +// LTX-2 revision (HIGH-nibble-first authority): fd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca // vLLM revision (swizzle transcription pinned against it): 5559679229bc961848b121ccdeaa8fa5d79bec98 // Both pinned vLLM sources are clean at that revision; the rest of that // checkout's worktree is NOT, so the anchor covers those two files only. @@ -13,7 +20,7 @@ // // Regenerate (one line; a trailing backslash in a // comment is a // -Werror=comment line continuation): -// python3 scripts/gen-ltx2-quant-goldens.py --vllm --checkpoint-root --out tests/vllm/models/ltx2_quant_goldens.inc +// python3 scripts/gen-ltx2-quant-goldens.py --vllm --ltx2 --checkpoint-root --out tests/vllm/models/ltx2_quant_goldens.inc #pragma once #include @@ -99,4 +106,44 @@ inline constexpr float kLtx2RealDitFp8Scale = 0.00103759766F; inline constexpr float kLtx2RealDitFp8HeadF32[] = {7.29560852e-05F, -0.00933837891F, -0.0186767578F, -0.0124511719F, 0.0415039062F, -0.00933837891F, 0.000907897949F, -0.0103759766F, 0.00466918945F, 0.0036315918F, 0.0134887695F, -0.0373535156F, -0.00259399414F, 0.0103759766F, -0.00415039062F, -0.0036315918F, -0.0155639648F, -0.0249023438F, 0.0114135742F, -0.000583648682F, -0.0498046875F, -0.0124511719F, -0.0114135742F, 0.0415039062F, 0.00233459473F, 0.00337219238F, -0.000648498535F, 0.00778198242F, 0.0145263672F, 0.00415039062F, -0.00233459473F, -0.00726318359F}; inline constexpr int64_t kLtx2RealDitFp8HeadF32Count = 32; +// --------------------------------------------------------------------- +// Phase L9a: the FIRST-PARTY NVFP4 DiT vs the FP8 DiT as an ORACLE +// ltx-2.5-22b-distilled-transformer-nvfp4.safetensors +// module model.diffusion_model.transformer_blocks.0.attn1.to_q +// control model.diffusion_model.transformer_blocks.0.attn1.to_k +// logical [4096, 4096]; weight_scale stored [4096, 256] = the cuBLAS-PADDED framing +// [4096, 256]. torchao's to_blocked framing would be [1024, 1024]; +// both dress the SAME bytes. This file carries NO torchao_nvfp4 marker. +// +// The first 2048 scale bytes are a valid standalone SWIZZLED [128, 16] grid +// (the row-tile index is 0 below row 128, so n_col_blocks drops out). +// Rows [1, 33, 65, 97, 2, 34, 66, 98] span all four 32-row quarters of that tile. +inline constexpr int64_t kLtx2RealDitNvfp4OutFeatures = 4096; +inline constexpr int64_t kLtx2RealDitNvfp4InFeatures = 4096; +inline constexpr int64_t kLtx2RealDitNvfp4DeclaredScaleShape[] = {4096, 256}; +inline constexpr const char* kLtx2RealDitNvfp4Module = + "model.diffusion_model.transformer_blocks.0.attn1.to_q"; +inline constexpr int64_t kLtx2RealDitNvfp4ScaleTileRows = 128; +inline constexpr int64_t kLtx2RealDitNvfp4ScaleTileCols = 16; +inline constexpr int64_t kLtx2RealDitNvfp4RowElems = 256; +inline constexpr int64_t kLtx2RealDitNvfp4Rows[] = {1, 33, 65, 97, 2, 34, 66, 98}; +inline constexpr int64_t kLtx2RealDitNvfp4RowCount = 8; +inline constexpr float kLtx2RealDitNvfp4Scale2 = 0.0002784729F; + +inline constexpr uint8_t kLtx2RealDitNvfp4ScaleTile[] = {73, 73, 76, 75, 78, 71, 73, 74, 79, 75, 70, 76, 75, 70, 73, 68, 73, 70, 75, 71, 76, 75, 75, 73, 73, 73, 77, 82, 77, 75, 77, 74, 77, 73, 70, 76, 72, 72, 72, 74, 80, 75, 67, 75, 74, 75, 79, 74, 73, 70, 76, 77, 74, 74, 73, 74, 75, 74, 77, 71, 74, 75, 75, 71, 75, 73, 71, 74, 74, 75, 76, 78, 78, 75, 73, 74, 76, 74, 73, 75, 76, 76, 70, 79, 74, 75, 79, 74, 81, 75, 77, 80, 75, 72, 75, 77, 77, 80, 79, 81, 83, 80, 77, 88, 74, 73, 73, 75, 78, 73, 78, 76, 73, 73, 76, 74, 76, 70, 78, 78, 74, 75, 75, 75, 76, 73, 75, 80, 94, 83, 88, 92, 75, 74, 75, 71, 120, 106, 100, 107, 70, 70, 73, 73, 76, 76, 72, 83, 73, 72, 70, 72, 74, 79, 75, 77, 76, 72, 74, 77, 77, 73, 76, 73, 73, 78, 71, 79, 75, 75, 80, 69, 74, 75, 73, 70, 74, 74, 73, 75, 73, 72, 79, 73, 72, 75, 74, 73, 77, 74, 73, 69, 80, 73, 74, 73, 74, 75, 74, 73, 75, 73, 74, 73, 77, 76, 80, 72, 78, 72, 73, 80, 76, 73, 76, 76, 71, 71, 72, 72, 75, 74, 70, 75, 78, 71, 77, 81, 76, 75, 75, 82, 73, 74, 78, 72, 74, 79, 74, 81, 77, 72, 75, 73, 73, 72, 77, 72, 74, 75, 74, 77, 76, 75, 75, 74, 72, 78, 75, 76, 74, 73, 75, 77, 78, 75, 73, 76, 81, 76, 80, 74, 70, 74, 76, 75, 81, 74, 84, 78, 72, 77, 78, 77, 72, 73, 79, 76, 76, 74, 74, 87, 74, 73, 73, 71, 71, 78, 70, 76, 77, 74, 74, 74, 74, 74, 75, 75, 77, 72, 75, 73, 76, 73, 74, 76, 76, 78, 73, 83, 77, 74, 83, 78, 73, 75, 69, 74, 82, 77, 78, 89, 70, 73, 72, 74, 75, 75, 79, 71, 76, 77, 80, 75, 81, 77, 73, 75, 75, 71, 72, 73, 77, 74, 78, 76, 73, 74, 68, 79, 71, 75, 73, 76, 78, 74, 76, 77, 74, 69, 71, 78, 77, 71, 72, 74, 73, 73, 75, 77, 74, 76, 74, 75, 77, 74, 74, 76, 79, 72, 74, 72, 74, 75, 76, 79, 75, 70, 73, 73, 72, 78, 78, 73, 75, 79, 75, 75, 78, 73, 73, 79, 75, 76, 76, 84, 74, 76, 70, 74, 72, 67, 74, 76, 85, 69, 73, 78, 74, 73, 77, 74, 80, 75, 74, 76, 75, 73, 77, 73, 73, 71, 75, 73, 80, 74, 75, 74, 76, 74, 71, 73, 73, 74, 78, 78, 72, 74, 76, 71, 76, 72, 74, 75, 77, 75, 72, 78, 81, 73, 82, 87, 81, 76, 79, 77, 73, 76, 76, 79, 72, 80, 72, 73, 108, 92, 96, 103, 81, 76, 72, 86, 117, 109, 101, 100, 72, 73, 74, 73, 76, 70, 73, 75, 76, 75, 70, 75, 72, 74, 76, 69, 77, 72, 76, 75, 75, 78, 72, 75, 80, 74, 74, 74, 73, 73, 76, 71, 73, 73, 73, 74, 76, 74, 80, 79, 74, 75, 75, 76, 72, 79, 69, 72, 73, 71, 71, 76, 74, 78, 80, 74, 70, 79, 75, 71, 72, 70, 77, 73, 76, 73, 79, 74, 69, 74, 76, 74, 72, 76, 77, 76, 74, 81, 79, 71, 74, 73, 75, 79, 71, 75, 73, 68, 74, 70, 76, 74, 74, 79, 74, 78, 74, 74, 75, 75, 73, 73, 79, 72, 75, 76, 78, 75, 75, 74, 73, 76, 72, 80, 80, 73, 76, 78, 76, 74, 80, 72, 78, 76, 84, 75, 79, 71, 77, 75, 80, 70, 72, 73, 76, 76, 76, 76, 77, 76, 72, 77, 78, 80, 98, 88, 87, 77, 72, 75, 72, 76, 97, 104, 111, 100, 73, 71, 70, 76, 73, 78, 74, 76, 76, 77, 73, 74, 71, 74, 77, 73, 75, 77, 73, 80, 76, 70, 76, 76, 76, 76, 77, 77, 74, 74, 75, 73, 73, 79, 74, 73, 75, 76, 74, 73, 75, 71, 77, 74, 74, 71, 73, 76, 73, 75, 75, 76, 73, 73, 78, 71, 74, 73, 72, 76, 73, 78, 77, 73, 76, 82, 75, 76, 75, 74, 77, 73, 72, 78, 75, 75, 80, 69, 75, 74, 75, 77, 73, 71, 75, 74, 78, 75, 80, 78, 74, 72, 74, 74, 73, 76, 78, 78, 80, 76, 74, 72, 72, 73, 72, 72, 70, 77, 82, 72, 73, 73, 77, 75, 75, 74, 76, 72, 79, 70, 80, 68, 74, 73, 78, 72, 79, 75, 77, 71, 75, 80, 81, 75, 74, 72, 73, 75, 77, 77, 79, 71, 78, 68, 73, 75, 74, 76, 76, 76, 76, 74, 73, 77, 72, 76, 72, 74, 76, 72, 73, 75, 67, 71, 76, 74, 77, 71, 76, 76, 73, 74, 74, 70, 73, 74, 77, 78, 77, 78, 94, 77, 83, 75, 77, 77, 74, 75, 80, 74, 77, 77, 77, 75, 73, 71, 76, 75, 73, 77, 78, 76, 72, 72, 73, 76, 75, 72, 74, 71, 74, 73, 72, 74, 78, 73, 75, 74, 75, 71, 78, 74, 73, 73, 74, 75, 74, 73, 74, 70, 73, 73, 75, 75, 73, 73, 73, 74, 69, 74, 73, 75, 73, 75, 72, 72, 73, 72, 71, 73, 74, 75, 76, 71, 74, 73, 74, 75, 75, 75, 78, 75, 74, 81, 74, 78, 74, 80, 78, 74, 75, 77, 80, 75, 75, 80, 78, 76, 75, 74, 77, 75, 72, 72, 75, 78, 78, 78, 77, 73, 69, 75, 76, 80, 73, 69, 72, 82, 75, 72, 73, 73, 75, 75, 76, 72, 74, 75, 73, 72, 74, 71, 71, 81, 75, 73, 75, 74, 79, 73, 75, 76, 78, 70, 72, 76, 72, 73, 76, 75, 77, 74, 74, 75, 72, 78, 77, 75, 75, 76, 73, 75, 68, 70, 114, 89, 99, 94, 74, 75, 75, 74, 99, 100, 105, 99, 76, 74, 77, 77, 74, 78, 74, 73, 73, 72, 73, 75, 75, 80, 73, 71, 74, 69, 77, 73, 80, 78, 81, 76, 78, 78, 73, 74, 76, 74, 69, 71, 75, 76, 79, 77, 81, 74, 82, 75, 83, 75, 83, 78, 80, 76, 78, 76, 77, 70, 76, 75, 83, 72, 79, 74, 77, 74, 77, 73, 73, 74, 81, 75, 71, 73, 82, 72, 81, 73, 71, 72, 79, 74, 72, 73, 78, 77, 73, 70, 75, 74, 74, 74, 82, 74, 79, 76, 77, 72, 73, 74, 78, 76, 77, 78, 73, 78, 79, 73, 74, 72, 81, 75, 74, 77, 75, 76, 74, 78, 73, 70, 84, 75, 79, 74, 81, 82, 86, 80, 82, 74, 73, 72, 77, 80, 79, 80, 76, 74, 82, 74, 77, 75, 78, 75, 74, 76, 74, 74, 80, 76, 78, 75, 92, 101, 86, 84, 73, 73, 69, 75, 100, 95, 120, 108, 77, 73, 75, 72, 75, 75, 78, 74, 77, 77, 80, 74, 67, 74, 74, 74, 73, 73, 77, 72, 73, 73, 70, 75, 83, 72, 80, 72, 76, 77, 76, 74, 83, 77, 75, 80, 77, 75, 79, 81, 76, 71, 77, 78, 81, 75, 74, 78, 80, 75, 71, 72, 74, 73, 83, 72, 74, 74, 80, 70, 76, 76, 79, 77, 82, 76, 75, 80, 80, 73, 78, 74, 73, 76, 76, 72, 78, 72, 81, 73, 76, 75, 74, 76, 69, 77, 80, 73, 78, 77, 84, 76, 80, 78, 76, 72, 85, 78, 79, 74, 81, 75, 72, 74, 73, 73, 76, 75, 72, 77, 71, 74, 80, 75, 67, 74, 70, 78, 73, 81, 78, 73, 84, 72, 78, 78, 75, 79, 80, 77, 79, 73, 81, 72, 78, 75, 81, 76, 78, 81, 73, 72, 76, 74, 75, 77, 82, 71, 79, 76, 81, 75, 73, 75, 70, 65, 75, 74, 74, 80, 76, 73, 78, 75, 80, 76, 85, 74, 74, 68, 76, 74, 80, 77, 70, 78, 79, 76, 80, 72, 79, 90, 84, 78, 81, 73, 77, 77, 80, 81, 88, 75, 83, 73, 69, 74, 75, 77, 79, 71, 80, 72, 79, 72, 76, 76, 77, 73, 71, 74, 73, 73, 72, 75, 82, 73, 71, 74, 74, 77, 78, 77, 73, 75, 73, 77, 74, 73, 74, 73, 77, 73, 82, 76, 76, 75, 68, 80, 70, 76, 81, 75, 68, 74, 75, 77, 76, 76, 71, 76, 77, 74, 75, 74, 81, 72, 81, 81, 75, 73, 75, 76, 73, 73, 79, 80, 74, 73, 75, 80, 81, 75, 72, 72, 83, 79, 73, 76, 78, 77, 76, 73, 74, 76, 83, 75, 77, 81, 73, 77, 73, 77, 80, 73, 78, 78, 77, 73, 76, 75, 83, 72, 75, 72, 73, 77, 80, 73, 74, 77, 73, 74, 69, 82, 83, 77, 81, 77, 74, 74, 77, 73, 79, 75, 74, 76, 77, 72, 77, 76, 82, 75, 81, 75, 84, 75, 73, 79, 80, 72, 74, 75, 79, 74, 95, 119, 92, 91, 73, 74, 82, 70, 99, 94, 117, 105, 75, 73, 77, 76, 77, 76, 74, 74, 73, 73, 76, 78, 75, 73, 72, 71, 72, 74, 74, 71, 73, 75, 69, 78, 68, 75, 72, 73, 72, 74, 74, 72, 75, 71, 75, 76, 75, 85, 74, 74, 75, 74, 78, 73, 78, 74, 72, 82, 71, 78, 72, 74, 74, 85, 74, 75, 74, 75, 76, 74, 76, 77, 76, 74, 73, 81, 74, 74, 69, 70, 74, 73, 76, 80, 73, 72, 77, 74, 72, 74, 72, 78, 75, 70, 75, 75, 72, 75, 70, 77, 73, 76, 78, 76, 75, 77, 78, 78, 74, 70, 79, 70, 74, 74, 70, 71, 78, 74, 74, 71, 75, 78, 77, 80, 76, 79, 79, 81, 81, 78, 73, 72, 74, 70, 73, 82, 72, 76, 79, 82, 70, 75, 74, 75, 73, 76, 72, 72, 72, 79, 73, 74, 78, 70, 92, 90, 88, 89, 72, 73, 79, 73, 101, 96, 101, 119, 69, 72, 75, 77, 73, 79, 74, 76, 73, 77, 72, 73, 74, 77, 78, 73, 74, 74, 70, 77, 73, 74, 72, 76, 73, 76, 70, 75, 76, 81, 69, 77, 75, 81, 69, 73, 74, 73, 73, 75, 76, 73, 78, 72, 73, 73, 70, 76, 74, 77, 79, 78, 73, 79, 75, 74, 72, 73, 73, 74, 76, 76, 76, 73, 69, 73, 73, 70, 74, 71, 79, 75, 73, 73, 72, 70, 74, 76, 74, 74, 76, 79, 77, 75, 79, 76, 74, 75, 76, 74, 76, 72, 76, 80, 70, 74, 78, 78, 74, 73, 76, 77, 76, 74, 75, 72, 74, 76, 74, 74, 73, 75, 73, 75, 74, 73, 75, 74, 71, 71, 76, 74, 74, 70, 75, 73, 79, 75, 74, 72, 74, 75, 73, 75, 72, 69, 73, 73, 76, 78, 75, 74, 73, 74, 76, 76, 77, 82, 76, 74, 73, 75, 71, 75, 80, 81, 75, 74, 76, 67, 75, 80, 74, 76, 80, 77, 74, 75, 72, 84, 74, 77, 73, 77, 72, 74, 75, 75, 76, 76, 74, 83, 76, 77, 75, 75, 73, 74, 81, 79, 80, 78, 74, 75, 72, 76, 76, 75, 72, 74, 73, 73, 76, 77, 75, 76, 74, 74, 75, 69, 73, 78, 80, 73, 73, 78, 75, 73, 73, 73, 72, 76, 77, 76, 74, 76, 76, 78, 75, 78, 73, 74, 73, 77, 74, 72, 72, 74, 73, 73, 72, 74, 75, 73, 74, 75, 77, 79, 72, 73, 74, 73, 69, 77, 75, 74, 74, 73, 73, 75, 81, 76, 69, 73, 76, 83, 75, 75, 74, 76, 74, 75, 77, 78, 75, 77, 75, 78, 76, 75, 76, 75, 73, 73, 78, 80, 74, 77, 75, 69, 70, 74, 74, 81, 73, 75, 72, 74, 73, 73, 71, 70, 75, 73, 74, 77, 73, 82, 71, 78, 74, 73, 75, 72, 73, 75, 78, 74, 73, 77, 76, 73, 73, 77, 72, 69, 73, 85, 80, 75, 78, 73, 72, 75, 76, 81, 72, 78, 76, 76, 74, 75, 71, 76, 101, 105, 95, 92, 73, 79, 76, 73, 100, 106, 91, 115, 77, 76, 74, 73, 74, 74, 74, 75, 75, 83, 73, 77, 75, 80, 76, 76}; +inline constexpr int64_t kLtx2RealDitNvfp4ScaleTileCount = 2048; + +inline constexpr uint8_t kLtx2RealDitNvfp4Packed[] = {191, 237, 54, 227, 106, 230, 126, 94, 108, 111, 244, 255, 179, 70, 185, 194, 159, 195, 209, 235, 223, 149, 49, 172, 245, 126, 246, 222, 138, 116, 29, 198, 170, 73, 39, 236, 41, 171, 26, 45, 216, 244, 231, 44, 102, 210, 197, 108, 75, 213, 238, 26, 94, 193, 98, 247, 117, 27, 201, 100, 102, 69, 231, 191, 236, 99, 36, 235, 198, 246, 204, 103, 217, 39, 16, 218, 205, 113, 11, 179, 17, 177, 167, 25, 53, 160, 34, 4, 124, 64, 84, 97, 13, 61, 218, 83, 80, 45, 172, 242, 207, 67, 178, 95, 46, 230, 107, 183, 241, 109, 16, 237, 69, 58, 193, 192, 173, 211, 143, 85, 221, 177, 205, 89, 30, 209, 199, 94, 59, 244, 205, 212, 90, 187, 145, 70, 228, 244, 201, 140, 102, 40, 240, 88, 73, 119, 35, 212, 155, 206, 206, 37, 97, 217, 154, 151, 187, 237, 86, 177, 79, 141, 248, 93, 124, 78, 153, 77, 227, 210, 39, 185, 46, 23, 34, 85, 41, 201, 165, 83, 18, 20, 183, 219, 196, 160, 146, 20, 91, 209, 250, 229, 224, 49, 115, 162, 84, 142, 91, 65, 2, 61, 161, 34, 159, 229, 237, 102, 114, 96, 133, 18, 165, 34, 153, 26, 192, 236, 92, 99, 247, 20, 42, 110, 248, 173, 22, 173, 223, 89, 79, 75, 42, 43, 49, 169, 23, 162, 73, 28, 28, 71, 155, 21, 10, 194, 17, 115, 24, 98, 61, 58, 200, 209, 207, 79, 247, 71, 222, 238, 215, 122, 71, 245, 125, 101, 117, 246, 197, 95, 218, 25, 173, 28, 248, 109, 131, 40, 212, 21, 179, 252, 204, 166, 52, 228, 70, 84, 112, 198, 145, 54, 67, 99, 223, 111, 214, 150, 124, 78, 132, 171, 122, 225, 244, 92, 196, 206, 156, 82, 228, 221, 244, 140, 127, 213, 31, 107, 101, 137, 41, 104, 241, 132, 136, 49, 52, 217, 201, 250, 102, 190, 138, 41, 187, 229, 20, 71, 5, 150, 235, 20, 76, 187, 19, 167, 67, 201, 20, 54, 59, 86, 129, 127, 9, 250, 228, 120, 211, 230, 221, 143, 133, 108, 157, 38, 215, 201, 210, 209, 68, 111, 68, 167, 162, 33, 106, 29, 191, 217, 223, 119, 120, 34, 245, 182, 92, 49, 217, 125, 230, 220, 161, 37, 166, 54, 104, 175, 169, 226, 49, 25, 255, 84, 90, 186, 92, 28, 146, 224, 97, 49, 18, 218, 143, 49, 18, 109, 123, 189, 232, 214, 86, 198, 73, 227, 167, 32, 200, 44, 83, 241, 174, 173, 250, 37, 31, 65, 126, 190, 85, 163, 33, 118, 31, 101, 225, 203, 65, 237, 248, 193, 152, 18, 52, 33, 156, 60, 86, 193, 73, 237, 75, 247, 213, 135, 96, 212, 148, 216, 156, 179, 185, 22, 186, 239, 193, 58, 218, 194, 181, 122, 29, 211, 57, 17, 147, 178, 71, 101, 196, 177, 111, 9, 190, 82, 218, 245, 225, 93, 206, 253, 81, 54, 67, 189, 65, 153, 136, 74, 144, 167, 242, 217, 237, 58, 208, 3, 54, 222, 88, 98, 159, 181, 25, 162, 113, 194, 230, 255, 219, 103, 12, 208, 229, 95, 5, 209, 75, 16, 65, 47, 106, 27, 229, 105, 87, 45, 247, 106, 91, 18, 206, 39, 181, 18, 210, 54, 238, 40, 4, 203, 228, 23, 236, 197, 100, 238, 194, 162, 125, 177, 44, 210, 14, 25, 125, 220, 84, 42, 195, 5, 234, 33, 20, 68, 138, 113, 70, 98, 228, 171, 123, 0, 238, 252, 59, 92, 144, 70, 85, 37, 76, 123, 44, 94, 42, 247, 109, 173, 118, 198, 180, 207, 81, 228, 152, 156, 92, 74, 231, 49, 60, 80, 207, 173, 73, 193, 125, 249, 166, 46, 211, 219, 116, 129, 224, 224, 38, 42, 244, 107, 254, 194, 244, 102, 221, 121, 122, 207, 227, 204, 196, 207, 107, 47, 204, 193, 76, 230, 79, 238, 102, 31, 87, 151, 37, 245, 70, 51, 34, 237, 108, 217, 118, 245, 67, 78, 41, 243, 169, 131, 113, 29, 88, 105, 73, 38, 16, 124, 62, 17, 114, 3, 35, 17, 233, 197, 61, 45, 116, 161, 70, 52, 125, 26, 35, 50, 145, 97, 154, 75, 92, 53, 222, 34, 232, 105, 150, 207, 177, 45, 223, 13, 157, 160, 193, 41, 82, 84, 52, 210, 67, 135, 25, 125, 45, 68, 82, 54, 172, 116, 53, 211, 147, 185, 177, 203, 207, 195, 35, 156, 214, 229, 39, 114, 205, 4, 101, 156, 92, 39, 239, 223, 66, 164, 61, 107, 61, 9, 81, 179, 33, 203, 199, 21, 92, 18, 205, 72, 250, 182, 81, 84, 217, 5, 92, 123, 37, 233, 79, 125, 116, 214, 145, 146, 198, 77, 237, 19, 114, 223, 209, 214, 37, 123, 71, 62, 189, 146, 250, 44, 81, 74, 50, 36, 159, 1, 175, 201, 236, 73, 128, 2, 235, 30, 63, 233, 216, 213, 148, 96, 225, 58, 206, 105, 190, 16, 20, 246, 20, 53, 211, 73, 244, 251, 119, 164, 121, 53, 223, 228, 77, 235, 181, 108, 209, 238, 29, 207, 199, 105, 93, 41, 106, 125, 213, 228, 95, 172, 102, 143, 148, 97, 43, 156, 38, 99, 198, 95, 188, 112, 100, 176, 75, 0, 66, 150, 155, 160, 255, 103, 238, 237, 69, 101, 237, 246, 78, 234, 54, 57, 238, 26, 164, 165, 117, 170, 6, 157, 128, 94, 18, 38, 84, 9, 17, 244, 0, 41, 235, 230, 69, 157, 71, 206, 19, 51, 169, 247, 227, 221, 107, 222, 156, 167, 118, 159, 84, 73, 248, 117, 213, 91, 62, 149, 66, 213, 44, 101, 86, 79, 248, 92, 243, 173, 188, 54, 187, 233, 69, 115, 134, 119, 85, 251, 70, 126, 237, 92, 211, 252, 76, 247, 104, 86, 41, 34, 55, 27, 182, 17, 43, 177, 86, 163, 97, 218, 82, 198, 72, 167, 195, 1, 34, 68, 167, 27, 206, 33, 187, 161, 84, 250, 165, 42, 133, 59, 195, 169, 12, 158, 137, 95, 61, 100, 218, 80, 44, 90, 135, 113, 99, 244}; +inline constexpr int64_t kLtx2RealDitNvfp4PackedCount = 1024; + +// FP8 `model.diffusion_model.transformer_blocks.0.attn1.to_q` rows [1, 33, 65, 97, 2, 34, 66, 98], elements 0..255, +// per-tensor scale 0.00167083740234375, decoded by torch. +inline constexpr float kLtx2RealDitFp8OracleF32[] = {-0.0016708374F, -0.00751876831F, -0.00626564026F, -0.0033416748F, 0.0016708374F, 0.00501251221F, -0.00459480286F, 0.00156641006F, 0.00459480286F, -0.000939846039F, -0.00501251221F, 0.00584793091F, 0.00751876831F, -0.00626564026F, 0.00417709351F, -0.00459480286F, 0.00375938416F, -0.00187969208F, 0.00459480286F, -0.00626564026F, -0.00543022156F, 0.00229740143F, -0.00501251221F, -0.00501251221F, -0.00135755539F, 0.00156641006F, 0.00229740143F, 0.00459480286F, -0.00156641006F, -0.000195801258F, -0.00208854675F, 0.00104427338F, -0.00114870071F, -0.00918960571F, -0.00271511078F, 0.00271511078F, -0.00417709351F, 0.00104427338F, -0.00751876831F, -0.0025062561F, -0.00459480286F, -0.00835418701F, -0.000574350357F, 0.00501251221F, 0.0025062561F, 0.000678777695F, -0.00187969208F, -0.00292396545F, -0.00584793091F, 0.0033416748F, 0.00584793091F, -0.00501251221F, -0.00626564026F, 0.00417709351F, -0.00292396545F, -0.00417709351F, -0.000208854675F, -0.000939846039F, 0.00626564026F, 0.00208854675F, 0.000339388847F, -0.00271511078F, -0.00187969208F, 0.00459480286F, -0.00104427338F, -0.00146198273F, 0.00271511078F, -0.00023496151F, 0.000783205032F, 0.00751876831F, -0.00543022156F, -0.00229740143F, 0.00114870071F, -0.000783205032F, -0.00156641006F, -0.00187969208F, 0.000626564026F, -0.00125312805F, 0.00125312805F, -0.0033416748F, -0.00375938416F, -0.000391602516F, -0.00668334961F, 0.00271511078F, -0.00459480286F, 0.00751876831F, 0.00104427338F, -0.0025062561F, 0.00626564026F, 0.00626564026F, -0.0033416748F, 0.00156641006F, -0.00229740143F, 0.00375938416F, 0.00584793091F, -0.00313282013F, 0.0025062561F, -0.00208854675F, -0.00417709351F, 0.00292396545F, -0.00459480286F, -0.00584793091F, 0.00023496151F, -0.00146198273F, 0.0033416748F, -0.00501251221F, -0.00229740143F, 0.000626564026F, 0.00584793091F, 0.00104427338F, -0.00751876831F, 0.00668334961F, 0.00751876831F, 0.00417709351F, 0.000574350357F, -0.00208854675F, -0.00292396545F, -0.000678777695F, 0.00543022156F, 0.00271511078F, 0.00668334961F, 0.00584793091F, 0.00271511078F, 0.00375938416F, -0.00543022156F, 0.00668334961F, -0.00187969208F, -0.00751876831F, -0.00543022156F, -0.0033416748F, 0.00668334961F, 0.00208854675F, 0.00146198273F, 0.0033416748F, -0.00584793091F, -0.00229740143F, -0.00271511078F, 0.00584793091F, -0.00835418701F, 0.00584793091F, -0.00375938416F, -0.00292396545F, 0.00751876831F, 0.00918960571F, -0.00584793091F, -0.000626564026F, 0.0016708374F, 0.00835418701F, 0.000574350357F, 0.000169694424F, -0.00501251221F, -0.0016708374F, -0.00313282013F, -0.00501251221F, 0.0100250244F, 0.00114870071F, 9.13739204e-05F, -0.00271511078F, -0.00208854675F, 0.00229740143F, 0.000835418701F, 0.000783205032F, -0.00375938416F, 0.00156641006F, -0.0025062561F, 0.0125312805F, 0.00135755539F, -0.000939846039F, 0.0033416748F, 0.00543022156F, -0.00229740143F, 9.7900629e-05F, 0.0016708374F, 0.0025062561F, 0.000365495682F, 0.00375938416F, 0.0108604431F, -0.00375938416F, 0.00417709351F, 6.5267086e-05F, 0.00501251221F, 0.00375938416F, 0.00835418701F, 0.00114870071F, -9.13739204e-05F, -0.00459480286F, 0.00292396545F, -0.00459480286F, -0.00501251221F, -0.00187969208F, 0.00543022156F, 0.00271511078F, 0.00417709351F, 0.000287175179F, 0.0016708374F, -0.00543022156F, -0.00187969208F, -0.00313282013F, -0.00751876831F, 0.00156641006F, -0.00313282013F, -0.00918960571F, 0.00313282013F, 0.00208854675F, -0.00229740143F, 0.00146198273F, 0.00459480286F, -0.00751876831F, 0.00125312805F, -0.00459480286F, -0.00417709351F, 0.00417709351F, 0.00459480286F, -0.00135755539F, -0.00146198273F, 0.00543022156F, -0.00626564026F, 0.000313282013F, 0.00375938416F, -0.00375938416F, 0.000678777695F, 5.22136688e-05F, -0.00501251221F, -0.00375938416F, 0.00292396545F, 0.00501251221F, 0.00187969208F, -0.00125312805F, -0.00375938416F, 0.000939846039F, -0.00271511078F, 0.000143587589F, -0.0016708374F, -0.00543022156F, -0.00375938416F, 0.0025062561F, -1.30534172e-05F, -0.00918960571F, 0.00417709351F, 0.00417709351F, -0.00417709351F, -0.00417709351F, -0.00271511078F, 0.000469923019F, -0.00375938416F, -0.00501251221F, 0.00459480286F, -0.000835418701F, 0.000678777695F, -0.00626564026F, -0.00501251221F, 0.00114870071F, -0.0033416748F, 0.0100250244F, 0.00417709351F, -0.00751876831F, 0.00271511078F, -0.00271511078F, -0.0100250244F, 0.0033416748F, -0.00417709351F, -0.00459480286F, -0.00459480286F, 0.0033416748F, 0.00543022156F, -0.00135755539F, -0.00271511078F, -0.00229740143F, -0.000678777695F, 0.000835418701F, 0.0033416748F, 0.00668334961F, -0.00543022156F, 0.00292396545F, -0.0100250244F, 0.00375938416F, -0.00375938416F, -0.000574350357F, -0.000156641006F, -0.00375938416F, 0.00584793091F, 0.00584793091F, 0.0016708374F, -0.000208854675F, -0.00918960571F, 7.83205032e-05F, 0.00417709351F, -0.000143587589F, 0.0033416748F, -0.00104427338F, 0.00918960571F, 0.00918960571F, 0.0016708374F, 0.00229740143F, -0.00459480286F, 0.00292396545F, -0.000835418701F, -0.0025062561F, -0.0025062561F, -0.00584793091F, -0.0033416748F, -0.00626564026F, 0.00114870071F, 0.00543022156F, 0.00543022156F, 0.00104427338F, -0.00375938416F, -0.000835418701F, -0.000939846039F, -0.00114870071F, -0.00114870071F, 0.00668334961F, -0.00229740143F, -0.00187969208F, -0.00543022156F, -0.00375938416F, 0.0033416748F, 0.00584793091F, -0.00187969208F, 0.000678777695F, 0.00292396545F, -0.0100250244F, -0.000208854675F, -0.00459480286F, -0.0100250244F, -0.000391602516F, 0.00543022156F, -0.00417709351F, 0.0100250244F, -0.00313282013F, 0.00417709351F, -0.00751876831F, -0.000835418701F, -0.000939846039F, 0.00417709351F, -0.00417709351F, -0.00543022156F, 0.00229740143F, -0.00375938416F, 0.00135755539F, 0.00125312805F, 0.00835418701F, -0.00229740143F, -0.000522136688F, 0.00125312805F, -0.00584793091F, 0.000835418701F, 0.00751876831F, 0.00156641006F, 0.00114870071F, 0.00375938416F, 0.00459480286F, 0.00229740143F, -0.000939846039F, -0.00459480286F, -0.000678777695F, -0.00271511078F, 0.00626564026F, 0.00584793091F, 0.00313282013F, 0.000939846039F, 0.00229740143F, 0.00156641006F, 0.00543022156F, -0.0033416748F, 0.0133666992F, -0.00668334961F, -0.00271511078F, -0.00417709351F, 0.00459480286F, -0.00208854675F, 0.00023496151F, -0.00135755539F, 0.0025062561F, 0.000730991364F, 0.00459480286F, 0.00543022156F, -0.00271511078F, -0.00543022156F, 0.00146198273F, -0.0133666992F, -0.00156641006F, -0.0100250244F, 0.00668334961F, -0.00918960571F, 0.00023496151F, 0.0033416748F, 0.00187969208F, 0.0150375366F, 0.00375938416F, -0.00208854675F, 0.00187969208F, 0.00835418701F, 0.00459480286F, -0.000261068344F, -0.0116958618F, 0.00668334961F, -0.00417709351F, 0.00459480286F, 0.00146198273F, 0.000469923019F, 0.0016708374F, 0.00208854675F, -0.00459480286F, -0.00104427338F, 0.000365495682F, 0.00156641006F, 0.00156641006F, -0.000522136688F, -0.00835418701F, -0.00501251221F, 0.00417709351F, -0.00501251221F, -0.00375938416F, 0.00501251221F, 0.00501251221F, 0.016708374F, 0.00229740143F, 0.0100250244F, 1.63167715e-05F, -0.000117480755F, 0.00835418701F, 0.00156641006F, 0.00292396545F, -0.00271511078F, 0.00918960571F, 0.00229740143F, 0.0033416748F, -0.00135755539F, -0.000783205032F, 0.000939846039F, -0.00229740143F, -0.0033416748F, 5.87403774e-05F, -0.00584793091F, -0.00375938416F, 0.00501251221F, -0.00313282013F, 0.00626564026F, 0.00187969208F, -0.00835418701F, 0.00918960571F, 0.00104427338F, 0.00375938416F, 0.00187969208F, -0.00146198273F, 0.00668334961F, -0.00626564026F, -0.00835418701F, -1.63167715e-05F, -0.00156641006F, -0.00543022156F, 0.000939846039F, 0.00543022156F, -0.00187969208F, -0.00417709351F, -0.00459480286F, -0.00918960571F, 0.00459480286F, -0.000469923019F, 0.00271511078F, -0.00918960571F, 0.00375938416F, -0.0025062561F, 0.00292396545F, -0.00292396545F, 0.00417709351F, -0.00584793091F, 0.00584793091F, 0.00229740143F, -0.00313282013F, -0.0016708374F, 0.00187969208F, 0.0217208862F, -0.00375938416F, 0.00313282013F, 0.00835418701F, -0.00208854675F, 0.00187969208F, -0.00835418701F, 0.000939846039F, -0.00292396545F, 0.00313282013F, 0.00835418701F, -0.000730991364F, -0.00187969208F, 0.000939846039F, 0.00375938416F, 0.000130534172F, -0.00156641006F, -0.00271511078F, 0.00156641006F, 0.00104427338F, 0.000835418701F, 0.00751876831F, 0.00187969208F, 0.000835418701F, -0.000182747841F, 0.00501251221F, 0.00104427338F, 0.00187969208F, -0.00417709351F, 0.00187969208F, -0.0016708374F, -0.00271511078F, 2.28434801e-05F, -0.00459480286F, 0.000522136688F, -0.0033416748F, -0.00751876831F, 0.0033416748F, -0.00835418701F, -0.00751876831F, 0.00668334961F, 0.00271511078F, 0.00751876831F, -0.00375938416F, -0.00501251221F, -0.00543022156F, -0.00626564026F, -0.00417709351F, 0.00668334961F, 0.00668334961F, -0.00104427338F, 0.00271511078F, 0.00668334961F, -0.00626564026F, 0.00375938416F, 0.00626564026F, -0.00375938416F, 0.00543022156F, 0.00375938416F, 0.00751876831F, 0.00417709351F, -0.00668334961F, 0.00459480286F, -0.00271511078F, 0.00417709351F, 0.0033416748F, -0.00751876831F, -0.00417709351F, -0.00156641006F, 0.000522136688F, -0.000730991364F, -0.00135755539F, -0.00543022156F, 0.000835418701F, -0.00417709351F, -0.0108604431F, -0.000156641006F, 0.00751876831F, -0.00501251221F, -0.000469923019F, 0.00271511078F, 0.0016708374F, 1.95801258e-05F, -0.00543022156F, 0.00417709351F, 0.000835418701F, 0.00501251221F, -0.00375938416F, 0.00375938416F, -0.016708374F, -0.00668334961F, -0.00543022156F, -0.00501251221F, -0.00229740143F, 0.0125312805F, 0.00459480286F, 0.00543022156F, -0.0100250244F, 0.00668334961F, 0.00501251221F, 0.0108604431F, 0.00668334961F, 0.00584793091F, 0.00835418701F, 3.91602516e-05F, -0.00271511078F, 0.00543022156F, -0.000835418701F, 0.000783205032F, 0.00229740143F, 0.00668334961F, 0.0033416748F, 0.00208854675F, 0.00501251221F, 0.00187969208F, -0.00417709351F, -0.00751876831F, 0.00501251221F, -0.00835418701F, -0.00417709351F, 0.00626564026F, -0.000313282013F, 0.00543022156F, 0.00835418701F, -0.00292396545F, 0.00313282013F, -0.00751876831F, -0.000143587589F, 0.00313282013F, -0.00187969208F, -0.00208854675F, 0.00918960571F, -0.00187969208F, -0.00626564026F, 0.000939846039F, -0.00918960571F, 0.00313282013F, 0.00417709351F, -0.00375938416F, -0.00375938416F, 0.0033416748F, -0.0033416748F, -0.00584793091F, -0.000939846039F, -0.00271511078F, 0.00459480286F, 0.00135755539F, -0.00626564026F, 0.00292396545F, -0.00501251221F, -0.00501251221F, -0.00835418701F, 0.00375938416F, -0.000169694424F, -0.00375938416F, 0.0100250244F, -0.00918960571F, -0.00543022156F, 0.00543022156F, 0.000678777695F, -0.00835418701F, 0.00626564026F, -0.00292396545F, 0.00626564026F, 0.00375938416F, -0.000522136688F, -0.000626564026F, 0.0025062561F, -0.000939846039F, 0.0125312805F, -0.000626564026F, -0.0200500488F, 0.00187969208F, -0.000522136688F, 0.00584793091F, -0.000208854675F, -0.000469923019F, 0.00501251221F, 0.000730991364F, 0.00417709351F, 0.00751876831F, -0.00918960571F, -0.00187969208F, -0.00271511078F, -0.000626564026F, -0.00918960571F, -0.0016708374F, 0.00626564026F, 0.00626564026F, -0.00208854675F, -0.00626564026F, -9.7900629e-06F, -0.00156641006F, 0.00104427338F, -0.000835418701F, -0.00271511078F, -0.0025062561F, -0.00668334961F, 0.00459480286F, 0.00114870071F, 0.00668334961F, 0.00751876831F, 0.0183792114F, 0.000104427338F, 0.00751876831F, -0.00187969208F, 0.0108604431F, -0.0108604431F, -0.00501251221F, 0.00135755539F, 0.00751876831F, 0.00668334961F, -0.00584793091F, -0.00459480286F, -0.00459480286F, 0.000626564026F, 0.00271511078F, -0.0016708374F, 0.0108604431F, 0.00417709351F, 0.00292396545F, -0.00417709351F, -0.000939846039F, 0.00125312805F, 0.00375938416F, 0.00313282013F, 0.00835418701F, 0.00313282013F, -0.00271511078F, 0.00626564026F, 0.00668334961F, -6.5267086e-05F, 0.000783205032F, 0.00918960571F, -0.00918960571F, 0.000208854675F, -0.000417709351F, -0.00751876831F, -0.00135755539F, -0.00668334961F, 0.00375938416F, 0.00751876831F, 6.5267086e-05F, -0.00459480286F, 0.0025062561F, -0.00668334961F, 0.00543022156F, -0.00459480286F, -0.00375938416F, -0.00023496151F, -0.00668334961F, -0.00023496151F, 0.00459480286F, 0.00626564026F, -0.00292396545F, -0.00114870071F, -0.00375938416F, 0.00125312805F, 0.00543022156F, -0.00417709351F, 0.00835418701F, -0.0025062561F, -0.000169694424F, -0.00626564026F, 0.00229740143F, -0.00584793091F, 0.00125312805F, 0.00375938416F, 0.00375938416F, 0.00835418701F, -0.0108604431F, 0.00459480286F, 0.0033416748F, -0.00146198273F, 0.0116958618F, -0.00156641006F, 0.00208854675F, 0.00229740143F, 0.00104427338F, 0.00543022156F, -0.00114870071F, 0.000626564026F, -0.00375938416F, -0.00208854675F, -0.00751876831F, -0.00375938416F, -0.000469923019F, -0.00375938416F, -0.00751876831F, 0.00751876831F, 0.00626564026F, 0.00668334961F, -0.000469923019F, 0.00114870071F, 0.00125312805F, -0.0108604431F, 0.00501251221F, -0.00229740143F, 0.00751876831F, 0.00501251221F, -0.0033416748F, 0.00271511078F, 0.00104427338F, -0.00584793091F, -0.000783205032F, 0.00918960571F, -0.00584793091F, -0.00751876831F, 0.00835418701F, -0.00501251221F, -0.00375938416F, -0.00146198273F, 0.000939846039F, 0.00125312805F, 0.00417709351F, -0.00146198273F, 0.00584793091F, 0.00208854675F, 0.00543022156F, 0.00584793091F, -8.48472118e-05F, -0.0016708374F, -0.00918960571F, -0.00156641006F, -0.000939846039F, -0.00584793091F, 0.00125312805F, 0.00271511078F, 0.000835418701F, 0.000939846039F, -0.00135755539F, -0.00918960571F, -0.0108604431F, 0.00543022156F, 0.00375938416F, 0.00626564026F, -0.0016708374F, -0.00271511078F, -0.00135755539F, 0.00543022156F, -0.00375938416F, 0.000939846039F, -0.00417709351F, -0.000835418701F, 0.00114870071F, -0.00626564026F, 0.00023496151F, 0.00543022156F, 0.000835418701F, 0.00187969208F, 0.00104427338F, 0.000574350357F, 0.00146198273F, -0.00459480286F, -0.00146198273F, -0.000287175179F, -0.00835418701F, 0.00208854675F, 0.000730991364F, 0.000522136688F, 0.000939846039F, 0.00501251221F, -0.00313282013F, 0.00668334961F, -0.00187969208F, -0.00156641006F, -0.00313282013F, -0.00501251221F, -5.87403774e-05F, -0.00313282013F, 0.00375938416F, 0.0033416748F, 0.00501251221F, -0.00208854675F, 0.00459480286F, 0.00417709351F, -0.000678777695F, -0.00835418701F, 0.00313282013F, -0.0016708374F, 0.0108604431F, 0.00229740143F, 0.000365495682F, -0.00375938416F, -0.000287175179F, 0.0025062561F, -0.00417709351F, 0.00584793091F, 0.0033416748F, -0.0125312805F, 0.00146198273F, -0.000783205032F, -0.00459480286F, -0.00114870071F, -0.00271511078F, -0.00501251221F, -0.000939846039F, 0.000835418701F, 0.00271511078F, 0.000339388847F, -0.00501251221F, 0.00156641006F, 0.000417709351F, 0.00501251221F, -0.00375938416F, -0.00146198273F, -0.00313282013F, 0.0033416748F, 0.00292396545F, -0.00114870071F, 0.00156641006F, 0.00125312805F, 0.000835418701F, 0.00668334961F, 0.00459480286F, 0.000730991364F, -0.00668334961F, 0.00459480286F, 0.00271511078F, -0.00543022156F, 0.000391602516F, -0.0025062561F, -0.0016708374F, 0.00543022156F, 0.00114870071F, -0.00918960571F, -0.00543022156F, -0.0133666992F, -0.000469923019F, -0.00459480286F, 0.000730991364F, -0.000835418701F, -0.000574350357F, 0.000783205032F, 0.00271511078F, 0.00375938416F, 0.00459480286F, 0.00229740143F, 0.000835418701F, -0.000469923019F, -0.00375938416F, 0.00229740143F, -0.0033416748F, 0.00459480286F, 0.00835418701F, -0.00375938416F, 0.000574350357F, 0.0033416748F, -0.00114870071F, -0.00584793091F, -0.00501251221F, 0.0033416748F, -0.00271511078F, -0.00835418701F, 0.0100250244F, -0.00543022156F, 0.00501251221F, -0.000261068344F, 0.0116958618F, 0.00835418701F, 0.000261068344F, -0.00501251221F, 0.00459480286F, -0.00125312805F, 0.00375938416F, -0.00501251221F, 0.00023496151F, -0.00125312805F, -0.00375938416F, -0.00292396545F, 0.00313282013F, -0.00229740143F, -0.000939846039F, 0.000574350357F, 0.00751876831F, -0.00229740143F, -0.00125312805F, -0.00626564026F, -0.0100250244F, -0.00375938416F, 0.000469923019F, 0.00229740143F, -0.00146198273F, -0.00459480286F, -0.00135755539F, -0.00375938416F, 0.00187969208F, -0.00271511078F, 0.00543022156F, 0.0116958618F, -0.00187969208F, 0.000574350357F, -0.00501251221F, -0.00543022156F, 0.0025062561F, 0.0033416748F, -0.00104427338F, 0.00125312805F, 0.00104427338F, -0.000626564026F, 0.00271511078F, -0.0033416748F, 0.00187969208F, 0.00271511078F, 0.00668334961F, 0.00584793091F, 0.00375938416F, -0.00313282013F, 0.0025062561F, -0.00208854675F, 0.000939846039F, 0.00543022156F, -0.00751876831F, 0.000417709351F, -0.000835418701F, -0.00229740143F, -0.00584793091F, 0.00459480286F, 0.00187969208F, -0.00375938416F, -0.00125312805F, -0.00668334961F, 0.0033416748F, -0.00543022156F, 0.000287175179F, 0.00375938416F, -0.00292396545F, -0.0025062561F, -0.00501251221F, -0.00584793091F, -0.00375938416F, 0.00375938416F, 0.000678777695F, 0.0016708374F, 0.00459480286F, 0.00501251221F, 0.00459480286F, -0.00459480286F, -0.00751876831F, 0.00543022156F, 0.000626564026F, -0.00104427338F, -0.00146198273F, -0.000469923019F, -0.000469923019F, 0.00626564026F, -0.0033416748F, -0.000939846039F, 0.000391602516F, -0.0025062561F, 0.016708374F, -0.0108604431F, 0.00187969208F, -0.00501251221F, -0.00125312805F, -0.00626564026F, -0.00543022156F, 0.0025062561F, -0.00229740143F, -0.00459480286F, 0.000169694424F, 3.2633543e-06F, 0.00292396545F, 0.00271511078F, 0.00751876831F, -0.00459480286F, -0.00835418701F, 0.00375938416F, -0.000130534172F, 0.00584793091F, 0.00104427338F, -0.00114870071F, -0.00751876831F, -0.00208854675F, 0.00417709351F, 0.000678777695F, -0.000522136688F, -0.00114870071F, 0.00114870071F, 0.00668334961F, 0.000835418701F, -0.00271511078F, 0.00114870071F, -0.0033416748F, 0.00375938416F, -0.00584793091F, -0.00501251221F, -0.0033416748F, -0.00135755539F, 0.00459480286F, 0.00501251221F, 0.000117480755F, -0.00187969208F, -0.00271511078F, 0.000130534172F, -0.0033416748F, 0.00313282013F, 0.00292396545F, -0.00543022156F, 0.000156641006F, 0.00501251221F, -0.00417709351F, 0.000939846039F, 0.00375938416F, -0.00292396545F, 0.00104427338F, 0.000208854675F, 0.00375938416F, 0.000574350357F, 0.00187969208F, -0.0100250244F, 0.00668334961F, -0.00146198273F, 0.00114870071F, -0.0025062561F, -0.00459480286F, 0.0033416748F, 0.00626564026F, -0.000678777695F, 0.0033416748F, 0.00751876831F, 0.000939846039F, -0.0033416748F, -0.00668334961F, 0.00626564026F, 0.00584793091F, -0.00114870071F, 0.00375938416F, -0.00187969208F, 0.000391602516F, 0.000939846039F, -0.00208854675F, -0.00417709351F, 0.00104427338F, 0.00626564026F, -0.00156641006F, 0.0033416748F, 0.000469923019F, 0.000939846039F, -0.00292396545F, 0.00125312805F, 0.00125312805F, 0.00417709351F, -0.00501251221F, -0.00375938416F, 0.00104427338F, -0.000130534172F, 4.56869602e-05F, 0.0025062561F, -0.00229740143F, -0.00125312805F, -0.00375938416F, 0.0025062561F, 0.00023496151F, 0.00626564026F, -0.00417709351F, -0.00229740143F, -0.00229740143F, 0.00292396545F, 0.00417709351F, 0.0025062561F, -0.00417709351F, -0.00417709351F, -0.0033416748F, 0.00146198273F, -0.00146198273F, 0.00187969208F, 0.0100250244F, -0.00417709351F, -0.00229740143F, 0.000783205032F, 0.0016708374F, -0.0033416748F, -0.00543022156F, 0.00156641006F, 0.000208854675F, -0.00668334961F, 0.000835418701F, -0.000730991364F, 0.0108604431F, -0.00459480286F, -0.00626564026F, -0.00375938416F, 0.00459480286F, 0.0033416748F, 0.00208854675F, -0.00187969208F, -0.00375938416F, 0.00292396545F, 5.87403774e-05F, 0.00501251221F, -0.00751876831F, -0.00187969208F, 0.00208854675F, 0.000574350357F, 0.00023496151F, 0.00208854675F, 0.00187969208F, 0.00187969208F, -4.56869602e-05F, -0.000835418701F, 0.00584793091F, 0.000522136688F, 0.00229740143F, 0.00501251221F, 0.00417709351F, 0.000939846039F, -0.00375938416F, 0.00187969208F, -0.00114870071F, -0.00125312805F, 0.0100250244F, -0.00229740143F, 4.24236059e-05F, 9.7900629e-05F, -0.00668334961F, -0.00668334961F, -0.00918960571F, -0.00375938416F, 0.00229740143F, -0.00292396545F, 0.00459480286F, -0.00375938416F, -0.000783205032F, 0.000287175179F, 0.00417709351F, 0.00626564026F, 0.00501251221F, 0.00501251221F, 0.00156641006F, 0.00417709351F, 0.0033416748F, -0.00375938416F, 0.00918960571F, -0.00208854675F, 0.00187969208F, -0.00292396545F, 0.00501251221F, -0.00668334961F, 0.00135755539F, -0.00156641006F, -0.00835418701F, 0.00751876831F, 0.00459480286F, -0.00271511078F, -0.00114870071F, -0.00292396545F, 0.00543022156F, 0.00501251221F, -0.00208854675F, 0.00417709351F, -0.00187969208F, 0.00229740143F, -0.00229740143F, -0.00584793091F, 0.0033416748F, 0.000574350357F, -0.00459480286F, 0.00229740143F, -0.000626564026F, -0.000339388847F, -0.000835418701F, -0.00375938416F, 0.00543022156F, -0.00459480286F, 0.00375938416F, -0.00208854675F, -0.00751876831F, 0.0125312805F, 0.00271511078F, 0.00114870071F, 0.0033416748F, -0.00459480286F, 0.00501251221F, -4.89503145e-05F, -0.00208854675F, -0.00668334961F, -0.00104427338F, -0.0033416748F, 0.0025062561F, -0.000339388847F, -0.00271511078F, 0.000469923019F, 0.00626564026F, -0.00313282013F, -0.00668334961F, -0.000287175179F, -0.00104427338F, 0.00417709351F, 0.000939846039F, -0.00543022156F, -0.00375938416F, 0.00208854675F, -0.00459480286F, -0.0016708374F, 0.00835418701F, 0.00292396545F, -7.17937946e-05F, 0.000626564026F, -0.00543022156F, -9.13739204e-05F, -0.00626564026F, 0.000261068344F, 0.00125312805F, 0.00584793091F, 0.00156641006F, -0.000730991364F, -0.00626564026F, 0.00271511078F, 0.00417709351F, -0.0016708374F, -0.00543022156F, -0.00459480286F, -0.00271511078F, 0.000939846039F, -0.00668334961F, 0.0025062561F, 0.00543022156F, 0.00417709351F, -0.00271511078F, -0.0033416748F, 0.00626564026F, -0.000469923019F, 0.00584793091F, -0.000939846039F, -0.0025062561F, -0.00751876831F, -0.00417709351F, 0.0016708374F, -0.00229740143F, -0.00229740143F, -0.00229740143F, 0.00208854675F, -0.00208854675F, -0.00584793091F, 0.00543022156F, -0.00187969208F, 0.00146198273F, -0.00668334961F, -0.00271511078F, -0.00229740143F, -0.00271511078F, 0.000939846039F, 0.00271511078F, -0.00229740143F, -0.00459480286F, 0.00501251221F, 0.0025062561F, -0.00626564026F, -0.00543022156F, -0.00543022156F, 0.00459480286F, 0.00459480286F, 0.000783205032F, -0.00543022156F, 0.00459480286F, 0.00835418701F, -0.000522136688F, 0.00668334961F, 0.00156641006F, 0.00459480286F, -0.00751876831F, 0.00459480286F, 0.00292396545F, 0.00543022156F, 0.00187969208F, 0.0016708374F, 0.00146198273F, 0.00156641006F, -0.00543022156F, -0.00375938416F, 0.00668334961F, -0.00313282013F, -0.00375938416F, -0.000522136688F, 0.00751876831F, 0.00543022156F, -0.00751876831F, 0.00417709351F, 0.00292396545F, 0.00208854675F, 0.00292396545F, -0.00501251221F, 0.000939846039F, -0.000783205032F, -0.00835418701F, 0.00229740143F, -0.00208854675F, -0.00135755539F, -0.000261068344F, 0.00313282013F, 0.0116958618F, 0.000678777695F, 0.00135755539F, -0.00668334961F, 0.00501251221F, -0.000182747841F, 0.00751876831F, -0.000939846039F, 0.00375938416F, -0.000783205032F, 0.0016708374F, 0.00751876831F, 0.000730991364F, 8.48472118e-05F, 0.0133666992F, -0.00417709351F, 0.00271511078F, -0.0100250244F, 0.0016708374F, 0.00125312805F, 0.0116958618F, 0.00187969208F, 0.000365495682F, 0.00375938416F, 0.00271511078F, 0.00313282013F, 0.00125312805F, 0.000678777695F, -0.00584793091F, -0.000939846039F, -0.00271511078F, 0.0033416748F, 0.00187969208F, -0.00417709351F, 0.00146198273F, -0.0033416748F, 0.00835418701F, 0.0025062561F, -0.00146198273F, 0.000678777695F, 0.00271511078F, 0.00501251221F, 0.0016708374F, 0.00271511078F, 0.0183792114F, -0.00751876831F, 0.0016708374F, -0.00375938416F, 0.00375938416F, 0.00375938416F, 0.00375938416F, 0.00292396545F, -0.00187969208F, 0.000939846039F, 0.0116958618F, 0.00135755539F, -0.000835418701F, -0.0033416748F, 0.00668334961F, -0.00375938416F, 0.00292396545F, -0.00271511078F, 0.00187969208F, 0.00292396545F, -0.0033416748F, -0.00417709351F, 0.00135755539F, 0.00114870071F, -0.00501251221F, -0.000156641006F, 0.00543022156F, -0.000574350357F, -0.000469923019F, 0.00501251221F, -0.00271511078F, -0.00668334961F, -0.0033416748F, 0.00146198273F, 0.00208854675F, -0.00751876831F, -0.00668334961F, -0.0125312805F, 0.000261068344F, -0.00626564026F, -0.000730991364F, -0.00751876831F, -0.0016708374F, 0.000365495682F, -0.00417709351F, 0.000783205032F, 0.00208854675F, -0.00114870071F, 0.00417709351F, 0.0016708374F, 0.00375938416F, 0.00292396545F, 0.00229740143F, 0.00271511078F, -0.00375938416F, 0.00156641006F, 0.00313282013F, 0.00187969208F, -0.000287175179F, 0.00835418701F, 0.000417709351F, -0.000574350357F, 0.00835418701F, -0.00417709351F, 0.00135755539F, -0.00459480286F, 0.0025062561F, 0.00313282013F, 0.00417709351F, 0.0016708374F, 0.00208854675F, 0.00501251221F, -0.00156641006F, -0.00313282013F, 0.00835418701F, 0.0025062561F, 0.00208854675F, 0.00375938416F, -0.00375938416F, 0.00229740143F, -0.0025062561F, 0.00501251221F, -0.00543022156F, -0.000939846039F, -0.00501251221F, 0.00156641006F, -0.00751876831F, -0.00584793091F, -0.00668334961F, -0.0200500488F, -0.00751876831F, 0.00584793091F, 0.00313282013F, 0.00626564026F, -0.00156641006F, -0.00751876831F, -0.00459480286F, 0.00501251221F, -0.00543022156F, 0.00459480286F, 0.00114870071F, 0.00751876831F, 0.00835418701F, 0.00146198273F, -0.00271511078F, -0.00459480286F, 7.17937946e-05F, 0.00292396545F, 0.00668334961F, 0.00417709351F, -0.000678777695F, -0.0033416748F, 0.00543022156F, -0.0033416748F, 0.0016708374F, 0.00918960571F, -0.00626564026F, -0.00918960571F, -0.00375938416F, -0.00835418701F, 0.00271511078F, 0.00125312805F, -0.0016708374F, 0.00375938416F, 0.00208854675F, -0.00417709351F, 0.00668334961F, -0.00292396545F, 0.00292396545F, -0.00751876831F, 0.000417709351F, -0.00135755539F, 0.00751876831F, 0.000835418701F, -0.00313282013F, 0.00375938416F, 0.00271511078F, 0.000678777695F, -0.00501251221F, -0.00271511078F, -0.00459480286F, 0.0133666992F, 0.00156641006F, 0.00668334961F, 0.00543022156F, -0.00375938416F, 0.000626564026F, 0.00125312805F, -0.00292396545F, -0.00501251221F, 0.0033416748F, -0.000208854675F, -0.00918960571F, -0.00156641006F, -0.00208854675F, 0.00668334961F, 0.00417709351F, 0.000574350357F, 0.00501251221F, 0.00313282013F, -0.00187969208F, -0.00023496151F, 0.000104427338F, 0.0025062561F, 0.00208854675F, -0.00104427338F, 0.00417709351F, -0.00104427338F, 0.000626564026F, 0.00229740143F, -0.0033416748F, -0.000261068344F, 0.00135755539F, -0.00459480286F, 0.00459480286F, -0.00208854675F, 0.00918960571F, 0.00292396545F, -0.00375938416F, 0.00668334961F, -0.00104427338F, 0.000522136688F, -0.000574350357F, 0.0016708374F, -0.00292396545F, 0.00584793091F, 0.0033416748F, -0.00375938416F, -0.00626564026F, -0.00501251221F, 0.000730991364F, 0.00208854675F, 0.00543022156F, 0.000939846039F, -0.0025062561F, -0.00584793091F, -0.0033416748F, 0.000626564026F, -0.0025062561F, 0.00417709351F, 0.00114870071F, 0.00292396545F, 0.00584793091F, -0.00156641006F, 0.00208854675F, 0.00543022156F, 0.00146198273F, -0.0033416748F, -0.00292396545F, -0.00668334961F, -0.000730991364F, 0.0025062561F, -0.0125312805F, -0.00229740143F, 0.00187969208F, -0.00417709351F, 0.00584793091F, 0.000939846039F, 0.00417709351F, -0.0016708374F, 0.0033416748F, 0.00208854675F, 0.0025062561F, 0.00375938416F, -0.000522136688F, -0.00918960571F, 0.000261068344F, 0.000835418701F, -0.00156641006F, -0.00835418701F, -0.00313282013F, -0.00104427338F, -0.00584793091F, -0.00313282013F, 0.00292396545F, -0.000835418701F, 3.58968973e-05F, 9.7900629e-05F, 9.7900629e-05F, 0.00135755539F, -0.00459480286F, -0.00135755539F, 0.000574350357F, -0.00459480286F, 0.00187969208F, -0.00626564026F, -0.00459480286F, -0.000261068344F, -0.00375938416F, -0.000117480755F, -0.00271511078F, 0.00292396545F, -0.000626564026F, 0.00229740143F, 0.00417709351F, 0.000156641006F, -0.00626564026F, 0.000939846039F, 0.00292396545F, -0.00187969208F, -0.00417709351F, -0.00835418701F, 0.00918960571F, -0.000939846039F, -0.00292396545F, -0.00751876831F, 0.000678777695F, 0.000469923019F, 0.000939846039F, 0.00375938416F, -0.0100250244F, 0.00751876831F, 0.000626564026F, 0.00292396545F, 0.00229740143F, 0.00417709351F, -0.00375938416F, 0.0016708374F, 0.0025062561F, -0.000391602516F, -0.00751876831F, 0.00292396545F, -0.00751876831F, -0.00187969208F, 0.00751876831F, 0.00835418701F, -0.0016708374F, 0.0025062561F, 0.0108604431F, -0.00104427338F, 0.00292396545F, 0.00584793091F, -0.00459480286F, -0.0108604431F, -0.00751876831F, 0.00459480286F, 0.00375938416F, -0.00501251221F, -0.00626564026F, -0.0025062561F, -0.00313282013F, 0.00584793091F, 0.00668334961F, -0.00417709351F, -0.00417709351F, 0.000469923019F, -0.00626564026F, -0.00543022156F, 0.000730991364F, -0.00313282013F, -0.00292396545F, -0.00751876831F, -0.0025062561F, 0.00751876831F, 0.00626564026F, -0.000261068344F, 0.0033416748F, -0.00417709351F, 0.0016708374F, -0.000678777695F, 0.00668334961F, -0.00125312805F, 0.00751876831F, -0.00459480286F, -0.00375938416F, 0.00375938416F, -0.00501251221F, 0.0033416748F, 0.00417709351F, -0.00751876831F, -0.00114870071F, -0.0025062561F, 0.00668334961F, 0.00626564026F, -0.000469923019F, -0.00835418701F, -0.000626564026F, 0.0033416748F, 0.00543022156F, 0.000835418701F, 0.0016708374F, -0.0025062561F, -0.000339388847F, -0.00292396545F, 0.00187969208F, 0.00584793091F, 0.00668334961F, 0.00229740143F, -0.00313282013F, 0.00751876831F, 0.00459480286F, -0.00918960571F, -0.0025062561F, -0.0033416748F, 0.0100250244F, 0.000117480755F, 0.00668334961F, 0.00271511078F, -0.0025062561F, 0.000104427338F, 0.00375938416F, -0.00271511078F, 0.000365495682F, 0.00023496151F, 0.00417709351F, 0.00187969208F, -0.00125312805F, 0.00626564026F, -0.000783205032F, -0.00208854675F, -0.0016708374F, 0.000156641006F, -0.00918960571F, -0.00751876831F, 0.00668334961F, 0.00668334961F, -0.00543022156F, -0.00584793091F, -0.00584793091F, -0.00459480286F, 0.00313282013F, 0.00375938416F, 0.00584793091F, 0.00459480286F, -0.00668334961F, -0.00459480286F, -0.00835418701F, 0.00584793091F, 0.00313282013F, -0.00626564026F, -0.00501251221F, -0.00156641006F, 0.0025062561F, 0.00543022156F, 0.0016708374F, -0.000835418701F, -0.00543022156F, -0.00584793091F, 0.000939846039F, -0.00104427338F, -0.00187969208F, 0.0033416748F, -0.00135755539F, 0.00417709351F, 0.00918960571F, 0.00459480286F, -0.00146198273F, -0.00146198273F, 0.000339388847F, 0.00543022156F, -0.000939846039F, -0.00459480286F, -0.000156641006F, 0.000365495682F, 0.00459480286F, -0.00584793091F, 0.000626564026F, 0.0016708374F, 0.0016708374F, 0.00918960571F, 0.00668334961F, 0.00459480286F, 0.000287175179F, -0.00104427338F, 0.00135755539F, 0.000939846039F, -0.0125312805F, 0.00459480286F, 0.000469923019F, 0.000182747841F, 0.00208854675F, -0.00114870071F, -0.00584793091F, -0.00229740143F, -0.00584793091F, 0.00668334961F, 0.00313282013F, 0.0033416748F, -0.000835418701F, -0.00375938416F, 0.00292396545F, 0.00835418701F, -0.0025062561F, -0.00668334961F, 0.000365495682F, 0.00229740143F, 0.00229740143F, 0.00208854675F, -0.00104427338F, -0.000730991364F, -0.00668334961F, 0.00584793091F, -0.00501251221F, 0.0016708374F, -0.00375938416F, -0.00375938416F, 0.00417709351F, -0.00187969208F, -0.00271511078F, -0.00459480286F, -0.000417709351F, -0.00271511078F, -0.00125312805F, 0.00626564026F, 0.00626564026F, 0.00417709351F, -0.000313282013F, -0.00501251221F, 0.00313282013F, 0.0025062561F, 0.00187969208F, -0.000365495682F, -0.00543022156F, 1.63167715e-05F, 0.00543022156F, 0.0025062561F, -0.0033416748F, 0.0025062561F, 0.00271511078F, -0.0016708374F, 0.00271511078F, -0.00751876831F, -0.000678777695F, 0.00459480286F, 0.00375938416F, 0.00208854675F, -0.00501251221F, 0.00543022156F, 0.00146198273F, -0.00375938416F, 0.00835418701F, 0.00459480286F, 0.00584793091F, 0.00626564026F, 0.00375938416F, -0.0108604431F, -0.00668334961F, -8.48472118e-05F, 0.0033416748F, -0.0025062561F, -0.00751876831F, 0.0025062561F, -0.00125312805F, -0.00417709351F, -0.0016708374F, -0.00292396545F, 0.00156641006F, 0.00584793091F, -0.00187969208F, -0.00187969208F, -0.00584793091F, -0.000678777695F, 0.00271511078F, 0.00417709351F, 0.00751876831F, 0.00208854675F, -0.000195801258F, 0.00584793091F, 0.00751876831F, 0.00668334961F, 0.00417709351F, 0.0033416748F, -0.00668334961F, -0.00208854675F, 0.00229740143F, 0.00626564026F, 0.00751876831F, -0.00543022156F, -0.00584793091F, -0.00459480286F, 0.00459480286F, -0.00292396545F, -0.00417709351F, 0.0025062561F, -0.00835418701F, -0.0033416748F, 0.00271511078F, -0.00271511078F, -0.00835418701F, 0.00835418701F, 0.00668334961F, -0.00023496151F, 0.00417709351F, 0.00668334961F, 0.0025062561F, -0.000939846039F, 0.00313282013F, 0.00208854675F, 0.00313282013F, 0.0150375366F, 0.00156641006F, -0.00417709351F, -0.00375938416F, 0.0116958618F, 0.00187969208F, 0.00156641006F, 0.00229740143F, -0.0033416748F, -0.0033416748F, 0.00135755539F, 0.00501251221F, 0.00543022156F, -0.00146198273F, 0.0025062561F, 0.00626564026F, 0.000678777695F, -0.00375938416F, -0.00187969208F, 0.00459480286F, 0.00156641006F, -0.00375938416F, 0.00543022156F, 0.00292396545F, -0.000313282013F, -0.00146198273F, 0.00918960571F, -0.00375938416F, 0.00292396545F, 0.000208854675F, 0.00125312805F, 0.00135755539F, 0.00156641006F, 0.0033416748F, 0.00313282013F, -0.00187969208F, 0.0100250244F, 0.000939846039F, -0.00229740143F, -0.0033416748F, -0.00668334961F, 0.00187969208F, 0.000939846039F, -0.00292396545F, -0.00271511078F, -0.00208854675F, 0.000574350357F, 0.00626564026F, 0.0033416748F, -0.0108604431F, -0.00146198273F, -0.00156641006F, 0.00584793091F, 0.0016708374F, -0.00146198273F, -0.000287175179F, 0.00543022156F, 0.0025062561F, -0.00271511078F, -0.00417709351F, 0.00229740143F, -0.00187969208F, -0.000835418701F, 0.000469923019F, -0.0033416748F, -0.00104427338F, -0.00751876831F, -9.13739204e-05F, -0.000522136688F, 0.00501251221F, -0.0100250244F, 0.00208854675F, -0.00417709351F, 0.00626564026F, 0.00417709351F, -0.0033416748F, -0.00135755539F, 0.00501251221F, -1.95801258e-05F, 0.00114870071F, -0.00292396545F, 0.00375938416F, -0.00146198273F, -4.56869602e-05F, 0.00835418701F, 0.00751876831F, 0.000939846039F, 0.00584793091F, 0.00229740143F, -0.00751876831F, 0.00292396545F}; +inline constexpr int64_t kLtx2RealDitFp8OracleF32Count = 2048; + +// FP8 `model.diffusion_model.transformer_blocks.0.attn1.to_k` rows [1, 33, 65, 97, 2, 34, 66, 98], elements 0..255, +// per-tensor scale 0.0029296875, decoded by torch. +inline constexpr float kLtx2RealDitFp8ControlF32[] = {-0.00128173828F, 0.00164794922F, 0.00805664062F, -0.000549316406F, 0.00476074219F, -0.000503540039F, 0.00659179688F, 0.00366210938F, 0.000640869141F, 0.00732421875F, 0.0087890625F, -0.005859375F, -0.00183105469F, -0.00146484375F, 0.00183105469F, 0.00805664062F, 0.00732421875F, -0.00164794922F, 0.00201416016F, 0.000411987305F, -0.000274658203F, 0.00119018555F, -0.000457763672F, 0.00402832031F, 0.0102539062F, 0.00366210938F, 0.0102539062F, -0.0205078125F, 0.00366210938F, 0.00952148438F, 0.00274658203F, -0.0102539062F, 0.000732421875F, -0.00952148438F, -0.005859375F, 0.00128173828F, -0.000549316406F, -0.00201416016F, 0.00012588501F, 0.00512695312F, -0.00329589844F, -0.00732421875F, 0.0146484375F, 0.00119018555F, -0.005859375F, -0.000411987305F, 0.00137329102F, -0.00256347656F, 0.00201416016F, -0.0131835938F, 0.00164794922F, 0.0146484375F, 0.00512695312F, -0.00119018555F, 0.00032043457F, -0.000457763672F, -0.00402832031F, -0.00805664062F, 0.00274658203F, -0.00439453125F, -0.00439453125F, -0.00366210938F, 0.00256347656F, 0.00439453125F, -0.00329589844F, -0.005859375F, 0.00805664062F, -0.0131835938F, -0.000915527344F, -0.00439453125F, -0.00659179688F, 0.0087890625F, -0.00366210938F, -0.000823974609F, 0.00128173828F, -0.00329589844F, -0.0131835938F, -0.00128173828F, -0.00549316406F, -0.000457763672F, 0.0029296875F, 0.00128173828F, -0.0029296875F, 0.0146484375F, -0.00329589844F, 0.00476074219F, 0.00512695312F, 0.00732421875F, -0.00366210938F, -0.00512695312F, -0.0161132812F, -0.000732421875F, -0.005859375F, -0.0029296875F, 0.005859375F, 0.00100708008F, -0.00512695312F, 0.0087890625F, 0.00732421875F, -0.00366210938F, -0.000686645508F, -0.0087890625F, 0.00439453125F, -0.00274658203F, 0.00439453125F, 0.00805664062F, 0.01171875F, 0.00366210938F, 0.00329589844F, 0.000503540039F, -0.000640869141F, -0.00402832031F, -0.00366210938F, 0.000549316406F, -0.00274658203F, 0.00329589844F, -0.00805664062F, -0.00201416016F, -0.00476074219F, -0.00659179688F, 0.000640869141F, -0.00476074219F, 0.00512695312F, -0.00952148438F, 0.00659179688F, 0.00274658203F, -0.00439453125F, -0.00256347656F, -0.01171875F, -0.0146484375F, -0.00659179688F, -0.0131835938F, -0.005859375F, -0.0131835938F, -0.0146484375F, 0.00366210938F, -0.000915527344F, 0.00549316406F, -0.00732421875F, 0.00329589844F, -0.00476074219F, -0.00366210938F, -0.00659179688F, -0.00366210938F, -0.00512695312F, 0.00659179688F, -0.0029296875F, -0.0146484375F, -0.00274658203F, -0.00439453125F, -0.00439453125F, -0.0131835938F, 0.00164794922F, -0.0146484375F, 0.00512695312F, 0.00256347656F, -0.0102539062F, -0.01171875F, -0.00402832031F, -0.0029296875F, -0.00549316406F, 0.00274658203F, 0.00805664062F, 0.00146484375F, -0.00805664062F, -0.00146484375F, -0.00512695312F, -0.00549316406F, 0.000732421875F, -0.00476074219F, 0.00366210938F, 0.000183105469F, 0.00329589844F, 0.00512695312F, 0.00952148438F, -0.00659179688F, 0.00732421875F, 0.00549316406F, 0.00146484375F, 0.00439453125F, 0.00012588501F, 0.005859375F, 0.00128173828F, -0.00549316406F, -0.000686645508F, -0.00476074219F, 0.01171875F, 0.0131835938F, -0.00732421875F, -0.00146484375F, -0.000915527344F, -0.00512695312F, 0.005859375F, -0.005859375F, 0.0146484375F, -0.0102539062F, -0.00183105469F, 0.00549316406F, 0.00952148438F, -0.00805664062F, -0.00952148438F, -0.0102539062F, 0.0102539062F, -0.00032043457F, 0.00012588501F, -0.00201416016F, 0.000137329102F, -0.00952148438F, -0.000640869141F, 0.00238037109F, 0.005859375F, -0.00274658203F, -0.000205993652F, 0.00109863281F, -0.0087890625F, -0.00476074219F, 0.000823974609F, -0.00329589844F, -0.00476074219F, 0.0109863281F, -0.005859375F, -0.0234375F, 0.00100708008F, 0.00439453125F, 0.00128173828F, -0.00439453125F, 0.005859375F, 0.00109863281F, -0.00329589844F, -0.00329589844F, -0.00659179688F, -0.00512695312F, 0.00439453125F, 0.000366210938F, 0.0146484375F, -0.00549316406F, -0.000732421875F, 0.00366210938F, -0.00732421875F, -0.00732421875F, -0.005859375F, 0.000228881836F, 0.005859375F, -0.000686645508F, -0.00659179688F, 0.000503540039F, 0.00238037109F, 0.00256347656F, -0.00549316406F, 0.00146484375F, -0.00329589844F, 0.00329589844F, 0.00146484375F, 0.017578125F, -0.00366210938F, 0.0087890625F, -0.00952148438F, 0.00100708008F, 0.00329589844F, 0.00201416016F, 0.00238037109F, -0.005859375F, -0.00549316406F, 0.00439453125F, -0.0029296875F, 0.00952148438F, -0.000366210938F, 0.000228881836F, 0.00439453125F, 0.00329589844F, -0.00201416016F, 0.0029296875F, -0.00238037109F, 0.00137329102F, -0.00439453125F, 0.000503540039F, -0.0161132812F, 0.00238037109F, -0.00439453125F, -0.00366210938F, -0.00012588501F, 0.00512695312F, 0.00476074219F, -0.017578125F, 0.00366210938F, 0.00219726562F, -0.00256347656F, -0.00659179688F, -0.0146484375F, -0.000205993652F, 0.0029296875F, 0.00439453125F, 0.00439453125F, 0.00805664062F, 0.00025177002F, 0.00732421875F, -0.00274658203F, -0.00219726562F, 0.00329589844F, -0.000640869141F, -0.00549316406F, 0.00512695312F, -0.0029296875F, 0.00805664062F, -0.00183105469F, 0.00659179688F, 0.00476074219F, 0.00549316406F, 0.00659179688F, -0.0102539062F, -0.0102539062F, -0.01171875F, -0.00512695312F, -0.000640869141F, 0.00164794922F, -0.00512695312F, -0.00439453125F, 0.0102539062F, -0.000205993652F, 0.0087890625F, -0.000228881836F, -0.00805664062F, 0.00659179688F, 0.000732421875F, -0.00805664062F, -0.00549316406F, -0.00402832031F, 0.00439453125F, 0.00366210938F, -0.00366210938F, -0.00732421875F, 0.00549316406F, -0.00805664062F, 0.00137329102F, -0.000274658203F, -0.0087890625F, 0.0087890625F, 0.00439453125F, -0.00659179688F, 0.00137329102F, 0.00256347656F, -0.00109863281F, -0.00201416016F, 0.00805664062F, -0.00238037109F, -0.00329589844F, 0.00256347656F, 0.000228881836F, -0.000274658203F, -0.00512695312F, -0.00732421875F, 0.00366210938F, -0.000205993652F, 0.005859375F, 0.00952148438F, 0.000205993652F, -0.0102539062F, -0.00512695312F, 0.00366210938F, -0.00805664062F, 0.0131835938F, 0.0087890625F, 0.01171875F, 0.00549316406F, 0.00476074219F, -0.000549316406F, -0.00732421875F, 0.00732421875F, -0.00805664062F, -0.00109863281F, 0.0029296875F, 0.00274658203F, -0.00366210938F, 0.00025177002F, 0.00119018555F, -0.00476074219F, -0.00659179688F, -0.00329589844F, 0.00256347656F, -0.01171875F, -0.0146484375F, -0.0087890625F, -0.0219726562F, -0.00732421875F, -0.00476074219F, -0.0102539062F, 0.0146484375F, -0.00402832031F, 0.00274658203F, -0.00201416016F, -0.0146484375F, 0.00100708008F, 0.00402832031F, -0.0161132812F, 0.01171875F, -0.0029296875F, -0.0087890625F, 0.00952148438F, -0.00219726562F, -0.000503540039F, 0.000102996826F, 0.00201416016F, -0.00732421875F, -0.000823974609F, -0.00476074219F, 0.000411987305F, 0.005859375F, -0.00476074219F, -0.00659179688F, -0.00659179688F, 0.005859375F, -0.000732421875F, 0.000366210938F, -0.005859375F, 0.00952148438F, 0.00012588501F, -0.00659179688F, 0.00366210938F, 0.00805664062F, -0.00219726562F, -0.01171875F, 0.0234375F, -0.00201416016F, -0.0029296875F, 0.00256347656F, 0.00183105469F, -0.00137329102F, 0.00402832031F, 0.00732421875F, 0.000549316406F, 0.00256347656F, 0.00146484375F, -0.00219726562F, 0.0087890625F, -0.005859375F, -0.00549316406F, 0.00238037109F, 0.00439453125F, 0.00329589844F, -0.017578125F, -0.00137329102F, -0.000183105469F, 0.0087890625F, 0.00512695312F, 0.00274658203F, 0.0087890625F, 0.00805664062F, 0.0087890625F, 0.00137329102F, 0.00952148438F, 0.03515625F, -0.0131835938F, 0.00366210938F, 0.00274658203F, 0.000274658203F, -0.00952148438F, -0.00402832031F, -0.000915527344F, 0.00402832031F, 0.0087890625F, 0.00256347656F, 0.0109863281F, -0.0029296875F, -0.0146484375F, -0.0102539062F, -0.005859375F, -0.00952148438F, -0.00256347656F, -0.00549316406F, 0.00164794922F, 0.005859375F, 0.00402832031F, 0.00952148438F, -0.01171875F, -0.000183105469F, 0.000411987305F, -0.00659179688F, -0.00549316406F, -0.00549316406F, 0.00183105469F, -0.0029296875F, -0.00329589844F, 0.00329589844F, 0.00329589844F, 3.43322754e-05F, 0.00805664062F, 0.00109863281F, 0.005859375F, 0.00732421875F, 0.005859375F, 0.0146484375F, -0.00549316406F, -0.00439453125F, -0.00146484375F, -0.000171661377F, 0.00329589844F, -0.0102539062F, -0.005859375F, -0.00549316406F, -0.00549316406F, 0.000915527344F, -0.00402832031F, 0.00146484375F, -0.000503540039F, -0.000915527344F, -0.00329589844F, -0.00100708008F, -0.00146484375F, -0.00439453125F, -0.00219726562F, 0.00732421875F, -0.005859375F, 0.01171875F, -0.00402832031F, 0.00402832031F, -0.005859375F, 0.0131835938F, 0.0102539062F, -0.000503540039F, 8.01086426e-05F, 0.0102539062F, 0.0131835938F, -0.0029296875F, -0.00219726562F, 0.00549316406F, 0.000915527344F, -0.00805664062F, -0.0146484375F, 0.0109863281F, 0.0190429688F, 0.00119018555F, -0.00439453125F, 0.00732421875F, -0.00219726562F, -0.0102539062F, 0.00119018555F, 0.00164794922F, -0.0131835938F, -0.00128173828F, -0.0219726562F, 0.005859375F, 0.00476074219F, 0.00238037109F, -0.00732421875F, -0.00137329102F, -0.00183105469F, -0.00256347656F, 0.000823974609F, 0.0131835938F, -0.0029296875F, -0.0102539062F, -0.00402832031F, -0.00476074219F, -0.00119018555F, -0.00402832031F, -0.00366210938F, -0.00183105469F, 0.000595092773F, -0.0029296875F, -0.0109863281F, -0.00439453125F, 0.00256347656F, 0.0109863281F, 0.0029296875F, 0.00366210938F, -0.005859375F, 0.0029296875F, 0.00805664062F, -0.00183105469F, -0.0131835938F, -0.00256347656F, 0.00476074219F, 0.0029296875F, -0.00183105469F, 0.000549316406F, -0.005859375F, -0.00439453125F, 0.00732421875F, 0.00329589844F, 0.00146484375F, -0.00366210938F, 0.00439453125F, -0.0029296875F, -0.00952148438F, -0.00476074219F, -0.00732421875F, -0.00659179688F, 0.01171875F, -0.00952148438F, -0.000503540039F, -0.00439453125F, 0.0146484375F, -0.00659179688F, 0.00805664062F, 0.00100708008F, -0.00119018555F, -6.29425049e-05F, -0.00329589844F, -0.00476074219F, 0.000457763672F, -0.00476074219F, -0.000823974609F, 0.00659179688F, 0.0131835938F, -6.86645508e-05F, 0.00366210938F, 0.00659179688F, -0.005859375F, -0.00659179688F, 0.00512695312F, 0.00732421875F, 0.005859375F, 0.0102539062F, -0.00805664062F, 0.0029296875F, 0.0102539062F, 0.0161132812F, 0.000366210938F, 0.00549316406F, -0.00512695312F, 0.00329589844F, -0.0146484375F, -0.00952148438F, 0.00805664062F, 0.00659179688F, -0.000915527344F, 0.00549316406F, 0.00732421875F, 0.00201416016F, 0.00164794922F, 0.00512695312F, 0.0087890625F, -0.00439453125F, 0.000457763672F, 0.00146484375F, -0.00805664062F, -0.00549316406F, -0.00732421875F, 0.00732421875F, -0.00805664062F, -0.00659179688F, -0.000228881836F, 0.0087890625F, 0.00219726562F, -0.00329589844F, 0.00256347656F, -0.005859375F, 0.0109863281F, 0.00238037109F, -0.0109863281F, 0.01171875F, 0.000503540039F, -9.15527344e-05F, 0.00366210938F, -0.00238037109F, -0.00805664062F, 0.00732421875F, -0.0087890625F, -0.00366210938F, -0.000148773193F, -0.00439453125F, 0.00146484375F, -0.0109863281F, -0.00659179688F, 0.0146484375F, 0.0146484375F, 0.0146484375F, -0.00439453125F, -0.00201416016F, 0.00805664062F, -0.0146484375F, -8.01086426e-05F, 0.00119018555F, 0.00128173828F, -0.00659179688F, -0.00952148438F, -0.00512695312F, 0.0234375F, 0.0146484375F, 0.00329589844F, -0.00032043457F, -0.00146484375F, -0.005859375F, -0.0087890625F, -0.00732421875F, -0.00659179688F, -0.000595092773F, 0.00659179688F, 0.00659179688F, -0.0205078125F, 0.00238037109F, -0.00659179688F, -0.0029296875F, 0.005859375F, -0.00732421875F, -0.01171875F, 0.0102539062F, 0.00439453125F, 0.00732421875F, -0.00238037109F, 0.000457763672F, 0.0131835938F, -0.00238037109F, -0.00805664062F, 0.00732421875F, 0.00659179688F, 0.00805664062F, 0.000457763672F, -0.0087890625F, -0.00402832031F, 0.00512695312F, -0.00476074219F, 0.00732421875F, 0.00439453125F, 0.00952148438F, -0.000595092773F, 0.00659179688F, 0.00952148438F, 0.00219726562F, 0.0102539062F, -0.00128173828F, -0.00476074219F, -0.00476074219F, -0.00128173828F, -0.00274658203F, -0.017578125F, -0.00439453125F, 0.00952148438F, -0.00659179688F, -0.0263671875F, -0.0029296875F, 0.00201416016F, 0.0102539062F, -0.00659179688F, 0.01171875F, 0.00439453125F, -0.0102539062F, -0.00659179688F, 0.00274658203F, -0.00402832031F, -0.00164794922F, -0.00256347656F, -0.005859375F, -0.00128173828F, 0.00402832031F, -0.00439453125F, -0.0029296875F, 0.00732421875F, 0.0131835938F, 0.00219726562F, -0.00732421875F, -0.005859375F, -0.00659179688F, -0.00366210938F, 0.00476074219F, 0.00439453125F, 0.00439453125F, -0.00512695312F, 0.00952148438F, -0.00476074219F, 0.00366210938F, -0.00201416016F, -0.0029296875F, 0.00952148438F, 0.00659179688F, -0.000503540039F, 0.00805664062F, 0.000183105469F, 0.0161132812F, -0.00256347656F, -0.00128173828F, -0.00476074219F, -0.000205993652F, -0.000915527344F, -0.000732421875F, 0.00128173828F, -0.00128173828F, 0.01171875F, -0.000205993652F, 0.00732421875F, 0.00732421875F, 0.00366210938F, -0.0109863281F, 4.57763672e-05F, 0.00805664062F, -0.005859375F, 0.00439453125F, -0.00732421875F, -0.00476074219F, -0.00732421875F, -3.43322754e-05F, 0.00329589844F, 0.0109863281F, -0.00238037109F, 0.00402832031F, -0.01171875F, 0.00146484375F, 0.01171875F, -0.0087890625F, 0.00238037109F, -0.005859375F, -0.00256347656F, 0.005859375F, 0.005859375F, 0.000732421875F, 0.0146484375F, -0.00659179688F, -0.00952148438F, -0.00329589844F, -0.0102539062F, -0.005859375F, -0.000915527344F, 0.0102539062F, 0.00146484375F, -0.0029296875F, -0.00476074219F, -0.00100708008F, 0.00128173828F, 0.00274658203F, 0.0205078125F, -0.000411987305F, 0.00164794922F, 0.00439453125F, 0.00512695312F, 0.00366210938F, 0.000457763672F, 0.000732421875F, -0.00100708008F, 0.00476074219F, -0.00025177002F, -0.0102539062F, -0.005859375F, -0.00183105469F, 0.00137329102F, -0.00329589844F, -0.00201416016F, -0.00366210938F, 0.00238037109F, -0.00402832031F, 0.00805664062F, -0.00659179688F, -0.00366210938F, -0.00183105469F, 0.00100708008F, 0.0029296875F, -0.000823974609F, -0.00805664062F, 0.00476074219F, 0.00238037109F, -0.00805664062F, 0.00952148438F, 0.00476074219F, -0.000411987305F, -0.005859375F, 0.00329589844F, 0.00146484375F, -0.00732421875F, 0.005859375F, -0.00128173828F, 0.00128173828F, -0.00402832031F, 0.000137329102F, 0.000823974609F, -0.00549316406F, 0.00732421875F, 0.00512695312F, -0.00659179688F, 0.0102539062F, 0.000205993652F, -0.00183105469F, -0.00439453125F, 0.00256347656F, 0.00732421875F, -0.00805664062F, -0.00659179688F, -0.00329589844F, -0.00439453125F, -0.00366210938F, -0.0087890625F, 0.00402832031F, -0.00439453125F, -0.005859375F, 0.00512695312F, 0.00238037109F, -0.00256347656F, 0.0102539062F, 0.00732421875F, 0.0146484375F, 0.00366210938F, 0.005859375F, -0.0109863281F, -0.00549316406F, -0.00952148438F, 0.00439453125F, 0.0219726562F, -0.0131835938F, 0.000915527344F, 0.00183105469F, 0.00256347656F, 0.0131835938F, -0.00238037109F, 0.000549316406F, -0.00549316406F, 0.0219726562F, -0.00128173828F, -0.01171875F, 0.005859375F, -0.00366210938F, -0.00137329102F, -0.0029296875F, -0.01171875F, -0.005859375F, 0.00659179688F, -0.00366210938F, 0.00238037109F, 0.0102539062F, -0.00512695312F, -0.00146484375F, 0.00805664062F, -0.00329589844F, 0.00329589844F, 0.00952148438F, 0.00805664062F, -1.71661377e-05F, 0.005859375F, 0.01171875F, -0.00732421875F, -0.00659179688F, 0.0029296875F, 0.00274658203F, -0.01171875F, -0.00732421875F, 0.01171875F, 0.0087890625F, -0.000366210938F, -0.00012588501F, -0.00137329102F, -0.00476074219F, 0.00512695312F, 0.00439453125F, -0.00476074219F, -0.00219726562F, 0.0087890625F, -0.00100708008F, 0.0161132812F, 0.000915527344F, 0.00274658203F, -0.00439453125F, 0.00128173828F, 0.00439453125F, 0.005859375F, -0.00402832031F, 0.0161132812F, 0.0029296875F, -0.00238037109F, 0.00183105469F, -0.000503540039F, 0.005859375F, 0.00137329102F, -0.000343322754F, 0.0146484375F, -0.00732421875F, 0.0146484375F, -0.0029296875F, 0.000457763672F, -0.00402832031F, 0.00952148438F, -0.00732421875F, 0.000732421875F, 0.0146484375F, 0.01171875F, -0.01171875F, -0.00164794922F, 0.000823974609F, -0.00366210938F, 0.00659179688F, 0.01171875F, -0.00201416016F, -0.00402832031F, -0.00128173828F, 0.01171875F, 0.00183105469F, -0.01171875F, -0.005859375F, 0.00274658203F, -0.017578125F, 0.005859375F, -0.00659179688F, 0.0087890625F, -0.0131835938F, 0.00659179688F, 0.00146484375F, -0.0087890625F, 0.00732421875F, -0.00476074219F, -0.00164794922F, -0.00952148438F, 0.000297546387F, -0.00476074219F, 0.00164794922F, -0.00366210938F, -0.00952148438F, -0.00805664062F, 0.00128173828F, 0.00256347656F, -0.00476074219F, -0.017578125F, -0.00659179688F, 0.005859375F, 0.0087890625F, 0.00549316406F, -0.00329589844F, -0.00659179688F, 0.00025177002F, 0.00952148438F, -0.00512695312F, 0.00164794922F, -0.005859375F, -0.00238037109F, -0.00732421875F, -0.00256347656F, -0.00329589844F, 0.01171875F, 0.00732421875F, 0.00366210938F, 0.00274658203F, -0.00549316406F, -0.0029296875F, 0.00549316406F, -0.00439453125F, 0.00439453125F, -0.00952148438F, 0.00329589844F, 0.005859375F, 0.0146484375F, -0.000640869141F, -0.0029296875F, -0.0146484375F, -0.000640869141F, -0.00952148438F, 0.00805664062F, 0.00274658203F, -0.005859375F, -0.000503540039F, -0.0190429688F, -0.005859375F, 0.00183105469F, -0.00732421875F, 0.00805664062F, -0.0102539062F, 0.0087890625F, 0.0263671875F, 0.0087890625F, 0.0146484375F, -0.00366210938F, 0.00183105469F, -0.00100708008F, -0.00329589844F, 0.00137329102F, -0.00201416016F, -0.0131835938F, 0.00439453125F, -0.0205078125F, 0.00439453125F, 0.00164794922F, 0.000183105469F, -0.00329589844F, -0.0029296875F, 0.01171875F, -0.000366210938F, 0.000915527344F, -0.00256347656F, 0.00329589844F, -0.00512695312F, 0.00659179688F, 0.00366210938F, 0.00439453125F, -0.00366210938F, -0.0102539062F, 0.00402832031F, -0.0029296875F, -0.00256347656F, -0.0087890625F, -0.000823974609F, -0.00549316406F, -0.00219726562F, 0.00329589844F, 0.00439453125F, -0.000915527344F, 0.00109863281F, -0.0102539062F, -0.00366210938F, -0.000732421875F, 0.00439453125F, 0.0102539062F, 0.00238037109F, -0.00659179688F, 0.00201416016F, -0.00100708008F, 0.0029296875F, 0.00659179688F, -0.00805664062F, -0.000411987305F, -0.00164794922F, 0.0131835938F, -0.0161132812F, 0.00402832031F, 0.00732421875F, 0.00109863281F, -0.0029296875F, -0.00329589844F, 0.00952148438F, 0.00549316406F, -0.000457763672F, -0.00732421875F, -0.00402832031F, 0.00274658203F, -0.00549316406F, -0.00439453125F, -0.000823974609F, 0.00183105469F, 0.00256347656F, 0.0146484375F, -0.00476074219F, 0.00439453125F, 0.00476074219F, 0.00659179688F, -0.000411987305F, -0.00201416016F, -0.00805664062F, 0.00256347656F, -0.00659179688F, -0.00732421875F, -0.00659179688F, 0.00238037109F, -0.0102539062F, -0.0131835938F, 0.00256347656F, -0.00137329102F, 0.00549316406F, -0.00238037109F, -0.00183105469F, 0.00952148438F, 0.00512695312F, -0.0029296875F, -0.000915527344F, 0.000366210938F, -0.00164794922F, -0.00183105469F, -0.00476074219F, 0.005859375F, 0.005859375F, -0.00032043457F, -0.0102539062F, -4.00543213e-05F, 0.00183105469F, -0.0087890625F, -0.00329589844F, -0.0205078125F, 0.0146484375F, -0.000503540039F, -0.00329589844F, -0.00476074219F, 0.00274658203F, 0.00183105469F, 0.00366210938F, -0.00201416016F, -0.0087890625F, 0.0109863281F, 0.01171875F, 0.000274658203F, -0.01171875F, -0.00659179688F, 0.00366210938F, 0.00952148438F, -0.0029296875F, 0.00512695312F, 0.00549316406F, 0.0131835938F, -0.005859375F, -0.00659179688F, 0.00146484375F, 0.0029296875F, 0.000640869141F, -0.00805664062F, -0.00805664062F, 0.00732421875F, -0.00274658203F, 0.00512695312F, -0.0190429688F, -0.0109863281F, -0.00732421875F, 0.00219726562F, 0.000640869141F, 0.0029296875F, -0.000915527344F, -0.0109863281F, -0.00137329102F, 0.0131835938F, 0.0087890625F, -0.005859375F, -0.00201416016F, -0.000549316406F, -0.00146484375F, 0.00183105469F, -0.00274658203F, -0.0029296875F, 0.00659179688F, 0.0146484375F, -0.00549316406F, 0.00164794922F, -0.00183105469F, 0.00549316406F, -0.000160217285F, 0.0087890625F, 0.0087890625F, -0.00402832031F, -0.00476074219F, -0.00402832031F, 0.00164794922F, 0.00952148438F, -0.005859375F, 0.00439453125F, -0.00439453125F, 0.00402832031F, -0.0146484375F, 0.00238037109F, -0.00476074219F, -0.00274658203F, -0.000595092773F, -0.005859375F, -0.00805664062F, 0.0109863281F, -0.00032043457F, -0.00219726562F, -0.00219726562F, 0.00100708008F, -0.00109863281F, 0.00329589844F, -0.00109863281F, -0.00659179688F, 0.00329589844F, -0.00183105469F, 0.00732421875F, -0.00659179688F, 0.0109863281F, 0.000366210938F, -0.00201416016F, 0.00164794922F, 0.00512695312F, 0.00732421875F, -0.000915527344F, -0.000160217285F, 0.000366210938F, -0.00219726562F, -0.005859375F, 0.0029296875F, -0.00402832031F, -0.00549316406F, 0.00732421875F, 0.00952148438F, 0.00219726562F, 0.0029296875F, -0.0131835938F, -0.00119018555F, 0.0087890625F, -0.00402832031F, -0.00659179688F, -0.00146484375F, 0.00805664062F, 0.00402832031F, 0.00549316406F, -0.0146484375F, 0.00952148438F, -0.000686645508F, -0.00146484375F, 0.0029296875F, -0.00952148438F, -0.00805664062F, -0.00549316406F, 0.00274658203F, -0.00476074219F, 0.00439453125F, -0.00238037109F, 0.00366210938F, 0.00805664062F, 0.000732421875F, -0.00329589844F, -0.00402832031F, -0.00439453125F, -0.00512695312F, -0.00164794922F, -0.00439453125F, -0.01171875F, -0.00146484375F, -0.00183105469F, -0.00366210938F, -0.00164794922F, -0.0146484375F, 0.00201416016F, -0.000595092773F, 0.00659179688F, 0.00238037109F, 0.000732421875F, -0.00329589844F, 0.0087890625F, -0.00366210938F, -0.0190429688F, -0.00219726562F, -0.005859375F, 0.000640869141F, -0.00256347656F, -0.00025177002F, -0.00201416016F, -0.000823974609F, 0.00659179688F, 0.005859375F, 0.00366210938F, 0.00128173828F, 0.00146484375F, -0.00732421875F, 0.00032043457F, 0.00805664062F, 0.00100708008F, -0.0087890625F, 0.00805664062F, -0.00032043457F, 0.00659179688F, -0.000640869141F, -0.00366210938F, 0.00274658203F, -0.005859375F, 0.00805664062F, 0.000732421875F, -0.00659179688F, -0.00659179688F, -0.00476074219F, -0.00183105469F, -0.0029296875F, -0.00805664062F, 0.00201416016F, 0.0102539062F, 0.00732421875F, -0.000686645508F, 0.00732421875F, -0.00732421875F, 0.00219726562F, 0.00201416016F, -0.000160217285F, 0.00219726562F, -0.00805664062F, -0.00805664062F, 0.00100708008F, -0.00952148438F, -0.00146484375F, -0.00201416016F, 0.00119018555F, 0.00366210938F, -0.0161132812F, 0.00128173828F, -0.00366210938F, 0.005859375F, -0.00183105469F, -0.000205993652F, 0.00512695312F, -0.00549316406F, -0.00805664062F, -0.00366210938F, 0.000503540039F, -0.0087890625F, -0.00659179688F, 0.0087890625F, 0.00219726562F, 0.000823974609F, -0.00402832031F, 0.000915527344F, 0.00146484375F, -0.00439453125F, 0.00476074219F, -0.00329589844F, -0.00274658203F, -0.00402832031F, 0.00109863281F, 0.00659179688F, 0.00100708008F, 0.000343322754F, 0.00549316406F, 0.00329589844F, -0.0102539062F, -0.000160217285F, 0.00329589844F, 0.000411987305F, 0.00183105469F, -0.00164794922F, -0.00549316406F, -0.00439453125F, 0.00439453125F, -0.00476074219F, -0.00329589844F, -0.01171875F, 0.00659179688F, 0.00402832031F, 0.0087890625F, -0.0102539062F, -0.00512695312F, 0.00732421875F, -0.000915527344F, -0.00732421875F, 0.00183105469F, 0.00032043457F, -0.00119018555F, -0.0131835938F, -0.00183105469F, -0.00512695312F, -0.00164794922F, -0.0109863281F, 0.005859375F, -0.017578125F, 0.00164794922F, 0.00219726562F, -0.00476074219F, -0.00274658203F, -0.00659179688F, 0.00732421875F, -0.00439453125F, -0.00732421875F, 0.00659179688F, 0.00329589844F, 0.0029296875F, -0.0087890625F, -0.00512695312F, -0.00256347656F, -0.005859375F, 0.0029296875F, -0.00402832031F, -0.005859375F, -0.00512695312F, -0.01171875F, 0.00732421875F, 0.00238037109F, 0.00805664062F, 0.00219726562F, -0.00952148438F, 0.00256347656F, -0.000411987305F, 0.0205078125F, -0.00952148438F, -0.00659179688F, 0.00366210938F, -0.00952148438F, 0.00476074219F, -0.0029296875F, -0.00366210938F, 0.0029296875F, 0.000549316406F, 0.00164794922F, 0.00439453125F, -0.0087890625F, -0.00025177002F, -0.00329589844F, -0.00128173828F, 0.000915527344F, -0.00402832031F, -0.01171875F, -0.0087890625F, -0.00805664062F, 0.00329589844F, -0.005859375F, 0.00119018555F, 0.00183105469F, -0.00164794922F, -0.0131835938F, 0.00402832031F, -0.00183105469F, 0.0087890625F, 0.00219726562F, -0.00100708008F, 6.29425049e-05F, -8.01086426e-05F, -0.0161132812F, -0.00805664062F, -0.01171875F, 0.00329589844F, 0.00402832031F, 0.00512695312F, -0.00219726562F, 0.000823974609F, -0.01171875F, -0.005859375F, 0.00274658203F, 0.00109863281F, -0.00366210938F, 0.00146484375F, -0.00439453125F, 0.00439453125F, 0.00805664062F, 0.01171875F, -0.0109863281F, -0.005859375F, -0.00805664062F, -0.00256347656F, -0.00219726562F, -0.00439453125F, -0.00439453125F, 0.00256347656F, -0.00402832031F, -0.00164794922F, 0.005859375F, 0.01171875F, 0.00659179688F, 0.00439453125F, -0.00146484375F, 0.00659179688F, 0.00659179688F, 0.00659179688F, -0.00952148438F, -0.00238037109F, 0.00100708008F, 0.00805664062F, 0.00137329102F, 0.00256347656F, 0.00201416016F, 0.00219726562F, 0.00219726562F, -0.00476074219F, -0.00549316406F, 0.00476074219F, -0.00329589844F, 0.005859375F, -0.00659179688F, -0.00952148438F, 0.01171875F, -0.00109863281F, 0.00659179688F, 0.0146484375F, 0.0109863281F, -0.00238037109F, 0.00512695312F, -5.14984131e-05F, -0.00219726562F, -0.000915527344F, 0.00109863281F, -0.00439453125F, 0.00329589844F, 0.00512695312F, 0.00109863281F, -0.00183105469F, 0.00366210938F, 0.00274658203F, 0.000823974609F, 0.00952148438F, -0.000915527344F, 0.0087890625F, 0.00402832031F, 0.005859375F, 0.00329589844F, 0.00659179688F, 0.00549316406F, -0.00402832031F, -0.000297546387F, 0.00274658203F, 0.0109863281F, -0.00201416016F, -0.0102539062F, 0.00100708008F, -0.01171875F, 0.00128173828F, 0.000732421875F, -0.00512695312F, -0.00512695312F, -0.00659179688F, -0.00183105469F, -5.7220459e-05F, 0.017578125F, 0.00164794922F, 0.00732421875F, 0.0029296875F, -0.00032043457F, -0.0102539062F, 0.005859375F, 0.00256347656F, -0.000915527344F, 0.00659179688F, 0.00659179688F, -0.00256347656F, 0.00659179688F, -0.000640869141F, -0.00109863281F, 0.00439453125F, -0.01171875F, -0.000366210938F, -0.00238037109F, 0.00402832031F, -0.000343322754F, -0.00329589844F, 0.00329589844F, 0.005859375F, 0.005859375F, -0.000595092773F, -0.005859375F, 0.005859375F, -0.00183105469F, 0.000686645508F, 0.00164794922F, -0.000686645508F, -0.00732421875F, -0.0102539062F, 0.00805664062F, 0.0087890625F, -0.00952148438F, 0.0029296875F, -0.000411987305F, 0.00805664062F, -0.00439453125F, 0.0131835938F, 0.00256347656F, -0.00402832031F, 0.00476074219F, -0.00256347656F, 0.000137329102F, -0.0146484375F, 0.00952148438F, 0.00329589844F, -0.00164794922F, 0.00100708008F, -0.00109863281F, 0.0102539062F, -0.00183105469F, 0.00659179688F, 0.00100708008F, -0.000171661377F, -0.000503540039F, -0.00329589844F, -0.00274658203F, 0.00183105469F, -4.00543213e-05F, -0.00146484375F, 0.005859375F, 0.00512695312F, 0.00805664062F, 0.00439453125F, 0.00256347656F, -0.00659179688F, -0.00183105469F, -0.00274658203F, 0.00183105469F, 0.0087890625F, 0.00732421875F, -0.000171661377F, 0.00366210938F, -0.00476074219F, -0.0029296875F, -0.00109863281F, 0.00219726562F, 0.00549316406F, 0.0146484375F, -0.000823974609F, 0.00732421875F, 0.00366210938F, 0.00201416016F, -0.00952148438F, 0.00201416016F, 0.00146484375F, -0.0087890625F, -0.0029296875F, 0.00805664062F, 0.00659179688F, -0.0029296875F, -0.0102539062F, -0.000915527344F, -0.000503540039F, -0.00439453125F, 0.00512695312F, 0.00439453125F, -0.00732421875F, -0.00805664062F, 0.00219726562F, 0.00146484375F, -0.00805664062F, 0.00659179688F, 0.00256347656F, -0.00952148438F, 0.00659179688F, 0.0109863281F, 0.00366210938F, 0.00805664062F, -0.00952148438F, 0.00128173828F, -0.00439453125F, 0.00402832031F, -2.28881836e-05F, -0.00805664062F, 0.0190429688F, 0.0146484375F, -0.00659179688F, 0.0102539062F, -0.00732421875F, 0.00732421875F, -0.00402832031F, 0.00201416016F, 0.00329589844F, 0.0131835938F, -0.0029296875F, -0.00119018555F, 0.005859375F, 0.00549316406F, 0.00439453125F, 0.00732421875F, -0.0131835938F, 0.00183105469F, -0.0087890625F, 0.00329589844F, 0.00659179688F, -0.0087890625F, 0.000595092773F, -0.00164794922F, -0.00512695312F, 0.000595092773F, 0.00659179688F, -0.00128173828F, -0.0131835938F, -0.00100708008F, -0.005859375F, -0.00805664062F, -0.00256347656F, -0.00439453125F, 0.00476074219F, 0.00100708008F, 0.000366210938F, 0.00732421875F, -0.0087890625F, -0.000549316406F, 0.00402832031F, 0.005859375F, 5.14984131e-05F, -0.0102539062F, -0.00732421875F, 0.00952148438F, -0.00805664062F, 0.00549316406F, -0.00476074219F, 0.0102539062F, -0.00659179688F, -0.00219726562F, 0.00805664062F, 0.00805664062F, 0.00109863281F, 0.0029296875F, 0.00164794922F, 0.01171875F, -0.0087890625F, 0.00549316406F, -0.00549316406F, 0.00439453125F, 0.00659179688F, -0.0234375F, 0.00439453125F, 0.017578125F, 0.00402832031F, -0.0131835938F, -0.005859375F, -0.000732421875F, 0.00256347656F, 0.00238037109F, -0.00659179688F, 0.0190429688F, -0.00402832031F, -0.00366210938F, 0.000595092773F, 0.005859375F, -0.00366210938F, -0.000823974609F, -0.00137329102F, -0.000915527344F, 0.0087890625F, 0.00366210938F, -0.00201416016F, -0.00119018555F, 0.00659179688F, -0.00201416016F, 0.00549316406F, -0.00183105469F, 0.00366210938F, 0.00659179688F, 0.00732421875F, -0.0087890625F, -0.00329589844F, -0.00183105469F, 0.00219726562F, 0.00256347656F, 0.00329589844F, -0.00146484375F, 0.00238037109F, 0.00137329102F, 0.00274658203F, 0.00329589844F, -0.00952148438F, -0.00659179688F, 0.00402832031F, -0.00183105469F, 0.00439453125F, -0.005859375F, 0.00274658203F, -0.00476074219F, -0.00201416016F, 0.0131835938F, 0.00183105469F, 0.00366210938F, 0.00476074219F, 0.00732421875F, 0.0029296875F, 1.14440918e-05F, -0.00366210938F, -0.00659179688F, 0.00366210938F, 0.00476074219F, 0.000274658203F, -0.0087890625F, -0.00329589844F, 0.0161132812F, -0.00366210938F, 0.00109863281F, -0.00549316406F, 0.00256347656F, 0.0219726562F, 0.0029296875F, -0.0029296875F, -0.000823974609F, 0.00164794922F, -0.00732421875F, -0.00659179688F, 0.0131835938F, 0.00439453125F, -0.00659179688F, 0.00219726562F, 0.00805664062F, 0.00402832031F, 0.000411987305F, 0.00183105469F, -0.00549316406F, 0.0029296875F, 0.005859375F, -0.00238037109F, -0.0029296875F, -0.00025177002F, -0.00183105469F, 0.00512695312F, 0.00366210938F, 0.005859375F, -0.00164794922F, -0.000274658203F, 0.00439453125F, -0.0109863281F, -0.0161132812F, -0.00366210938F, -0.0029296875F, 0.00146484375F, 0.00549316406F, -0.00146484375F, 0.00109863281F, 0.00439453125F, -0.00402832031F, 0.00238037109F, -0.0102539062F, -0.00219726562F, 0.00100708008F, -0.00952148438F, 0.00109863281F, -0.00219726562F, -0.00439453125F, -0.000823974609F, -0.000549316406F, 0.00219726562F, -0.00219726562F, -0.00732421875F, 0.0029296875F, -0.005859375F, -0.0029296875F, 0.00659179688F, -0.000823974609F, 0.005859375F, -0.00219726562F, -0.00164794922F, -0.00128173828F, 0.00439453125F, 0.000823974609F, 0.00952148438F, 0.00329589844F, -0.00183105469F, 0.00366210938F, -0.00109863281F, 0.0109863281F, 0.00512695312F, 0.00119018555F, -0.00183105469F, 4.00543213e-05F, 0.000411987305F, -0.00164794922F, 0.00201416016F, 0.00366210938F, 0.00659179688F, -0.0087890625F, -0.00732421875F, 0.01171875F, 0.0146484375F, 0.00183105469F, 0.00439453125F, -0.005859375F, 0.0102539062F, -0.00256347656F, 0.00329589844F, 0.00366210938F, -0.00274658203F, -0.00128173828F, -0.00183105469F, 0.00183105469F, -0.00549316406F, -0.0131835938F, 0.0131835938F, 0.00512695312F, 0.00512695312F, -0.00549316406F, -0.00439453125F, -0.00329589844F, -0.00274658203F, 0.00732421875F, 0.00146484375F, 0.0109863281F, -0.00732421875F, 0.0102539062F, -0.00366210938F, -0.00164794922F, 0.000102996826F, -0.00137329102F, 0.000205993652F, 0.00732421875F, 0.01171875F, 0.01171875F, -0.00238037109F, -0.00512695312F, -0.000823974609F, 0.000823974609F, 0.00366210938F, 4.00543213e-05F, -0.00219726562F, 0.005859375F, 0.0131835938F, 0.00146484375F, -0.005859375F, 0.00201416016F, 0.00366210938F, 0.00732421875F, 0.000411987305F, -0.00183105469F, 0.00183105469F, 0.00201416016F, 0.00128173828F, -0.0263671875F, -0.00274658203F, -0.00402832031F, 0.00137329102F, 0.000549316406F, -0.00183105469F, -0.00238037109F, -0.00183105469F, 0.00366210938F, -0.00402832031F, -0.000366210938F, -0.00366210938F, 0.00732421875F, 0.00219726562F, 0.000183105469F, 0.00329589844F, -0.00366210938F, 0.00201416016F, 0.00549316406F, 0.00512695312F, -0.00256347656F, 0.000732421875F, -0.00659179688F, 8.58306885e-05F, -0.00032043457F, -0.005859375F, 0.0234375F, -0.00659179688F, -0.000549316406F, 0.00164794922F, 0.00137329102F, 0.00659179688F, -0.01171875F, 0.00238037109F, -0.0029296875F, 0.00659179688F, -0.00164794922F, -0.00512695312F, -0.00439453125F, -0.00805664062F, 0.00146484375F, -0.00512695312F, 0.00109863281F, 0.00183105469F, -0.00329589844F, -0.00549316406F, 0.00219726562F, 0.00366210938F, 0.000732421875F, 0.00146484375F, 0.00476074219F, 0.00164794922F, 0.00549316406F, 0.00274658203F, 0.00274658203F, -0.00659179688F, -0.00164794922F, 0.005859375F, 0.005859375F, -0.005859375F, 0.00659179688F, 0.00476074219F, -0.00439453125F, -0.000640869141F, 0.00476074219F, -0.00512695312F, 0.00732421875F, -0.0109863281F, -0.00439453125F, 0.00329589844F, 0.0109863281F, -0.00274658203F, 0.00109863281F, -0.01171875F, -0.00439453125F, 0.01171875F, -0.00659179688F, -0.00549316406F, 0.00201416016F, -0.00100708008F, 0.005859375F, -0.00366210938F, 0.00274658203F, -0.00659179688F, 0.00805664062F, 0.000732421875F, 0.00164794922F, 0.00512695312F, -0.00137329102F, 0.00238037109F, -0.00732421875F}; +inline constexpr int64_t kLtx2RealDitFp8ControlF32Count = 2048; + } // namespace vllm_test diff --git a/tests/vllm/models/test_ltx2_device.cpp b/tests/vllm/models/test_ltx2_device.cpp index 6026ed72c..6bf3e8329 100644 --- a/tests/vllm/models/test_ltx2_device.cpp +++ b/tests/vllm/models/test_ltx2_device.cpp @@ -715,14 +715,23 @@ TEST_CASE("ltx2 device: HOST weights on a device queue are REFUSED by name") { // STAGES AND RUNS. F8_E4M3 plus an F32 scalar is the dequant path phase L6 // already implements. // * The first-party `ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` -// (18.72 GB, 7876 tensors) does NOT stage, and the refusal is L6's and is -// correct. That file carries NO `.torchao_nvfp4` marker at all, and its -// `weight_scale` is [4096, 256] — the LINEAR [N, K/16] layout — where -// `Ltx2DequantTorchaoNvfp4ToBf16` expects the SWIZZLED [1024, 1024] form. -// The two have the same element count, so reading one as the other -// type-checks and permutes every scale within a 128x4 tile; L6 refuses by -// name rather than doing that. The LINEAR-scale read is LOADER work, owed -// against the L6 surface and not this one — the forward never sees it. +// (18.72 GB, 7876 tensors) ALSO stages and runs, since phase L9a +// (.agents/specs/nvfp4-nibble-order.md). +// +// It used to be refused, and the refusal carried a WRONG diagnosis worth +// recording: it said the file stored a LINEAR [N, K/16] group scale. It does +// not. The bytes were SWIZZLED all along, declared in the cuBLAS-padded +// framing [4096, 256] rather than the `to_blocked` [1024, 1024] the loader +// knew — and for every layer in that file the padded framing is NUMERICALLY +// IDENTICAL to the linear shape, which is exactly why a shape test could not +// tell them apart and why the wrong diagnosis looked right. +// +// The file also packs element 2j in the HIGH nibble, where torchao, ModelOpt +// and our default put it in the LOW one. Nothing about the SHAPES could have +// revealed that: it was found by correlating the dequantized weights against +// the vonkaiser FP8 DiT of the same base weights (0.9956 correct vs 0.032 +// wrong), which is now a committed gate in test_ltx2_loader rather than a +// one-off measurement. // // WHAT IT DOES AND DOES NOT CLAIM. There is no golden at this geometry and there // is no oracle: vLLM-Omni carries no native LTX-2.5 path, so nothing here is a diff --git a/tests/vllm/models/test_ltx2_loader.cpp b/tests/vllm/models/test_ltx2_loader.cpp index 7cbce1c32..47b810d33 100644 --- a/tests/vllm/models/test_ltx2_loader.cpp +++ b/tests/vllm/models/test_ltx2_loader.cpp @@ -345,12 +345,19 @@ SyntheticDit BuildSyntheticDit(const Ltx2DitParams& p, Ltx2DitQuant quant, lin[static_cast(r * pcols + g)]; } } + // The synthetic NVFP4 DiT mirrors the SHIPPED one, which is Lightricks' + // nvfp4-prequant: no torchao marker, the cuBLAS-PADDED scale framing, and + // HIGH-nibble-first packing. It used to declare the to_blocked framing with + // no marker — a combination NEITHER producer emits, which the discriminator + // now refuses by name. That refusal is correct and this fixture was the + // thing that was wrong: a synthetic file has to be a file some real producer + // could have written, or it gates a shape nothing ships. std::vector want(static_cast(rows * cols)); vllm::DequantNvfp4ToBf16(packed.data(), lin_live.data(), scale2, rows, cols, - want.data()); + want.data(), vllm::Nvfp4NibbleOrder::kHighFirst); out.entries.push_back({pre + spec.name, "U8", {rows, cols / 2}, PackBytes(packed)}); - out.entries.push_back({pre + spec.name + "_scale", "F8_E4M3", - {prows / 4, pcols * 4}, PackBytes(sw)}); + out.entries.push_back({pre + spec.name + "_scale", "F8_E4M3", {prows, pcols}, + PackBytes(sw)}); out.entries.push_back({pre + spec.name + "_scale_2", "F32", {}, std::string(reinterpret_cast(&scale2), 4)}); out.expected[spec.name] = want; @@ -652,8 +659,8 @@ TEST_CASE("ltx2 loader: a torchao module built from the shipped bytes dequantize g.nbytes = sizeof(float); std::vector bf16(static_cast(out_features * in_features), 0); - vllm::Ltx2DequantTorchaoNvfp4ToBf16("video_aggregate_embed", w, s, g, out_features, - in_features, bf16.data()); + vllm::Ltx2DequantNvfp4ToBf16("video_aggregate_embed", w, s, g, out_features, in_features, + vllm::Ltx2Nvfp4Producer::kTorchao, bf16.data()); const size_t head = static_cast(vllm_test::kLtx2RealTeWeightHeadF32Count); std::vector got(head, 0.0F); for (size_t i = 0; i < head; ++i) got[i] = Bf16ToF32(bf16[i]); @@ -675,8 +682,8 @@ TEST_CASE("ltx2 loader: a torchao module built from the shipped bytes dequantize vllm::StTensor w1 = w; w1.data = ones.data(); std::vector probe(bf16.size(), 0); - vllm::Ltx2DequantTorchaoNvfp4ToBf16("video_aggregate_embed", w1, s, g, out_features, - in_features, probe.data()); + vllm::Ltx2DequantNvfp4ToBf16("video_aggregate_embed", w1, s, g, out_features, in_features, + vllm::Ltx2Nvfp4Producer::kTorchao, probe.data()); const size_t probe_n = static_cast(out_features) * static_cast(in_features); std::vector probe_got(probe_n, 0.0F); std::vector probe_want(probe_n, 0.0F); @@ -697,8 +704,9 @@ TEST_CASE("ltx2 loader: a torchao module built from the shipped bytes dequantize // [out, in/16] type-checks on element count. It must not be accepted. vllm::StTensor bad = s; bad.shape = {out_features, in_features / 16}; - CHECK_THROWS(vllm::Ltx2DequantTorchaoNvfp4ToBf16("video_aggregate_embed", w, bad, g, - out_features, in_features, bf16.data())); + CHECK_THROWS(vllm::Ltx2DequantNvfp4ToBf16("video_aggregate_embed", w, bad, g, out_features, + in_features, vllm::Ltx2Nvfp4Producer::kTorchao, + bf16.data())); // The PACKED-WIDTH refusal has to say something a reader can act on. Naming // the stored shape on both sides of "is X but the module is X" is a @@ -706,8 +714,9 @@ TEST_CASE("ltx2 loader: a torchao module built from the shipped bytes dequantize // stored shape implies. std::string width_what; try { - vllm::Ltx2DequantTorchaoNvfp4ToBf16("video_aggregate_embed", w, s, g, out_features, - in_features / 2, bf16.data()); + vllm::Ltx2DequantNvfp4ToBf16("video_aggregate_embed", w, s, g, out_features, + in_features / 2, vllm::Ltx2Nvfp4Producer::kTorchao, + bf16.data()); } catch (const std::exception& e) { width_what = e.what(); } @@ -732,16 +741,20 @@ TEST_CASE("ltx2 loader: the NVFP4 refusal names a tensor the checkpoint ACTUALLY const std::string scale_name = pre + "patchify_proj.weight_scale"; // Corrupt ONLY the stored SHAPE of one scale sidecar: the same 512 bytes - // declared as the PADDED-LINEAR [128, 4] rather than the swizzled [32, 16]. - // Same element count, so the safetensors reader accepts it and the refusal - // fires inside the torchao helper, which is the call site under test. + // declared in the torchao `to_blocked` framing [32, 16] rather than the + // cuBLAS-padded [128, 4] this (marker-less) file actually uses. Same element + // count, so the safetensors reader accepts it and the refusal fires in the + // producer resolver, which is the call site under test. It is also a REAL + // disagreement rather than a nonsense shape: [32, 16] is exactly what a torchao + // file of this geometry would store, so nothing but the missing marker says it + // is wrong. std::set present; bool patched = false; for (StEntry& e : syn.entries) { present.insert(e.name); if (e.name == scale_name) { - REQUIRE(e.shape == std::vector{32, 16}); - e.shape = {128, 4}; + REQUIRE(e.shape == std::vector{128, 4}); + e.shape = {32, 16}; patched = true; } } @@ -998,6 +1011,253 @@ TEST_CASE("ltx2 loader: the shipped torchao text encoder's widths are the LOGICA } } +// =========================================================================== +// 5b. The FIRST-PARTY NVFP4 DiT, held against the FP8 DiT as an ORACLE +// +// .agents/specs/nvfp4-nibble-order.md section 5.2. The two shipped DiTs quantize +// the SAME base weights, so the FP8 file is an independent oracle for the NVFP4 +// read — which matters because NOTHING else here can tell a correct dequant from +// a plausible wrong one. A wrong nibble order or a wrong scale indexing produces +// finite, correctly shaped, correctly scaled garbage. +// +// The CONTROL arm is what makes this a gate rather than a witness: the same NVFP4 +// read is also correlated against a DIFFERENT module's FP8 weights, and that must +// COLLAPSE. Without it, a fixture that passes on any two finite arrays would look +// exactly like a fixture that works (spec ltx-2-5.md section 7.0(c)). +// =========================================================================== + +namespace { + +// Pearson correlation. Non-finite on either side is a FAILURE, never a 0 — the +// same polarity issue #449 hardened MaxAbsDiff for. +double Correlation(const std::vector& a, const float* b, size_t n) { + REQUIRE(a.size() == n); + REQUIRE(n > 1); + const vllm_test::MaxAbsDiffScanResult scan = vllm_test::MaxAbsDiffScan(a.data(), b, n); + if (!scan.ok()) { + FAIL_CHECK("correlation: NON-FINITE operand at index " + << scan.bad_index << " (got = " << scan.bad_got + << ", want = " << scan.bad_want << ")"); + return std::numeric_limits::quiet_NaN(); + } + double ma = 0.0, mb = 0.0; + for (size_t i = 0; i < n; ++i) { + ma += static_cast(a[i]); + mb += static_cast(b[i]); + } + ma /= static_cast(n); + mb /= static_cast(n); + double num = 0.0, da = 0.0, db = 0.0; + for (size_t i = 0; i < n; ++i) { + const double x = static_cast(a[i]) - ma; + const double y = static_cast(b[i]) - mb; + num += x * y; + da += x * x; + db += y * y; + } + REQUIRE(da > 0.0); + REQUIRE(db > 0.0); + return num / std::sqrt(da * db); +} + +double RelRms(const std::vector& a, const float* b, size_t n) { + double num = 0.0, den = 0.0; + for (size_t i = 0; i < n; ++i) { + const double d = static_cast(a[i]) - static_cast(b[i]); + num += d * d; + den += static_cast(b[i]) * static_cast(b[i]); + } + REQUIRE(den > 0.0); + return std::sqrt(num / den); +} + +// Unswizzle the real [128, 16] scale tile and decode the committed real rows. +// `order` is the thing under test: both arms are exercised by the gate below, so +// the "wrong" one is not hypothetical. +std::vector DequantRealNvfp4RowsWith(vllm::Nvfp4NibbleOrder order) { + const int64_t rows = vllm_test::kLtx2RealDitNvfp4ScaleTileRows; + const int64_t cols = vllm_test::kLtx2RealDitNvfp4ScaleTileCols; + const int64_t elems = vllm_test::kLtx2RealDitNvfp4RowElems; + const int64_t nrows = vllm_test::kLtx2RealDitNvfp4RowCount; + REQUIRE(vllm_test::kLtx2RealDitNvfp4ScaleTileCount == rows * cols); + REQUIRE(vllm_test::kLtx2RealDitNvfp4PackedCount == nrows * elems / 2); + REQUIRE(cols == elems / 16); + + // The REAL unswizzle, on the REAL bytes. + std::vector linear(static_cast(rows * cols), 0); + vllm::Ltx2UnswizzleNvfp4BlockScale(vllm_test::kLtx2RealDitNvfp4ScaleTile, + static_cast(rows * cols), rows, cols, + linear.data()); + + std::vector got; + got.reserve(static_cast(nrows * elems)); + for (int64_t i = 0; i < nrows; ++i) { + const int64_t row = vllm_test::kLtx2RealDitNvfp4Rows[i]; + std::vector bf16(static_cast(elems), 0); + vllm::DequantNvfp4ToBf16( + vllm_test::kLtx2RealDitNvfp4Packed + static_cast(i * elems / 2), + linear.data() + static_cast(row * cols), + vllm_test::kLtx2RealDitNvfp4Scale2, 1, elems, bf16.data(), order); + for (int64_t j = 0; j < elems; ++j) { + got.push_back(Bf16ToF32(bf16[static_cast(j)])); + } + } + return got; +} + +// RESOLVE the producer from the artifact's own marker state and declared shape, +// then decode with the order that producer implies. +// +// The resolver is INSIDE this helper deliberately. If it ever returned kTorchao +// for this file the nibble order would flip, the correlation would collapse, and +// the gate below would fail — which is what ties "the discriminator is right" to +// "the numbers are right" instead of testing them as two unrelated facts. +std::vector DequantRealNvfp4Rows() { + const std::vector declared( + vllm_test::kLtx2RealDitNvfp4DeclaredScaleShape, + vllm_test::kLtx2RealDitNvfp4DeclaredScaleShape + 2); + // The real file carries NO marker for this module — that absence IS the + // evidence, so it is passed as one. + const vllm::Ltx2Nvfp4Producer producer = vllm::Ltx2ResolveNvfp4Producer( + vllm_test::kLtx2RealDitNvfp4Module, nullptr, declared, + vllm_test::kLtx2RealDitNvfp4OutFeatures, vllm_test::kLtx2RealDitNvfp4InFeatures); + CHECK(producer == vllm::Ltx2Nvfp4Producer::kNvfp4Prequant); + const vllm::Nvfp4NibbleOrder order = vllm::Ltx2Nvfp4NibbleOrderFor(producer); + CHECK(order == vllm::Nvfp4NibbleOrder::kHighFirst); + return DequantRealNvfp4RowsWith(order); +} + +} // namespace + +TEST_CASE("ltx2 loader: the first-party NVFP4 DiT agrees with the FP8 DiT") { + const size_t n = static_cast(vllm_test::kLtx2RealDitNvfp4RowCount * + vllm_test::kLtx2RealDitNvfp4RowElems); + REQUIRE(vllm_test::kLtx2RealDitFp8OracleF32Count == static_cast(n)); + REQUIRE(vllm_test::kLtx2RealDitFp8ControlF32Count == static_cast(n)); + + const std::vector got = DequantRealNvfp4Rows(); + + const double corr = Correlation(got, vllm_test::kLtx2RealDitFp8OracleF32, n); + const double rel = RelRms(got, vllm_test::kLtx2RealDitFp8OracleF32, n); + const double ctrl = Correlation(got, vllm_test::kLtx2RealDitFp8ControlF32, n); + + // VALUES, not booleans (spec section 7.0). + INFO("corr(oracle) = " << corr << " rel_rms = " << rel << " corr(control) = " << ctrl); + CHECK(corr >= 0.99); + CHECK(rel <= 0.15); + // The control MUST collapse, or this fixture cannot separate right from wrong. + CHECK(ctrl < 0.2); + + // ── AND THE WRONG ORDER MUST FAIL, MEASURED RATHER THAN ASSUMED ───────────── + // + // A gate that only ever runs the right answer proves the right answer is + // self-consistent. Running the SAME bytes low-first is the defect this whole + // change exists to prevent, so its collapse is asserted here — the mutation + // built into the gate rather than left for a reviewer to perform. + const std::vector wrong = + DequantRealNvfp4RowsWith(vllm::Nvfp4NibbleOrder::kLowFirst); + const double wrong_corr = Correlation(wrong, vllm_test::kLtx2RealDitFp8OracleF32, n); + const double wrong_rel = RelRms(wrong, vllm_test::kLtx2RealDitFp8OracleF32, n); + INFO("low-first (WRONG for this file): corr = " << wrong_corr + << " rel_rms = " << wrong_rel); + CHECK(wrong_corr < 0.2); + CHECK(wrong_rel > 0.5); + // Both readings decode the same MULTISET per group, so absmax cannot separate + // them. Stated as a gated fact: it is why "the output looked reasonable" is not + // evidence here, and why the correlation is what this gate reduces to. + double got_absmax = 0.0, wrong_absmax = 0.0; + for (size_t i = 0; i < n; ++i) { + got_absmax = std::max(got_absmax, std::fabs(static_cast(got[i]))); + wrong_absmax = std::max(wrong_absmax, std::fabs(static_cast(wrong[i]))); + } + INFO("absmax right = " << got_absmax << " wrong = " << wrong_absmax); + CHECK(got_absmax == wrong_absmax); +} + +TEST_CASE("ltx2 loader: the producer discriminator, every row of the table") { + // .agents/specs/nvfp4-nibble-order.md section 3.2. The marker decides, the shape + // corroborates, and every other combination REFUSES. + const int64_t out_features = 4096, in_features = 4096; + const std::vector blocked = + vllm::Ltx2Nvfp4ToBlockedScaleShape(out_features, in_features); + const std::vector padded = + vllm::Ltx2Nvfp4PaddedScaleShape(out_features, in_features); + CHECK(blocked == std::vector{1024, 1024}); + CHECK(padded == std::vector{4096, 256}); + + vllm::Ltx2TorchaoNvfp4Marker marker; + marker.format = "torchao_nvfp4"; + marker.block_size = 16; + marker.is_swizzled_scales = true; + + // Marker present + to_blocked framing -> torchao, LOW-first. + const vllm::Ltx2Nvfp4Producer torchao = vllm::Ltx2ResolveNvfp4Producer( + "m", &marker, blocked, out_features, in_features); + CHECK(torchao == vllm::Ltx2Nvfp4Producer::kTorchao); + CHECK(vllm::Ltx2Nvfp4NibbleOrderFor(torchao) == vllm::Nvfp4NibbleOrder::kLowFirst); + + // Marker absent + cuBLAS-padded framing -> nvfp4-prequant, HIGH-first. + const vllm::Ltx2Nvfp4Producer prequant = + vllm::Ltx2ResolveNvfp4Producer("m", nullptr, padded, out_features, in_features); + CHECK(prequant == vllm::Ltx2Nvfp4Producer::kNvfp4Prequant); + CHECK(vllm::Ltx2Nvfp4NibbleOrderFor(prequant) == vllm::Nvfp4NibbleOrder::kHighFirst); + + // THE TWO DISAGREEMENTS. Each is a shape that is perfectly valid for the OTHER + // producer, so neither is malformed — which is exactly why picking one would be + // silent and wrong. + CHECK_THROWS( + vllm::Ltx2ResolveNvfp4Producer("m", &marker, padded, out_features, in_features)); + CHECK_THROWS( + vllm::Ltx2ResolveNvfp4Producer("m", nullptr, blocked, out_features, in_features)); + + // A shape that is neither framing, with and without a marker. + const std::vector nonsense = {512, 2048}; + CHECK_THROWS( + vllm::Ltx2ResolveNvfp4Producer("m", &marker, nonsense, out_features, in_features)); + CHECK_THROWS( + vllm::Ltx2ResolveNvfp4Producer("m", nullptr, nonsense, out_features, in_features)); + + // Both refusals must NAME the module, which is what "refuse by name" claims. + for (bool with_marker : {true, false}) { + std::string what; + try { + vllm::Ltx2ResolveNvfp4Producer("transformer_blocks.0.attn1.to_q", + with_marker ? &marker : nullptr, + with_marker ? padded : blocked, out_features, + in_features); + } catch (const std::exception& e) { + what = e.what(); + } + const std::string msg = "what: " + what; + INFO(msg); + CHECK(what.find("transformer_blocks.0.attn1.to_q") != std::string::npos); + } +} + +TEST_CASE("ltx2 loader: the two scale framings are the same buffer, differently dressed") { + // Why the shape cannot discriminate alone, as a gated fact rather than a claim + // in a comment: for every geometry the shipped DiT uses, the cuBLAS-padded + // framing has the SAME element count as the to_blocked one AND is numerically + // identical to the LINEAR [N, K/16] shape. + struct Geom { int64_t out, in; }; + const Geom geoms[] = {{4096, 4096}, {2048, 2048}, {16384, 4096}, {4096, 16384}, + {8192, 2048}, {2048, 4096}, {4096, 256}}; + for (const Geom& g : geoms) { + const std::vector blocked = vllm::Ltx2Nvfp4ToBlockedScaleShape(g.out, g.in); + const std::vector padded = vllm::Ltx2Nvfp4PaddedScaleShape(g.out, g.in); + const std::string msg = "geom [" + std::to_string(g.out) + ", " + std::to_string(g.in) + "]"; + INFO(msg); + CHECK(blocked[0] * blocked[1] == padded[0] * padded[1]); + // The trap: padded == linear [N, K/16] exactly, because N % 128 == 0 and + // (K/16) % 4 == 0 for all of these. + CHECK(padded == std::vector{g.out, g.in / 16}); + // ...and the two FRAMINGS are never equal to each other, so a marker-bearing + // file can always be told from a marker-less one by shape once the marker has + // already narrowed it. + CHECK(blocked != padded); + } +} + // =========================================================================== // 6. Whole-model materialization, on synthetic files // =========================================================================== diff --git a/tests/vllm/test_nvfp4_dequant.cpp b/tests/vllm/test_nvfp4_dequant.cpp index b94c92fc7..d8bb263c1 100644 --- a/tests/vllm/test_nvfp4_dequant.cpp +++ b/tests/vllm/test_nvfp4_dequant.cpp @@ -1,5 +1,6 @@ #include +#include #include #include #include @@ -136,3 +137,58 @@ TEST_CASE("DequantNvfp4ToBf16 two rows two groups") { // Row 1 elem 0: 4.0 * (1.5 * 2.0) = 12.0 CHECK(vt::BF16ToF32(out[in_dim + 0]) == doctest::Approx(12.0F)); } + +// --- The NIBBLE ORDER, which is a PRODUCER convention and not a fact about +// NVFP4. .agents/specs/nvfp4-nibble-order.md. +// +// The default is LOW-first: element 2j in the low nibble. That is torchao's +// `pack_uint4` (torchao/prototype/mx_formats/kernels.py:160, +// `uint8_data[::2] | uint8_data[1::2] << 4`) and what vLLM's own reader assumes +// (`break_fp4_bytes`, nvfp4_emulation_utils.py:321-324). Lightricks' +// `nvfp4-prequant` packs the other way (ltx-kernels/docs/NVFP4.md:27-29), and the +// H3 community converter does too (minimax_h3.h:1500-1503). +// +// WHY THIS NEEDS A GATE AND NOT A COMMENT: reading a file with the wrong order +// transposes every adjacent pair. Magnitudes, per-group extrema and every +// summary statistic survive intact — only the POSITIONS move — so nothing +// downstream can notice. The final CHECK states that blindness outright. +TEST_CASE("DequantNvfp4ToBf16 nibble order is selectable and defaults to low-first") { + // One group. Element 0 = +0.5 (0x1) low, element 1 = +6.0 (0x7) high. + std::vector packed(8, 0x00); + packed[0] = 0x71; + std::vector scale = {0x38}; // fp8-e4m3 = 1.0 + const float ws2 = 1.0F; + + std::vector lo(16, 0xFFFF), hi(16, 0xFFFF), dflt(16, 0xFFFF); + DequantNvfp4ToBf16(packed.data(), scale.data(), ws2, 1, 16, dflt.data()); + DequantNvfp4ToBf16(packed.data(), scale.data(), ws2, 1, 16, lo.data(), + vllm::Nvfp4NibbleOrder::kLowFirst); + DequantNvfp4ToBf16(packed.data(), scale.data(), ws2, 1, 16, hi.data(), + vllm::Nvfp4NibbleOrder::kHighFirst); + + // The DEFAULT must be low-first, bit for bit. This is what makes "every caller + // predating the parameter is unchanged" true by construction rather than by + // inspection of call sites. + CHECK(dflt == lo); + + // Low-first: element 0 is the LOW nibble (0x1 = +0.5). + CHECK(vt::BF16ToF32(lo[0]) == doctest::Approx(0.5F)); + CHECK(vt::BF16ToF32(lo[1]) == doctest::Approx(6.0F)); + // High-first: element 0 is the HIGH nibble (0x7 = +6.0). Exactly transposed. + CHECK(vt::BF16ToF32(hi[0]) == doctest::Approx(6.0F)); + CHECK(vt::BF16ToF32(hi[1]) == doctest::Approx(0.5F)); + + // The two differ ONLY by a pairwise swap, and every element is still present. + for (int j = 0; j < 8; ++j) { + CHECK(lo[2 * j] == hi[2 * j + 1]); + CHECK(lo[2 * j + 1] == hi[2 * j]); + } + + // THE BLINDNESS, stated as a gated fact: the two readings are the same multiset, + // so no magnitude summary can tell them apart. Anything checking "is the output + // finite / correctly scaled / the right absmax" passes under BOTH. + std::vector lo_sorted = lo, hi_sorted = hi; + std::sort(lo_sorted.begin(), lo_sorted.end()); + std::sort(hi_sorted.begin(), hi_sorted.end()); + CHECK(lo_sorted == hi_sorted); +} From 7d08e220af5fa0c3f3b4ca6231b98c98e05e1a63 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 12 Aug 2026 23:38:23 +0000 Subject: [PATCH 3/6] fix(ltx-2.5): an apostrophe in refusal prose breaks "refuse BY NAME" (#435) The loader refuses in single quotes, and users grep the checkpoint with what those quotes contain. An apostrophe in ordinary prose opens a quote that closes on the next one, so the message starts naming a fragment of its own explanation instead of a tensor. The marker-present branch shipped one: "under the other's assumption" made the refusal quote 'transformer_blocks.0.attn1.to_q.weight_scale' ... under the other's with an ODD number of delimiters. The whole-file refusal path already asserted this invariant; the resolver's own two refusals did not, which is how it got in. The discriminator case now scans both of them: quote count must be EVEN, at least one pair, and every quoted span must equal the tensor the refusal is about. Reinstating the apostrophe as a mutation turns that case red on "single-quote count = 3", so the assertion is gated rather than asserted. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .../model_executor/models/ltx2_loader.cpp | 2 +- tests/vllm/models/test_ltx2_loader.cpp | 27 ++++++++++++++++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/src/vllm/model_executor/models/ltx2_loader.cpp b/src/vllm/model_executor/models/ltx2_loader.cpp index d2591fc21..2b43a5b94 100644 --- a/src/vllm/model_executor/models/ltx2_loader.cpp +++ b/src/vllm/model_executor/models/ltx2_loader.cpp @@ -241,7 +241,7 @@ Ltx2Nvfp4Producer Ltx2ResolveNvfp4Producer(const std::string& module, "producer also packs the OPPOSITE nibble order. The marker says " "torchao and the shape says nvfp4-prequant; refusing rather " "than picking one, because each reading is finite and wrong " - "under the other's assumption." + "under the assumption of the other." : ". The declaration and the stored shape disagree.") + " See .agents/specs/nvfp4-nibble-order.md section 3.2."); } diff --git a/tests/vllm/models/test_ltx2_loader.cpp b/tests/vllm/models/test_ltx2_loader.cpp index 47b810d33..b5010a57b 100644 --- a/tests/vllm/models/test_ltx2_loader.cpp +++ b/tests/vllm/models/test_ltx2_loader.cpp @@ -25,6 +25,7 @@ // than papered over with a fabricated manifest. #include +#include #include #include #include @@ -1217,7 +1218,14 @@ TEST_CASE("ltx2 loader: the producer discriminator, every row of the table") { CHECK_THROWS( vllm::Ltx2ResolveNvfp4Producer("m", nullptr, nonsense, out_features, in_features)); - // Both refusals must NAME the module, which is what "refuse by name" claims. + // Both refusals must NAME the module, which is what "refuse by name" claims — + // and every SINGLE-QUOTED span in them must be a tensor name, because that is + // the convention the whole loader refuses in and the convention a user greps + // the checkpoint with. An apostrophe in ordinary prose silently breaks it: it + // opens a quote that closes on the next one, so the message starts "naming" a + // fragment of its own explanation. This suite already asserts that for the + // whole-file refusal path; asserting it HERE too is what stopped a stray + // "the other's assumption" from shipping in the marker-present branch. for (bool with_marker : {true, false}) { std::string what; try { @@ -1231,6 +1239,23 @@ TEST_CASE("ltx2 loader: the producer discriminator, every row of the table") { const std::string msg = "what: " + what; INFO(msg); CHECK(what.find("transformer_blocks.0.attn1.to_q") != std::string::npos); + // An even count means every quote pairs; an odd one means prose ate a + // delimiter. + const int64_t quotes = std::count(what.begin(), what.end(), '\''); + INFO("single-quote count = " << quotes); + CHECK(quotes % 2 == 0); + CHECK(quotes >= 2); + // ...and each pair really is the tensor this refusal is about. + size_t open = what.find('\''); + while (open != std::string::npos) { + const size_t close = what.find('\'', open + 1); + if (close == std::string::npos) break; + const std::string quoted = what.substr(open + 1, close - open - 1); + const std::string qmsg = "quoted span: " + quoted; + INFO(qmsg); + CHECK(quoted == "transformer_blocks.0.attn1.to_q.weight_scale"); + open = what.find('\'', close + 1); + } } } From 127405e9597ad22c6822c67a88555f3e58c2f6f3 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 12 Aug 2026 23:46:51 +0000 Subject: [PATCH 4/6] test(ltx-2.5): gate the producer resolution across ALL 1176 quantized modules (#435) The correlation gate proves ONE module decodes correctly. It does not prove the discriminator resolves the same way for the other 1175, and "I sampled a few and they agreed" is precisely how the linear-layout diagnosis got written. So the shipped NVFP4 DiT's own header goes in as a fixture (7876 tensors, names and shapes only, no weight bytes) and every U8 weight in it is pushed through Ltx2ResolveNvfp4Producer. Measured: packed=1176 markers=0 weight_scale_2=1176 input_scale=1176 resolved kNvfp4Prequant=1176 padded framing=1176 to_blocked framing=0 ambiguous_with_linear=1176 The last number is the one worth keeping. For EVERY module in this file the cuBLAS-padded framing is numerically identical to the linear [N, K/16] shape, so a shape test could not have separated swizzled from linear for a single one of them. That is now a counted, gated fact instead of an argument in a comment. markers=0 is the other load-bearing one: torchao always writes its sidecar, so zero of them across 7876 tensors is what excludes torchao and licenses the marker-less arm. input_scale=1176 records the export as W4A4-shaped, whose activation scales this weight-only path discards exactly as vLLM's own W4A16 method does (modelopt.py:1264-1268). test_ltx2_loader 23/2432 -> 24/4793. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- tests/vllm/models/ltx2_nvfp4_dit_manifest.inc | 7900 +++++++++++++++++ tests/vllm/models/test_ltx2_loader.cpp | 66 + 2 files changed, 7966 insertions(+) create mode 100644 tests/vllm/models/ltx2_nvfp4_dit_manifest.inc diff --git a/tests/vllm/models/ltx2_nvfp4_dit_manifest.inc b/tests/vllm/models/ltx2_nvfp4_dit_manifest.inc new file mode 100644 index 000000000..9aea93316 --- /dev/null +++ b/tests/vllm/models/ltx2_nvfp4_dit_manifest.inc @@ -0,0 +1,7900 @@ +// GENERATED by scripts/gen-minimax-h3-safetensors-manifest.py — DO NOT EDIT BY HAND. +// +// Tensor manifest of a REAL safetensors checkpoint (Lightricks/LTX-2.5 diffusion_models/ltx-2.5-22b-distilled-transformer-nvfp4.safetensors): 7876 tensors, +// names/dtypes/shapes only — no weight bytes. Captured from the file's own header via +// a local read of the file on the NAS. See .agents/specs/ltx-2-5.md. +#pragma once + +#include + +namespace vllm_test { + +inline constexpr int64_t kLtx25Nvfp4DitTensorCount = 7876; + +struct Ltx25Nvfp4DitTensor { + const char* name; + const char* dtype; + int64_t shape[5]; // 0-padded (Conv3d weights are rank 5) + int32_t rank; +}; + +inline constexpr Ltx25Nvfp4DitTensor kLtx25Nvfp4DitTensors[] = { + {"model.diffusion_model.adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.adaln_single.linear.bias", "BF16", {36864, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.adaln_single.linear.weight", "BF16", {36864, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_adaln_single.linear.bias", "BF16", {18432, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_adaln_single.linear.weight", "BF16", {18432, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.learnable_registers", "BF16", {128, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.0.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.1.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.2.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.3.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.4.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.5.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.6.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_embeddings_connector.transformer_1d_blocks.7.ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_patchify_proj.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_patchify_proj.weight", "BF16", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_proj_out.bias", "BF16", {128, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_proj_out.weight", "BF16", {128, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_prompt_adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_prompt_adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_prompt_adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_prompt_adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_prompt_adaln_single.linear.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.audio_prompt_adaln_single.linear.weight", "BF16", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.audio_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_a2v_gate_adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_a2v_gate_adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_a2v_gate_adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_a2v_gate_adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_a2v_gate_adaln_single.linear.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_a2v_gate_adaln_single.linear.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_audio_scale_shift_adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_audio_scale_shift_adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_audio_scale_shift_adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_audio_scale_shift_adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_audio_scale_shift_adaln_single.linear.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_audio_scale_shift_adaln_single.linear.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_v2a_gate_adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_v2a_gate_adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_v2a_gate_adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_v2a_gate_adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_v2a_gate_adaln_single.linear.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_v2a_gate_adaln_single.linear.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_video_scale_shift_adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_video_scale_shift_adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_video_scale_shift_adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_video_scale_shift_adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.av_ca_video_scale_shift_adaln_single.linear.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.av_ca_video_scale_shift_adaln_single.linear.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.patchify_proj.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.patchify_proj.weight", "BF16", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.proj_out.bias", "BF16", {128, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.proj_out.weight", "BF16", {128, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.prompt_adaln_single.emb.timestep_embedder.linear_1.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.prompt_adaln_single.emb.timestep_embedder.linear_1.weight", "BF16", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.prompt_adaln_single.emb.timestep_embedder.linear_2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.prompt_adaln_single.emb.timestep_embedder.linear_2.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.prompt_adaln_single.linear.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.prompt_adaln_single.linear.weight", "BF16", {8192, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.0.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.1.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.10.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.11.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.12.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.13.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.14.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.15.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.16.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.17.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.18.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.19.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.2.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.20.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.21.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.22.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.23.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.24.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.25.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.26.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.27.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.28.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.29.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.3.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.30.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.31.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.32.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.33.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.34.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.35.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.36.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.37.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.38.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.39.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.4.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.40.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.41.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.42.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.attn2.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_attn2.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_out.0.weight", "BF16", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_q.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.audio_to_video_attn.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_k.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.42.video_to_audio_attn.to_v.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.attn2.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_attn2.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_out.0.weight", "BF16", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_q.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.audio_to_video_attn.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_k.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.43.video_to_audio_attn.to_v.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.attn2.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_attn2.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_out.0.weight", "BF16", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_q.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.audio_to_video_attn.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_k.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.44.video_to_audio_attn.to_v.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.attn2.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_attn2.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_out.0.weight", "BF16", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_q.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.audio_to_video_attn.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_k.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.45.video_to_audio_attn.to_v.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.attn2.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_attn2.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_out.0.weight", "BF16", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_q.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.audio_to_video_attn.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_k.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.46.video_to_audio_attn.to_v.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.attn2.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn1.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_attn2.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_ff.net.0.proj.weight", "BF16", {8192, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_ff.net.2.weight", "BF16", {2048, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_k.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_out.0.weight", "BF16", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_q.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.audio_to_video_attn.to_v.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_k.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_out.0.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_q.weight", "BF16", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.47.video_to_audio_attn.to_v.weight", "BF16", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.5.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.6.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.7.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.8.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn2.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_k.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_k.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_out.0.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_out.0.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_q.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_q.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_v.weight", "U8", {4096, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_v.weight_scale", "F8_E4M3", {4096, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn1.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_attn2.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.0.proj.bias", "BF16", {8192, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.0.proj.weight", "U8", {8192, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.0.proj.weight_scale", "F8_E4M3", {8192, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.2.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.2.weight", "U8", {2048, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.2.weight_scale", "F8_E4M3", {2048, 512, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_prompt_scale_shift_table", "F32", {2, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_scale_shift_table", "F32", {9, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_k.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_k.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_out.0.weight", "U8", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_out.0.weight_scale", "F8_E4M3", {4096, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_q.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_q.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_v.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_v.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.audio_to_video_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.ff.net.0.proj.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.ff.net.0.proj.weight", "U8", {16384, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.ff.net.0.proj.weight_scale", "F8_E4M3", {16384, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.ff.net.0.proj.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.ff.net.2.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.ff.net.2.weight", "U8", {4096, 8192, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.ff.net.2.weight_scale", "F8_E4M3", {4096, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.ff.net.2.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.prompt_scale_shift_table", "F32", {2, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.scale_shift_table", "F32", {9, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.scale_shift_table_a2v_ca_audio", "F32", {5, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.scale_shift_table_a2v_ca_video", "F32", {5, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.k_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.q_norm.weight", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_gate_logits.weight", "BF16", {32, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_k.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_k.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_k.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_k.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_k.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_out.0.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_out.0.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_out.0.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_out.0.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_out.0.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_q.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_q.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_q.weight", "U8", {2048, 1024, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_q.weight_scale", "F8_E4M3", {2048, 128, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_q.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_v.bias", "BF16", {2048, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_v.input_scale", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_v.weight", "U8", {2048, 2048, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_v.weight_scale", "F8_E4M3", {2048, 256, 0, 0, 0}, 2}, + {"model.diffusion_model.transformer_blocks.9.video_to_audio_attn.to_v.weight_scale_2", "F32", {0, 0, 0, 0, 0}, 0}, + {"model.diffusion_model.video_embeddings_connector.learnable_registers", "BF16", {128, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.0.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.1.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.2.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.3.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.4.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.5.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.6.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.k_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.q_norm.weight", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_gate_logits.bias", "BF16", {32, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_gate_logits.weight", "BF16", {32, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_k.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_k.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_out.0.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_out.0.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_q.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_q.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_v.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.attn1.to_v.weight", "BF16", {4096, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.ff.net.0.proj.bias", "BF16", {16384, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.ff.net.0.proj.weight", "BF16", {16384, 4096, 0, 0, 0}, 2}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.ff.net.2.bias", "BF16", {4096, 0, 0, 0, 0}, 1}, + {"model.diffusion_model.video_embeddings_connector.transformer_1d_blocks.7.ff.net.2.weight", "BF16", {4096, 16384, 0, 0, 0}, 2}, +}; + +} // namespace vllm_test diff --git a/tests/vllm/models/test_ltx2_loader.cpp b/tests/vllm/models/test_ltx2_loader.cpp index b5010a57b..b5ae1ab2a 100644 --- a/tests/vllm/models/test_ltx2_loader.cpp +++ b/tests/vllm/models/test_ltx2_loader.cpp @@ -46,6 +46,7 @@ #include "support/max_abs_diff.h" #include "ltx2_fp8_dit_manifest.inc" +#include "ltx2_nvfp4_dit_manifest.inc" #include "ltx2_nvfp4_te_manifest.inc" #include "ltx2_quant_goldens.inc" @@ -1259,6 +1260,71 @@ TEST_CASE("ltx2 loader: the producer discriminator, every row of the table") { } } +TEST_CASE("ltx2 loader: EVERY quantized module of the shipped NVFP4 DiT resolves alike") { + // The correlation gate proves ONE module decodes correctly. This proves the + // producer resolves the SAME way for all 1176 of them, straight off the real + // header — because "I sampled a few and they agreed" is how the linear-layout + // diagnosis got written in the first place. + REQUIRE(vllm_test::kLtx25Nvfp4DitTensorCount == 7876); + const std::string pre = vllm::kLtx2DitCheckpointPrefix; + + // A name -> shape index over the manifest, so the scale lookup is not O(n^2). + std::map by_name; + for (const auto& t : vllm_test::kLtx25Nvfp4DitTensors) by_name[t.name] = &t; + + int64_t packed = 0, markers = 0, scale2 = 0, input_scale = 0, resolved_prequant = 0; + int64_t padded_framing = 0, would_be_blocked = 0, ambiguous_with_linear = 0; + for (const auto& t : vllm_test::kLtx25Nvfp4DitTensors) { + const std::string name = t.name; + if (EndsWith(name, ".torchao_nvfp4")) ++markers; + if (EndsWith(name, ".weight_scale_2")) ++scale2; + if (EndsWith(name, ".input_scale")) ++input_scale; + if (std::string(t.dtype) != "U8") continue; + ++packed; + REQUIRE(t.rank == 2); + const int64_t out_features = t.shape[0]; + const int64_t in_features = t.shape[1] * 2; // TWO values per byte + + const auto it = by_name.find(name + "_scale"); + REQUIRE(it != by_name.end()); + const std::vector scale_shape = ShapeOf(it->second->shape, it->second->rank); + + // No marker ANYWHERE in this file, so every module takes the marker-less arm. + const vllm::Ltx2Nvfp4Producer p = vllm::Ltx2ResolveNvfp4Producer( + name, nullptr, scale_shape, out_features, in_features); + if (p == vllm::Ltx2Nvfp4Producer::kNvfp4Prequant) ++resolved_prequant; + if (scale_shape == vllm::Ltx2Nvfp4PaddedScaleShape(out_features, in_features)) { + ++padded_framing; + } + if (scale_shape == vllm::Ltx2Nvfp4ToBlockedScaleShape(out_features, in_features)) { + ++would_be_blocked; + } + // THE AMBIGUITY, counted rather than asserted in prose: for these geometries + // the padded framing IS the linear [N, K/16] shape, so a shape test alone + // could not have told swizzled from linear for a single one of them. + if (scale_shape == std::vector{out_features, in_features / 16}) { + ++ambiguous_with_linear; + } + } + INFO("packed=" << packed << " markers=" << markers << " scale2=" << scale2 + << " input_scale=" << input_scale << " prequant=" << resolved_prequant + << " padded=" << padded_framing << " blocked=" << would_be_blocked + << " ambiguous_with_linear=" << ambiguous_with_linear); + CHECK(packed == 1176); + // torchao ALWAYS writes its marker. Zero of them is what excludes torchao, and + // it is the whole basis of the marker-less arm. + CHECK(markers == 0); + CHECK(scale2 == 1176); + // W4A4-shaped export: the activation scales are present and UNUSED on this + // weight-only path, exactly as vLLM's W4A16 method discards them + // (modelopt.py:1264-1268). + CHECK(input_scale == 1176); + CHECK(resolved_prequant == 1176); + CHECK(padded_framing == 1176); + CHECK(would_be_blocked == 0); + CHECK(ambiguous_with_linear == 1176); +} + TEST_CASE("ltx2 loader: the two scale framings are the same buffer, differently dressed") { // Why the shape cannot discriminate alone, as a gated fact rather than a claim // in a comment: for every geometry the shipped DiT uses, the cuBLAS-padded From 91258637812ef8949bebf7786b55778610fdd477 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 13 Aug 2026 02:04:27 +0000 Subject: [PATCH 5/6] spec(nvfp4): the Outcome -- what was measured, and the three things rejected (#435) Records what neither the code nor git does: the four-way correlation table, the three designs rejected and why, why the default is low-first, and the GB10 run where BOTH shipped DiTs stage and forward. The FP8 arm's absmax reproduces the previously recorded GB10 figures to the digit. That is what makes the NVFP4 numbers beside it evidence rather than merely new -- the harness is shown to be the same one that produced the accepted result. Also records what was NOT obtained: the device case's MESSAGE text was cut by a tail in the run harness, and the capture re-run timed out on the GPU lock (FLOCK_EXIT=1) against a box saturated by two other rows. The facts it would have printed are asserted among the 5626 passing assertions and gated from the committed manifest, so this is cosmetic -- said plainly rather than quietly omitted. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/nvfp4-nibble-order.md | 70 ++++++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/.agents/specs/nvfp4-nibble-order.md b/.agents/specs/nvfp4-nibble-order.md index 56bbf8136..479847a32 100644 --- a/.agents/specs/nvfp4-nibble-order.md +++ b/.agents/specs/nvfp4-nibble-order.md @@ -205,5 +205,71 @@ with finite non-degenerate output. ## 7. Outcome -To be written when this lands: what was measured, what was rejected, and why the default is -low-first. +**What was measured.** The first-party NVFP4 DiT is SWIZZLED and HIGH-nibble-first, and both +halves were established against an oracle that is not ours — the `vonkaiser` FP8 DiT of the +same base weights. Re-derived after a full-disk incident on the build host and identical to +every digit: + +| reading | rms | corr vs FP8 | rel rms | +|---|---|---|---| +| LINEAR / low-first (what the original brief asked for) | 0.013564 | 0.000414 | 1.786 | +| LINEAR / high-first | 0.013564 | 0.257746 | 1.558 | +| SWIZZLED / low-first (the shipped dequant) | 0.009208 | 0.032296 | 1.394 | +| **SWIZZLED / high-first** | **0.009208** | **0.995560** | **0.0946** | + +The committed gate re-measures this from real bytes at 0.994968 corr / 0.100672 rel rms, with +the control at 0.00362 and the deliberately-wrong nibble order at 0.00514. + +**What was rejected.** + +1. *A linear-layout arm.* The premise was wrong; building it would have made the file load and + render silently wrong. L9a stopped and returned `NEEDS_DECISION` instead. +2. *Discriminating by SHAPE.* Gated as impossible for this artifact: all 1176 quantized modules + have a cuBLAS-padded scale shape numerically identical to the linear one + (`ambiguous_with_linear == 1176`). The marker leads; the shape only corroborates. +3. *H3's nibble-swap-at-load.* Correct for H3, which also feeds a Marlin fp4-resident path, and + wasteful here: LTX-2.5 dequantizes to bf16, so a swap would rewrite 9.4 GB of packed weights + to feed a consumer that could read them in place. §3.1 records the condition that reverses + this decision. +4. *"The forward ran and the output was finite."* The gate asserts absmax is IDENTICAL under + both nibble orders, so every magnitude summary is blind to the defect by construction. Only + the correlation separates them. + +**Why the default is low-first.** It is what torchao's `pack_uint4` writes +(`kernels.py:160`), what torchao's own unpacker reads (`:137-139`), and what vLLM's two +independent readers assume (`nvfp4_emulation_utils.py:321-324` and `:101-108`). Every caller +predating this change consumes a ModelOpt or torchao checkpoint, so a defaulted parameter makes +"nothing else moves" true by construction — and the ten-gate sweep confirmed it, eight of them +byte-identical. + +**§4.2 closed, no defect.** torchao is low-first on three independent witnesses, so the shipped +text-encoder arm was already correct. Its gate is now anchored to those sources rather than +comparing our helper to itself. + +**Gates.** Ten NVFP4 gates plus nine LTX/video gates green; the only counts that moved are the +two suites this change adds cases to (`test_nvfp4_dequant` 4/47 -> 5/69, `test_ltx2_loader` +20/2363 -> 24/4793). Three mutations were run and all three turned the suite red: forcing +low-first (3 cases), resolving a marker-less padded file as torchao (3 cases), and reinstating +an apostrophe in a refusal (1 case). Tree restored byte-for-byte after each. + +**On the GPU (dgx.casa, GB10, one `flock -w 2700 $HOME/gpu.lock` hold 02:31:03 to 02:37:15).** +Both arms stage device-resident and forward, `Status: SUCCESS!`: + +| arm | file | assertions | output absmax | +|---|---|---|---| +| NVFP4 | `ltx-2.5-22b-distilled-transformer-nvfp4.safetensors` (18,721,432,024 B, 7876 tensors) | 5626 | video 0.275391 / audio 1.02344 | +| FP8 | `ltx-2.5-22b-distilled-fp8.safetensors` (6124 tensors) | 5624 | video 0.300781 / audio 2.14062 | + +The FP8 figures reproduce the previously recorded GB10 run to the digit, which is what makes +the NVFP4 figures next to them trustworthy rather than merely new. The two arms differ because +they are different quantizations AND different configurations: the NVFP4 file declares a config +(`double_precision_rope`, `av_ca_timestep_scale_multiplier = 1000`, both asserted after +adoption), while the FP8 file declares none and runs under manifest defaults. + +**Not obtained.** The device case's MESSAGE text (printed tensor count, staging seconds, config +source) was cut by a `tail` in the harness on the first run, and a second run to capture it +timed out on the GPU lock after 45 minutes (`FLOCK_EXIT=1`) against a saturated box. The +underlying facts are asserted rather than printed — the geometry CHECKs are among the 5626 that +passed — and 7876 tensors / 48 blocks / video 4096 / audio 2048 / 21.004B parameters are gated +from the committed manifest. Cosmetic, and not worth further contended GPU time while another +row is rendering. From dbf8a52f5dfc51194218d999fcc5a67f2a8f1512 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 13 Aug 2026 08:12:45 +0000 Subject: [PATCH 6/6] fix(nvfp4): "refused by name" was FALSE for the case that matters, and the sweep proving nothing (#435, #598) Six review findings on the L9a nibble-order change. The implementation is sound and the correctness argument holds; every one of these is an accuracy or coverage defect in what the change CLAIMS. F1 -- a safety guarantee that is false, stated to users. Three places said any other marker/shape combination is "refused by name". It is not. A marker-less NVFP4 checkpoint whose weight_scale is stored LINEAR [N, K/16] has, for every geometry with N % 128 == 0 and G % 4 == 0, a shape numerically IDENTICAL to the cuBLAS-padded framing. Ltx2ResolveNvfp4Producer returns kNvfp4Prequant for it, the loader then unswizzles scales that were never swizzled AND reads high-first, and the result is finite, correctly shaped, correctly scaled, WRONG. The refusal branch is unreachable for that entire class. That is not an exotic file. LINEAR [N, K/16] is what ModelOpt, llm-compressor and compressed-tensors all write -- vLLM's own readers allocate exactly that shape at modelopt.py:1335-1345 and compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4.py:73-76 (pin 555967922) -- and none of the three emits a .torchao_nvfp4 sidecar. So the marker's absence excludes torchao and NOTHING ELSE. The decision is still right, because there is no better evidence in the file, and that was checked rather than assumed: the shipped DiT's __metadata__ carries exactly config, gemma_source_checkpoint, model_version and license -- no quantization_config, no producer key, no nvfp4/torchao/quant substring anywhere in the config, and no tensor name mentioning the quantizer over all 7876 tensors. Unlike MiniMax-H3, whose community checkpoint DID name its converter, there is nothing here to key on. So the gate is NOT weakened; the sentence is corrected, in ltx2_loader.h, docs/USAGE.md and the spec's risk table, and the residual hazard becomes a tracked condition at spec section 3.1.1 beside the existing H3 reversal condition. F2 -- the ten-gate "byte-identical" evidence is half vacuous, and two live gates are blind. Instrumented DequantNvfp4ToBf16 with a call counter and re-ran mutation M3 (flip the shared default to kHighFirst) across all ten: RED test_nvfp4_dequant 6 calls, test_gguf_nvfp4 6, test_ltx2_loader 294 GREEN test_qwen3_forward 30 calls, test_minimax_h3 44 calls / 57,395 asserts GREEN the other five make ZERO calls on a default run Three of ten can see a nibble-order change. Five never execute the function, so their identical counts are evidence of nothing about it. Two execute it live and still pass -- and one of those is the project's OTHER high-first family, gated by exactly the statistic this spec proves is blind. If H3 is ever routed through the new kHighFirst while MiniMaxH3Nvfp4HighNibbleFirst() stays default-ON, the two compose into a double flip and nothing in the tree fires. Opened #598 for a nibble-sensitive H3 gate, linked from the roadmap issue table and the spec. F3 -- a recorded number the gate does not produce. Section 7 recorded the wrong-order arm at 0.00514, which matches no row of its own table and no line of the gate. The committed gate reports corr -0.00239115 / rel 1.41856. The control on the same sentence, 0.00362, was exact. Section 5.1 also named "the loader refuses the shape" as THE red-before for a change with two independent halves; a throw can only ever demonstrate the layout half, so both reds are now stated separately with the value each produces. F4 -- a comment contradicted by four call sites. nvfp4_dequant.h said the order is "never inferred and never defaulted per call site". It IS defaulted, and deliberately: minimax_h3_nvfp4.cpp:112, minimax_h3_device.cpp:1311, qwen3_5.cpp:1298 and dense_nvfp4_gemm.h all rely on it. The seam that genuinely has no default is Ltx2DequantNvfp4ToBf16. Comment corrected to say which is which and why. F5 -- the correlation tolerance was loose in one direction and unbounded in the other, RE-BOUNDED strictly stronger. rel_rms is a PREDICTED QUANTITY (the disagreement two different quantizations of the same base weights must show, measured 0.100672), not an error budget. `rel <= 0.15` alone admitted a +10% uniform group-scale error, and Pearson correlation is scale-INVARIANT -- corr reads 0.994968 to every printed digit for every multiplier -- so nothing else in the gate could catch it. It also had no floor, so an arm that reproduced the FP8 oracle exactly scored rel 0.0 / corr 1.0 and passed every correlation assertion. Replaced by a band: 0.085 <= rel <= 0.115, plus corr <= 0.998. Proven strictly stronger by A/B on the SAME binary path, only the bound differing: x1.09 group-scale error inside DequantNvfp4ToBf16, rel 0.1411 old `rel <= 0.15` GREEN 45/45 Status: SUCCESS! band RED CHECK( 0.1411 <= 0.115 ) is NOT correct! the FP8 oracle handed to the gate as its own answer old bounds every CORRELATION assertion passes; only the incidental absmax equality fires band RED on the result itself: rel >= 0.085 AND corr <= 0.998 Both mutations are now permanent arms of the gate rather than one-off logs, in the same style as the wrong-nibble arm: the gate asserts that a x1.10 scale error moves corr by less than 1e-6 (measured 7.35e-11) while pushing rel out of the band, and that the oracle-against-itself lands outside it too. F6 -- record the limit honestly. Section 1.1 claimed "three mutually consistent and independent witnesses". torchao is neither installed nor vendored on this box (import fails; no mx_formats/kernels.py on the filesystem), so its two lines are upstream source only, not executable here. And vLLM's break_fp4_bytes is reached only from dequantize_to_dtype, which serves the ModelOpt and compressed-tensors paths -- vLLM's torchao path (torchao.py:290-318) delegates to torchao.quantization.quantize_ and never calls it. So vLLM witnesses ModelOpt, not torchao's writer. Section 1.1 now tabulates what each witness proves and how it was verified. The default is unchanged; nothing moves either way. Gates (Release, CUDA=OFF, BUILD_EXIT=0, no ENOSPC/BFD in any build log, df 84% throughout). ctest -N 409, full ctest 409/409 passed, 0 failed, 1 skipped (test_voxtral_e2e). The ten NVFP4 gates, nine unchanged and one moved: test_nvfp4_dequant 5 / 69 test_ops_nvfp4_matmul 4 / 1 test_gguf_nvfp4 14 / 2352 test_ops_moe_grouped 6 / 3 test_qwen36_weights 7 / 45 test_ops_nvfp4_fp4 22 / 919 test_qwen3_forward 7 / 1557 test_ltx2_device 13 / 498 test_minimax_h3 79 / 57395 test_ltx2_loader 24 / 4809 test_ltx2_loader 4793 -> 4809 is the +16 from the re-banding and its two built-in mutation arms; it is the suite this change adds assertions to. Tree restored byte-for-byte after every mutation. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/roadmap_v1.md | 1 + .agents/specs/nvfp4-nibble-order.md | 260 +++++++++++++++--- docs/USAGE.md | 18 +- .../model_loader/nvfp4_dequant.h | 19 +- .../vllm/model_executor/models/ltx2_loader.h | 41 ++- tests/vllm/models/test_ltx2_loader.cpp | 59 +++- 6 files changed, 344 insertions(+), 54 deletions(-) diff --git a/.agents/roadmap_v1.md b/.agents/roadmap_v1.md index e10435cb3..e7f826828 100644 --- a/.agents/roadmap_v1.md +++ b/.agents/roadmap_v1.md @@ -122,6 +122,7 @@ issue is not yet placed. Keyed record: update in place, never append. | [#337](https://github.com/mudler/vllm.cpp/issues/337) | `ROAD-V1-C1` | `FUSION-DENSE-MIGRATE`: the five dgx SACRED paged-engine gates are OWED after the merged-GEMM fold (`test_{commandr,glm4,minicpm,minicpm3,phi3}_paged_engine` SKIP on a CPU box) | bug | | [#338](https://github.com/mudler/vllm.cpp/issues/338) | `ROAD-V1-C1` | MiniCPM/MiniCPM3 hard-code SiLU: upstream `MiniCPMMLP` (`minicpm.py:219-226`) selects `FatreluAndMul` on `hidden_act == "fatrelu"` and raises otherwise; our `parse_config` never reads `hidden_act` | bug | | [#241](https://github.com/mudler/vllm.cpp/issues/241) | `ROAD-V1-H3` | MiniMax-H3: support the PRUNED (AdaLN timestep-curve) checkpoint variants | feature | +| [#598](https://github.com/mudler/vllm.cpp/issues/598) | `ROAD-V1-H3` | `test_minimax_h3` is nibble-BLIND: 44 live `DequantNvfp4ToBf16` calls and 57,395 assertions all stay green with the fp4 nibble order flipped, so H3's load-time swap and the new shared `kHighFirst` parameter can compose into a double flip undetected (spec `specs/nvfp4-nibble-order.md` §5.5) | bug | | [#471](https://github.com/mudler/vllm.cpp/issues/471) | `GATE-PIN-UNPINNED-SNAPSHOTS` | Checkpoint gates resolve their snapshot UNPINNED: 56 of 61 take whatever `directory_iterator` yields, including the three DFlash gates on a repo caching two materially different revisions (spec `specs/gate-pin-unpinned-snapshots.md`) | bug | | [#472](https://github.com/mudler/vllm.cpp/issues/472) | `GATE-PIN-UNPINNED-SNAPSHOTS` | 80 of 92 committed goldens record no checkpoint revision, so their gates cannot be pinned from evidence (`q3mxfp4`, DFlash, every `*_greedy` corpus) | bug | | [#482](https://github.com/mudler/vllm.cpp/issues/482) | `GATE-PIN-UNPINNED-SNAPSHOTS` | `check-snapshot-pins` FALSE-POSITIVES on the correct pinned form: a file naming the revision then enumerating shards inside `snapshots//` is reported UNPINNED, and the message names a C++ header with no Python equivalent. No live FP only because `online_gate.py:3631` case-shadows (`snapshot` parameter vs `snapshots` marker) | bug | diff --git a/.agents/specs/nvfp4-nibble-order.md b/.agents/specs/nvfp4-nibble-order.md index 479847a32..0705d4c2c 100644 --- a/.agents/specs/nvfp4-nibble-order.md +++ b/.agents/specs/nvfp4-nibble-order.md @@ -42,10 +42,34 @@ See `.agents/specs/ltx-2-5.md` §4.1 for the full table and the control. `ltx-2-5.md` §4.2 recorded an open question: our torchao text-encoder arm reads low-first, but its gate compared against goldens made by our own low-first helper — consistency, not -correctness. **torchao's source settles it: low-first.** Three mutually consistent and -independent witnesses (torchao's packer, torchao's unpacker, vLLM's two readers). The -shipped TE arm is correct. This spec's test work converts that self-referential gate into -a source-anchored one so it cannot regress into a question again. +correctness. The answer is **low-first**, and the shipped TE arm is correct. This spec's +test work converts that self-referential gate into a source-anchored one so it cannot +regress into a question again. + +**What each witness actually witnesses — because "three independent witnesses" overstated +it.** torchao is neither installed nor vendored on this box (`import torchao` → +`ModuleNotFoundError`; no `mx_formats/kernels.py` anywhere on the filesystem), so the two +torchao citations are read from upstream source rather than executed or diffed here: + +| Witness | What it proves | Verified how | +|---|---|---| +| `torchao/prototype/mx_formats/kernels.py:160` (`pack_uint4`) | torchao's WRITER is low-first | upstream source only — **not executable on this box** | +| the same file `:137-139` (its unpacker) | torchao's READER agrees with its writer | upstream source only — **not executable on this box** | +| vLLM `nvfp4_emulation_utils.py:321-324` (`break_fp4_bytes`) | the **ModelOpt / compressed-tensors** convention is low-first | pinned tree at `555967922`, read and line-anchored | +| the Triton path at `:101-108` | the same, a second way | pinned tree, read and line-anchored | + +The last two are **not** independent witnesses to torchao's convention. `break_fp4_bytes` +is reached only from `dequantize_to_dtype` (`:380`), which serves the ModelOpt and +compressed-tensors NVFP4 paths; vLLM's torchao path (`quantization/torchao.py:290-318`) +delegates to `torchao.quantization.quantize_` and torchao's own tensor subclasses and never +calls it. So vLLM witnesses ModelOpt, torchao's own source witnesses torchao, and the claim +that rests on all of them is the weaker "ModelOpt and torchao agree, on two sources of +differing strength" rather than "three independent confirmations". + +This does not move the decision: the default is unchanged, and every caller of the shared +dequant that predates this spec consumes ModelOpt or compressed-tensors bytes — the arm the +strong witness covers. It is recorded so the strength of the anchor is not overstated to +whoever next changes the default. ## 2. Scope @@ -94,6 +118,40 @@ and the load-time normalization adopted**, because a host-side dequant parameter a device GEMM that reads the packed bytes itself. That is the tracked condition, written where it will be found. +### 3.1.1 The SECOND tracked condition: the resolver does not refuse a linear file + +The marker-less arm is an inference, and §3.3 says why no shape test can corroborate it for +this artifact. The consequence, stated exactly rather than as the comfortable version: + +**A marker-less NVFP4 checkpoint whose `weight_scale` is stored LINEAR `[N, K/16]` is +RESOLVED as `nvfp4-prequant`, not refused.** For every geometry with `N % 128 == 0` and +`G % 4 == 0` — all 1176 quantized modules of the shipped DiT, gated as +`ambiguous_with_linear == 1176` — the cuBLAS-padded shape and the linear shape are the same +two integers. Such a file gets the 128x4 unswizzle applied to scales that were never +swizzled AND is read high-first. It loads, it renders, and it is wrong. + +Linear `[N, K/16]` is the normal case, not a corner one. It is what NVIDIA ModelOpt, +llm-compressor and compressed-tensors all write, and what vLLM's own readers allocate for on +disk (`modelopt.py:1335-1345`, `compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4.py:73-76`, +both `[out, in // group_size]` at the pin `555967922`). None of the three emits a +`.torchao_nvfp4` sidecar, so **the marker's absence excludes torchao and nothing else.** + +It is resolved this way anyway because there is no better evidence in the file, and that was +checked rather than assumed: the shipped DiT's `__metadata__` carries exactly `config`, +`gemma_source_checkpoint`, `model_version` and `license` — no `quantization_config`, no +producer key, no `nvfp4`/`torchao`/`quant` substring anywhere in the config, and no tensor +name mentioning the quantizer (`MARKER_LIKE == []` over all 7876 tensors, read straight from +the 1,179,408-byte header). MiniMax-H3's community checkpoint DID name its converter in +metadata; this one names nothing. So the decision stands, and it is the **claim** that was +wrong: this resolver refuses every combination the SHAPE can separate, and cannot refuse the +one it cannot see. + +**The tracked condition.** Before this loader is pointed at any marker-less NVFP4 artifact +other than the first-party LTX-2.5 DiT, §5.2's correlation gate must be re-run for THAT +artifact against an independent oracle. Passing on the DiT is evidence about the DiT. If a +producer key ever does appear in such a file, key on it and demote this inference — that is +strictly better evidence than the shape, which is worth none here. + ### 3.2 The producer discriminator (operator-RATIFIED, 2026-08-13) The scale layout and the nibble order are decided together, from the **`torchao_nvfp4` @@ -104,11 +162,16 @@ always writes that marker, so its absence excludes torchao. |---|---|---| | present, `is_swizzled_scales=true` | `[32*ceil(N/128), 16*ceil(G/4)]` (`to_blocked` framing) | torchao: unswizzle, **low**-first | | absent | `[round_up(N,128), round_up(G,4)]` (cuBLAS-padded framing) | `nvfp4-prequant`: unswizzle, **high**-first | -| anything else | — | **REFUSE BY NAME** | +| anything else the shape can SEPARATE | — | **REFUSE BY NAME** | `G = in_features / 16`. No "probably" branch: a marker with the wrong shape, an absent marker with the wrong shape, or a marker declaring an unimplemented combination all refuse. +**The third row's qualifier is load-bearing, not hedging.** Because §3.3 makes the padded +framing shape-identical to the LINEAR one for every geometry here, the marker-less arm cannot +refuse a genuinely linear file — it resolves it, and reads it swizzled and high-first. §3.1.1 +states that consequence in full and records the condition it imposes. + **Both framings describe the SAME bytes.** `[32*ceil(N/128), 16*ceil(G/4)]` and `[round_up(N,128), round_up(G,4)]` have equal element counts and the identical layout; they are two 2-D dresses on one flat buffer. `Ltx2UnswizzleNvfp4BlockScale` is already @@ -134,31 +197,83 @@ unimplementable as written. |---|---| | A defaulted parameter silently changes an existing caller | Every NVFP4 gate re-run with **assertion counts compared**, not just Status (§5) | | The high-first arm is wrong and output is merely finite | The correlation gate (§5.2) requires the 0.9956 / 9.46% signature; finiteness is not accepted as evidence | -| Marker-absent inference generalizes to a file it should not | Refuse by name on any other marker/shape combination; no fallback branch | +| Marker-absent inference generalizes to a file it should not | **NOT mitigated by refusal, and §3.1.1 says so in the tree.** Every combination the SHAPE can separate refuses by name, with no fallback branch — but a marker-less file whose scale is stored LINEAR `[N, K/16]` is shape-identical to the padded framing for every geometry here, so it RESOLVES and is read wrong. The mitigation is §5.2's correlation gate against an independent oracle, owed per artifact, plus the tracked condition at §3.1.1 and the same statement in `ltx2_loader.h` and `docs/USAGE.md` | | Two nibble mechanisms drift apart | §3.1 records the condition under which H3's is adopted here | | The correlation gate needs a 21 GB and an 18.7 GB file | It reads byte RANGES from the two headers, materializes ~128 rows, and is opt-in behind an env var like the other shipped-checkpoint cases | ## 5. Tests and gates -### 5.1 RED first -A case that dequantizes the real first-party DiT bytes and asserts the FP8-correlation -signature must FAIL before the change, for the right reason (the loader refuses the shape), -and pass after. +### 5.1 RED first — and there are TWO reds, not one + +This section originally named one red ("the loader refuses the shape") for a change with two +independent halves. A throw can only ever demonstrate the first half, so stating it as *the* +right reason mislabels the evidence for the second. Both are required: + +**(a) The layout red — a THROW.** Before the change the loader knew only the `to_blocked` +framing, so the shipped DiT's cuBLAS-padded `weight_scale` was refused by name and no bytes +were decoded at all. That is a genuine red and it is the one the pre-change tree produces. + +**(b) The nibble-order red — a NUMBER, because no throw exists for it.** Fixing (a) alone +makes the file load and produce values that are finite, correctly shaped, correctly scaled +and wrong; §7 gates that absmax is IDENTICAL under both orders, so nothing structural can +fire. The only instrument that separates them is the correlation against the independent FP8 +oracle, and the red it must produce is the collapse recorded in §7: corr **-0.00239115**, +rel rms **1.41856**, against **0.994968 / 0.100672** for the correct read. + +Red (b) is not run once and discarded. It is a permanent arm of the committed gate +(`test_ltx2_loader.cpp`, the `kLowFirst` arm), for the reason the whole spec exists: the +defect it detects leaves no other trace. ### 5.2 The correlation gate — the one nobody had The `vonkaiser` FP8 DiT and the first-party NVFP4 DiT quantize the same base weights, so the FP8 file is an **independent oracle** for the NVFP4 read. The gate: - dequantizes the same module from both files (rows 0..127), -- asserts Pearson correlation **>= 0.99** and relative rms error **<= 0.15**, +- asserts Pearson correlation in **[0.99, 0.998]**, +- asserts relative rms error in the **BAND [0.085, 0.115]** — not a ceiling; see below, - asserts a **control**: the same NVFP4 read against a DIFFERENT module's FP8 weights correlates **< 0.2**, so the gate proves it can tell right from wrong rather than passing on any two finite arrays, +- asserts the **wrong nibble order** collapses (corr < 0.2, rel rms > 0.5), +- asserts the two defects the band exists for, executed rather than described: a uniform + **x1.10 group-scale error** leaves corr unmoved to within 1e-6 and pushes rel above the + band, and the **oracle against itself** lands below the rel floor and above the corr + ceiling, - and emits the VALUES, never a boolean. Without the control the gate is `7.0(c)` again — a fixture that cannot separate a correct implementation from a plausible wrong one. +**Why rel rms is a BAND (revised after review; the original `<= 0.15` was one-sided).** +rel rms here is a PREDICTED QUANTITY, not an error budget: it is the disagreement two +different quantizations of the same base weights MUST show, measured at 0.100672. A +one-sided ceiling was wrong in both directions, and both were measured on the committed +bytes rather than argued: + +| arm | corr | rel rms | `<= 0.15` | band | +|---|---|---|---|---| +| correct read | 0.994968 | 0.100672 | pass | **pass** | +| uniform group scale x1.04 | 0.994968 | 0.111935 | pass | pass | +| uniform group scale x1.05 | 0.994968 | 0.116760 | pass | **RED** | +| uniform group scale x1.09 (in the dequant) | 0.994984 | 0.141100 | pass | **RED** | +| uniform group scale x1.10 (in the dequant) | 0.994964 | 0.150094 | RED by 0.06% | **RED** | +| uniform group scale x0.92 | 0.994968 | 0.122618 | pass | **RED** | +| the FP8 oracle itself ("too good") | 1.0 | 0.0 | pass | **RED** | +| wrong nibble order | -0.00239 | 1.41856 | RED | RED | + +Rows marked "in the dequant" are real source mutations of `DequantNvfp4ToBf16`'s +`group_scale`, so the bf16 store-rounding moves with them; the others scale the decoded +floats afterwards, which is why x1.10 reads 0.148922 there and 0.150094 here. + +Two facts drive it. **Pearson correlation is scale-invariant** — corr reads 0.994968 to every +printed digit for every multiplier in that table — so rel is the ONLY statistic in the gate +that can see a systematic scale error, and slack spent there is not recoverable elsewhere. +And **rel had no floor**, so an arm that reproduced the oracle exactly — the shape a fixture +takes when the "independent" oracle has quietly become the thing under test — passed with +room to spare. The band's width (+/- ~15% of the measured value) is for compiler and libm +drift only: every input is a committed byte array reduced by a sequential double loop, so +the value is deterministic run to run. + ### 5.3 Nibble order, source-anchored A unit case pinning both orders against hand-computed bytes, plus a generator-side anchor on torchao's `pack_uint4` line and `ltx-kernels`' `hi_first` documentation, so a convention @@ -167,24 +282,46 @@ change upstream is reported as a source change (§1.1's repair). ### 5.4 The discriminator Every row of §3.2's table, including each refusal, asserted by name. -### 5.5 Unchanged-behaviour proof (operator's non-negotiable) -These must be green with **identical case and assertion counts**, reported. Baselines -captured on this branch at `f400413e`, Release, CUDA=OFF: +### 5.5 Unchanged-behaviour proof (operator's non-negotiable), and its REAL reach -| Gate | Baseline | -|---|---| -| `test_nvfp4_dequant` | 4 / 47 | -| `test_gguf_nvfp4` | 14 / 2352 | -| `test_qwen36_weights` | 7 / 45 | -| `test_qwen3_forward` | 7 / 1557 | -| `test_minimax_h3` | 79 / 57395 | -| `test_ops_nvfp4_matmul` | 4 / 1 | -| `test_ops_moe_grouped` | 6 / 3 | -| `test_ops_nvfp4_fp4` | 22 / 919 | -| `test_ltx2_loader` | 20 / 2363 | -| `test_ltx2_device` | 13 / 498 | - -Counts only RISE, and only in the two suites this change adds cases to. **If any other +These must be green with **identical case and assertion counts**, reported. Baselines +captured on this branch at `f400413e`, Release, CUDA=OFF. + +**A byte-identical count is only evidence for a gate that EXECUTES the changed function.** +That was assumed rather than checked when this section was written, and the check changes +what the sweep is worth. Each gate was instrumented with a call counter on +`DequantNvfp4ToBf16` and then re-run under mutation **M3** — flip the shared default from +`kLowFirst` to `kHighFirst`, i.e. break every defaulting caller at once: + +| Gate | Baseline | Live calls into `DequantNvfp4ToBf16` | Under M3 | What the count proves | +|---|---|---|---|---| +| `test_nvfp4_dequant` | 4 / 47 | 6 | **RED** | sensitive | +| `test_gguf_nvfp4` | 14 / 2352 | 6 | **RED** | sensitive | +| `test_ltx2_loader` | 20 / 2363 | 294 | **RED** | sensitive | +| `test_qwen3_forward` | 7 / 1557 | 30 | GREEN | **BLIND — executes it and cannot see the flip** | +| `test_minimax_h3` | 79 / 57395 | 44 | GREEN | **BLIND — 57,395 assertions, none nibble-sensitive** | +| `test_qwen36_weights` | 7 / 45 | 0 | GREEN | vacuous (needs a checkpoint that is absent) | +| `test_ops_nvfp4_matmul` | 4 / 1 | 0 | GREEN | vacuous | +| `test_ops_moe_grouped` | 6 / 3 | 0 | GREEN | vacuous | +| `test_ops_nvfp4_fp4` | 22 / 919 | 0 | GREEN | vacuous | +| `test_ltx2_device` | 13 / 498 | 0 | GREEN | vacuous | + +So the honest statement is **three of ten** gates can see a nibble-order change. Five never +call the function, and their identical counts say nothing about it in either direction. Two +call it live and still pass — which is not a defect in this change (the default is unchanged, +so nothing moves) but IS a standing gap in those suites, and the reason "eight byte-identical" +was the wrong summary of this evidence. + +**The composition hazard that creates.** H3 reaches low-first by normalizing its bytes at +load (`MiniMaxH3Nvfp4SwapNibbles`, gated behind `MiniMaxH3Nvfp4HighNibbleFirst()`), then +calling the shared dequant with the default. If H3 is ever routed through the new +`kHighFirst` parameter while that swap stays on, the two compose into a double flip — the +original defect, restored — and **nothing in the tree fires**. Tracked as +[#598](https://github.com/mudler/vllm.cpp/issues/598): H3 owes a nibble-sensitive gate. The +instrument already exists; H3's own FL2VA-GGUF correlation is the same independent-oracle +technique §5.2 builds here. + +Counts only RISE, and only in the suites this change adds cases to. **If any other gate moves, STOP.** `Status:` is grepped every run: a thrown case DROPS the assertion count while still printing "passed". @@ -199,7 +336,10 @@ with finite non-degenerate output. not merely a nibble order and the premise is wrong again. - Any existing NVFP4 gate changes case or assertion count → STOP. - A checkpoint appears that needs a third convention, or a marker/shape combination outside - §3.2 → refuse by name and report; do not add a branch. + §3.2 that the shape CAN separate → refuse by name and report; do not add a branch. +- A marker-less NVFP4 checkpoint other than the first-party DiT is pointed at this loader → + STOP until §5.2's correlation gate is run for THAT artifact. It will not be refused; §3.1.1 + says why, and passing on the DiT is evidence about the DiT. - Any temptation to accept "the forward ran and the output was finite" as the correctness result → forbidden; §5.2 is the result. @@ -217,8 +357,18 @@ every digit: | SWIZZLED / low-first (the shipped dequant) | 0.009208 | 0.032296 | 1.394 | | **SWIZZLED / high-first** | **0.009208** | **0.995560** | **0.0946** | -The committed gate re-measures this from real bytes at 0.994968 corr / 0.100672 rel rms, with -the control at 0.00362 and the deliberately-wrong nibble order at 0.00514. +**What the committed gate reports, transcribed from its own output** (`test_ltx2_loader +--test-case="*NVFP4 DiT agrees*" -s`, Release, CUDA=OFF). An earlier revision of this section +recorded the wrong-order arm as `0.00514`, a number no row of the table above and no line of +the gate produces; the control on the same sentence, `0.00362`, was exact. Corrected against +the running gate rather than re-transcribed: + +| arm | corr | rel rms | +|---|---|---| +| oracle (the correct read) | **0.994968** | **0.100672** | +| control (a DIFFERENT module's FP8 weights) | **0.00362335** | — | +| deliberately-wrong nibble order (low-first) | **-0.00239115** | **1.41856** | +| absmax, both orders | identical at 0.0217285 | — | **What was rejected.** @@ -234,23 +384,45 @@ the control at 0.00362 and the deliberately-wrong nibble order at 0.00514. 4. *"The forward ran and the output was finite."* The gate asserts absmax is IDENTICAL under both nibble orders, so every magnitude summary is blind to the defect by construction. Only the correlation separates them. +5. *A one-sided rel-rms tolerance.* Revised after review into the band at §5.2: `<= 0.15` alone + admitted a +10% uniform group-scale error, and correlation is scale-invariant so nothing + else in the gate could have caught it. **Why the default is low-first.** It is what torchao's `pack_uint4` writes -(`kernels.py:160`), what torchao's own unpacker reads (`:137-139`), and what vLLM's two -independent readers assume (`nvfp4_emulation_utils.py:321-324` and `:101-108`). Every caller -predating this change consumes a ModelOpt or torchao checkpoint, so a defaulted parameter makes -"nothing else moves" true by construction — and the ten-gate sweep confirmed it, eight of them -byte-identical. - -**§4.2 closed, no defect.** torchao is low-first on three independent witnesses, so the shipped -text-encoder arm was already correct. Its gate is now anchored to those sources rather than -comparing our helper to itself. +(`kernels.py:160`), what torchao's own unpacker reads (`:137-139`), and what vLLM's readers of +ModelOpt / compressed-tensors bytes assume (`nvfp4_emulation_utils.py:321-324` and `:101-108`). +§1.1 records what each of those witnesses is actually worth — the two torchao lines are upstream +source, not executable on this box, and the two vLLM readers witness the ModelOpt convention +rather than torchao's writer. Every caller predating this change consumes a ModelOpt or +compressed-tensors checkpoint, so a defaulted parameter makes "nothing else moves" true by +construction. + +**§4.2 closed, no defect.** torchao is low-first, so the shipped text-encoder arm was already +correct. Its gate is now anchored to upstream sources rather than comparing our helper to +itself. The earlier phrasing — "three independent witnesses" — is corrected in §1.1: two of the +four cited lines are torchao's own writer and reader (one convention, seen twice), and the other +two are vLLM reading ModelOpt. **Gates.** Ten NVFP4 gates plus nine LTX/video gates green; the only counts that moved are the -two suites this change adds cases to (`test_nvfp4_dequant` 4/47 -> 5/69, `test_ltx2_loader` -20/2363 -> 24/4793). Three mutations were run and all three turned the suite red: forcing -low-first (3 cases), resolving a marker-less padded file as torchao (3 cases), and reinstating -an apostrophe in a refusal (1 case). Tree restored byte-for-byte after each. +suites this change adds cases to (`test_nvfp4_dequant` 4/47 -> 5/69, `test_ltx2_loader` +20/2363 -> 24/4809, the last +16 from the §5.2 re-banding and its two built-in mutation arms). +Mutations run, each with the tree restored byte-for-byte afterwards: + +| mutation | result | +|---|---| +| force low-first at the LTX call site | RED (3 cases) | +| resolve a marker-less padded file as torchao | RED (3 cases) | +| reinstate an apostrophe in a refusal | RED (1 case) | +| **M3: flip the shared default to `kHighFirst`** | RED in 3 of 10 gates; **GREEN in the other 7**, two of which execute the changed function 30 and 44 times. §5.5 has the per-gate call counts; the blindness is tracked as [#598](https://github.com/mudler/vllm.cpp/issues/598) | +| **M4: uniform group-scale error, applied inside `DequantNvfp4ToBf16`** | x1.10 -> rel **0.150094**, which the old `<= 0.15` catches by 0.06%. x1.09 -> rel **0.1411**: old bound **GREEN** (45/45, `Status: SUCCESS!`), §5.2 band **RED**. Same binary path, only the bound differs, so the band is strictly stronger. Now a permanent arm of the gate | +| **the FP8 oracle handed to the gate as its own answer** | Old bounds: every CORRELATION assertion passes — corr `1 >= 0.99`, rel `0 <= 0.15`, control `0.0026 < 0.2`, and both wrong-order arms — and only the incidental `got_absmax == wrong_absmax` fires, because `wrong` still reads the real bytes while `got` does not. The assertion that is supposed to BE the result passed. §5.2 band: **RED on the result itself**, `rel >= 0.085` and `corr <= 0.998`. Also now a permanent arm | + +**What the ten-gate sweep does NOT prove.** Eight-of-ten byte-identical was reported as the +unchanged-behaviour evidence. Five of those eight never call `DequantNvfp4ToBf16` at all on a +default run, so their identical counts are evidence of nothing about this change in either +direction. The claim that survives is narrower and still sufficient: the default is unchanged, +every pre-existing caller resolves to it by construction, and the three gates that CAN see a +nibble-order change are green. **On the GPU (dgx.casa, GB10, one `flock -w 2700 $HOME/gpu.lock` hold 02:31:03 to 02:37:15).** Both arms stage device-resident and forward, `Status: SUCCESS!`: diff --git a/docs/USAGE.md b/docs/USAGE.md index 94c588e71..1c86bb994 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -1680,9 +1680,21 @@ The two NVFP4 checkpoints were written by different producers that disagree abou both the group-scale framing and which nibble holds which weight, so the loader resolves the producer from the `torchao_nvfp4` marker: present means torchao (`to_blocked` framing, low-nibble-first), absent means the Lightricks -`nvfp4-prequant` tool (cuBLAS-padded framing, high-nibble-first). Any other -combination is refused by name rather than guessed, because both readings -type-check and produce finite, correctly scaled, wrong weights. See +`nvfp4-prequant` tool (cuBLAS-padded framing, high-nibble-first). A marker whose +stored scale shape contradicts it, and a marker-less file whose shape is the +`to_blocked` framing or neither framing, are refused by name rather than guessed, +because both readings type-check and produce finite, correctly scaled, wrong +weights. + +The refusal cannot cover everything, and the limit is worth knowing before you +point this loader at a checkpoint it was not built for. A marker-less NVFP4 file +whose `weight_scale` is stored **linear** `[N, K/16]` — what ModelOpt, +llm-compressor and compressed-tensors write, none of which emit a +`torchao_nvfp4` sidecar — has, whenever `N % 128 == 0` and `K/16 % 4 == 0`, a +shape indistinguishable from the cuBLAS-padded one. Such a file is resolved as +`nvfp4-prequant` and read swizzled and high-first: it loads, and it is wrong. +Only the LTX-2.5 DiT is gated against an independent oracle here, so treat any +other marker-less NVFP4 checkpoint as unsupported until it is. See `.agents/specs/nvfp4-nibble-order.md`. Two behaviours a caller has to know. `Ltx2LoadDitFromSafetensors` REFUSES the diff --git a/include/vllm/model_executor/model_loader/nvfp4_dequant.h b/include/vllm/model_executor/model_loader/nvfp4_dequant.h index e1f7ad3e5..51127b5ec 100644 --- a/include/vllm/model_executor/model_loader/nvfp4_dequant.h +++ b/include/vllm/model_executor/model_loader/nvfp4_dequant.h @@ -42,9 +42,22 @@ inline constexpr float kE2M1Lut[8] = {0.0F, 0.5F, 1.0F, 1.5F, // // Read the wrong way round, every adjacent fp4 pair is transposed. The result is // finite, correctly shaped and correctly scaled, so nothing downstream notices: -// a decoder emits fluent tokens, a DiT renders a plausible-but-wrong frame. It is -// therefore never inferred and never defaulted per call site — the loader that -// knows the producer passes it. +// a decoder emits fluent tokens, a DiT renders a plausible-but-wrong frame. +// +// This parameter IS defaulted, to kLowFirst, and that is the deliberate design: +// every caller predating .agents/specs/nvfp4-nibble-order.md consumes a ModelOpt +// or compressed-tensors checkpoint, which is low-first, so the default makes +// "this change moved nothing else" true BY CONSTRUCTION rather than by +// inspection. Four callers rely on it today — minimax_h3_nvfp4.cpp:112, +// minimax_h3_device.cpp:1311, qwen3_5.cpp:1298 and +// dense_nvfp4_gemm.h DequantNvfp4ToBLayout — and H3 reaches low-first by +// normalizing its bytes at load (MiniMaxH3Nvfp4SwapNibbles) rather than by +// passing an order. +// +// The seam where the order is NEVER defaulted is `Ltx2DequantNvfp4ToBf16` +// (ltx2_loader.h), which takes a resolved `Ltx2Nvfp4Producer` with no default, +// because that is the path where the two conventions actually meet and a default +// would let a caller that never thought about it get the silent wrong answer. enum class Nvfp4NibbleOrder { // Element 2j in the LOW nibble, 2j+1 in the HIGH. torchao's `pack_uint4` // (torchao/prototype/mx_formats/kernels.py:160, diff --git a/include/vllm/model_executor/models/ltx2_loader.h b/include/vllm/model_executor/models/ltx2_loader.h index 9c4bc09af..3d05132af 100644 --- a/include/vllm/model_executor/models/ltx2_loader.h +++ b/include/vllm/model_executor/models/ltx2_loader.h @@ -199,9 +199,44 @@ inline constexpr const char* kLtx2TorchaoNvfp4MarkerSuffix = ".torchao_nvfp4"; // THE SHAPE CANNOT DISCRIMINATE ALONE, which is why the marker leads. Every layer // of the first-party DiT has N % 128 == 0 and G % 4 == 0, so its cuBLAS-padded // shape is NUMERICALLY IDENTICAL to the linear [N, G] one. A shape test can never -// separate those two. It is used only to CORROBORATE what the marker decided, and -// any other combination is refused by name rather than resolved in either -// direction. +// separate those two. It is used only to CORROBORATE what the marker decided. +// +// WHAT THE REFUSAL DOES AND DOES NOT COVER — state it exactly, because the +// tempting sentence ("any other combination is refused by name") is FALSE and +// would be read as a safety guarantee. +// +// A combination the shape can SEPARATE is refused by name: a marker with the +// to_blocked shape resolves, a marker with any other shape refuses, no marker +// with the to_blocked shape refuses, and no marker with a shape that is neither +// framing refuses. What CANNOT be refused is the case the shape cannot see. A +// marker-less NVFP4 checkpoint whose `weight_scale` is stored LINEAR [N, K/16] +// — never swizzled — presents, for every geometry with N % 128 == 0 and +// G % 4 == 0, a shape numerically identical to the cuBLAS-padded one. It +// therefore resolves kNvfp4Prequant, gets the 128x4 unswizzle applied to scales +// that were never swizzled, and is read high-first. Finite, correctly shaped, +// correctly scaled, WRONG. +// +// That is not an exotic file. LINEAR [N, K/16] is precisely what NVIDIA ModelOpt, +// llm-compressor and compressed-tensors write, and it is what vLLM's own readers +// expect on disk (modelopt.py:1335-1345 and +// compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4.py:73-76 both allocate +// `weight_scale` as [out, in // 16]); none of the three emits a `.torchao_nvfp4` +// sidecar. So the marker's absence excludes torchao and NOTHING ELSE. +// +// It is resolved this way anyway, deliberately, because there is no better +// evidence in the file: the shipped DiT's `__metadata__` carries exactly +// `config`, `gemma_source_checkpoint`, `model_version` and `license` — no +// `quantization_config`, no producer key, and no tensor name mentioning the +// quantizer. Unlike MiniMax-H3, whose community checkpoint DID name its +// converter, there is nothing here to key on. The correlation gate is what +// converts that inference into a measurement, and it is why the gate is the +// result rather than "the forward ran". +// +// The consequence is a tracked condition, recorded in +// .agents/specs/nvfp4-nibble-order.md section 3.1 alongside the H3 one: THIS +// LOADER MUST NOT BE POINTED AT A SECOND MARKER-LESS NVFP4 PRODUCER without +// first re-running the correlation gate against an independent oracle for that +// artifact. It resolves such a file rather than refusing it. enum class Ltx2Nvfp4Producer { kTorchao, // marker present: to_blocked framing, low-nibble-first kNvfp4Prequant, // marker absent: cuBLAS-padded framing, high-nibble-first diff --git a/tests/vllm/models/test_ltx2_loader.cpp b/tests/vllm/models/test_ltx2_loader.cpp index b5ae1ab2a..44d4b2a2f 100644 --- a/tests/vllm/models/test_ltx2_loader.cpp +++ b/tests/vllm/models/test_ltx2_loader.cpp @@ -1146,10 +1146,67 @@ TEST_CASE("ltx2 loader: the first-party NVFP4 DiT agrees with the FP8 DiT") { // VALUES, not booleans (spec section 7.0). INFO("corr(oracle) = " << corr << " rel_rms = " << rel << " corr(control) = " << ctrl); CHECK(corr >= 0.99); - CHECK(rel <= 0.15); + // ── rel_rms IS A BAND, NOT A CEILING ─────────────────────────────────────── + // + // rel_rms here is not an error budget to stay under; it is a PREDICTED + // QUANTITY — the disagreement two different quantizations of the same base + // weights must show. Measured 0.100672 (spec section 7), which is NVFP4's own + // group-scaled E2M1 error combined with the oracle's fp8-e4m3 error. + // + // A one-sided `<= 0.15` was wrong in both directions, and both were measured + // on these bytes rather than argued: + // + // TOO LOOSE ABOVE. A uniform group-scale error of x1.10 lands at rel 0.1489 + // and PASSED. Pearson correlation cannot help: it is scale-INVARIANT, so a + // systematic scale defect leaves corr at 0.994968 to every printed digit. + // rel is the ONLY statistic in this gate that can see it, which is exactly + // why it may not be spent as slack. Both facts are asserted below. + // + // UNBOUNDED BELOW. rel had no floor at all, so an arm that reproduced the + // FP8 oracle EXACTLY — the shape a fixture takes when the "independent" + // oracle has quietly become the thing under test, this project's 7.0(c) + // failure — scored rel 0.0, corr 1.0 and passed with room to spare. + // + // The band is the measured value +/- about 15%. The width is for compiler and + // libm drift, NOT for run-to-run noise: every input here is a committed byte + // array and the reduction is a plain sequential double loop, so the value is + // deterministic. It admits at worst a ~4% uniform scale error, against the + // ~10% the ceiling alone admitted. + CHECK(rel >= 0.085); + CHECK(rel <= 0.115); + // ...and the same ceiling on the correlation, for the same reason: an oracle + // that agrees TOO well is not a better result, it is a broken oracle. + CHECK(corr <= 0.998); // The control MUST collapse, or this fixture cannot separate right from wrong. CHECK(ctrl < 0.2); + // ── THE TWO MUTATIONS THE BAND EXISTS FOR, RUN HERE ───────────────────────── + // + // Same discipline as the wrong-nibble arm below: the defect is executed by the + // gate rather than described in a comment a future reader has to re-derive. + + // (1) A uniform +10% group-scale error. Invisible to corr BY CONSTRUCTION; + // caught only by the band's upper edge. + std::vector overscaled(got); + for (float& v : overscaled) v = static_cast(static_cast(v) * 1.10); + const double over_corr = Correlation(overscaled, vllm_test::kLtx2RealDitFp8OracleF32, n); + const double over_rel = RelRms(overscaled, vllm_test::kLtx2RealDitFp8OracleF32, n); + INFO("x1.10 group scale: corr = " << over_corr << " rel_rms = " << over_rel); + // Pearson is scale-invariant: the defect moves corr by less than a printed digit. + CHECK(std::fabs(over_corr - corr) < 1e-6); + // The old `rel <= 0.15` admitted this. The band does not. + CHECK(over_rel > 0.115); + + // (2) The too-good arm: hand the gate the oracle itself. Rejected by the rel + // FLOOR and by the corr ceiling, neither of which existed before. + const std::vector perfect(vllm_test::kLtx2RealDitFp8OracleF32, + vllm_test::kLtx2RealDitFp8OracleF32 + n); + const double perfect_corr = Correlation(perfect, vllm_test::kLtx2RealDitFp8OracleF32, n); + const double perfect_rel = RelRms(perfect, vllm_test::kLtx2RealDitFp8OracleF32, n); + INFO("oracle vs itself: corr = " << perfect_corr << " rel_rms = " << perfect_rel); + CHECK(perfect_rel < 0.085); + CHECK(perfect_corr > 0.998); + // ── AND THE WRONG ORDER MUST FAIL, MEASURED RATHER THAN ASSUMED ───────────── // // A gate that only ever runs the right answer proves the right answer is