SYM completeness: requant primitives + Quantization layer + layer SYM arms

[LeoBuron]

Goal

Make full-SYM training a composable property of the framework: requant primitives + a generic Quantization layer close the accumulator-range gap between SYM layers (Linear and LayerNorm emit raw accumulator-range mantissas, violating the int16 inter-layer norm); per-layer SYM arms (conv, pools, CE) complete the kernel inventory.

Fixed entry point: the SYM_INT32 -> SYM_INT32 requant primitive with int16-range mantissas (qMaxBits <= 16). M1 acceptance: the chain Linear -> Quant -> LayerNorm -> Quant -> Linear as a full-SYM training step (#192).
Research anchor: M. Deutel, F. Hannig, C. Mutschler, J. Teich, "On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M Microcontrollers", IEEE TCAD 44(4) 2025, doi:10.1109/TCAD.2024.3484354, arXiv:2407.10734. Design decisions recorded 2026-06-11 (interactive design review, 4 rounds).

Roadmap

Stage	Content	PRs
Immediate (parallel)	#188 rename, conversionMatrix hygiene, CE-fwd guard, UBSan CI job	188-rename-half-away, conversion-matrix-hygiene, ce-forward-dispatch-guard, ci-ubsan-overflow
M1 = #192	requant primitives (dynamic + fixed-scale) + goldgen module + gold suite; then Quantization layer + factory + opt-in validator + chain test	192-requant-primitives then 192-quantization-layer
M2 = #45	#189 shared rescale helper first; Conv1d/ConvT1d SYM via own plan (LayerNorm pattern)	plan, then 2-3 PRs
M3	full-SYM ECG example (first end-to-end proof)	1 PR
M4	pools SYM (Max select / Avg scale-fold / Adaptive int-div)	1 PR
M5	CE SYM arm + Softmax stabilization + full-SYM HAR with accuracy measurement	2 PRs
parallel	docs (mutation catalog, padding note, integer-only open problem, pool mechanics)	1 PR

Ordering: #188 and the matrix hygiene land before the requant primitives (clean names + guarded matrix); primitives before the layer; M2 needs the #189 helper; M3 needs M1+M2; M4/M5/docs parallelize against M2+.

Task list

Immediate tier (parallel):

• arithmetic: roundHTE implements round-half-away-from-zero, not half-to-even — rename it #188 — rename HTE -> HALF_AWAY, roundHTE -> roundHalfAway
• tensor: conversionMatrix hygiene — BOOL OOB, missing NULL guard, silent unsupported conversions, wrong same-type byte count #199 — conversionMatrix hygiene
• loss: CrossEntropy forward has no dtype dispatch — SYM_INT32 mantissas silently reinterpreted as float bits #200 — CrossEntropy forward fail-fast guard
• ci: add UBSan job (signed-integer-overflow, float-cast-overflow) #204 — CI UBSan job

M1 (#192):

• feat: Quantization layer — SYM→SYM requantization + full layer wiring #192 — requant primitives, Quantization layer, factory, opt-in validator, full-SYM chain test

M2 (#45):

• arithmetic: SYM_INT32 bias/beta rescale into product-scale domain can overflow int32 (UB cast) #189 — overflow policy + shared rescale helper + flag-gated semantic guards (prerequisite)
• layer: Conv1d + Conv1dTransposed SYM_INT32 forward + backward #45 — Conv1d/ConvT1d SYM forward + backward (own plan)

M3–M5:

• examples: full-SYM ECG reconstruction AE (Conv SYM + Quantization layers + MSE) #207 — full-SYM ECG example
• layer: SYM_INT32 arms for MaxPool1d / AvgPool1d / AdaptiveAvgPool1d #205 — pools SYM arms
• loss: CrossEntropy SYM_INT32 arm (+ Softmax interplay) with accuracy-comparison acceptance #206 — CrossEntropy SYM arm + accuracy measurement
• layer: Softmax SYM_INT32 core lacks max-subtraction stabilization #201 — Softmax SYM max-subtraction (at the latest in M5)

Parallel / unscheduled:

• docs: mutation-class catalog, padding-convention note, integer-only requant open problem, SYM pool mechanics #209 — CONVENTIONS.md docs batch
• test: shared goldgen Python module + C tensor-builder helper lib + remove dead MORE_SOURCES #208 — goldgen migration + C test helpers + MORE_SOURCES removal
• serial: serializeLayer missing DROPOUT and LAYERNORM cases #186 — Serialize/Deserialize QUANTIZATION case (third missing case)
• tensor: qMaxBits=32 advertised but UB in convertFloatTensorToSymInt32Tensor (float clamp bound rounds to 2^31) #202 — qMaxBits=32 UB cast (unscheduled)
• optimizer: sgdStepM SYM_INT32 does not requantize the grad back (asymmetry vs sgdStep) #203 — sgdStepM grad write-back asymmetry (unscheduled)

Scope boundary (agreed 2026-06-11)

This program builds framework infrastructure: per-layer SYM kernels (fwd+bwd), requant primitives, the Quantization layer, the overflow policy, and the opt-in validator. The FQT research stages — layer-selection and bitwidth questions (#140), forward-pass quantization composition (#141), fully quantized training evaluation (#142), gradient checkpointing (#4/#138/#139) — are Jan's thesis work and CONSUME these primitives. This program enables #140/#141/#142 and never closes them. #189 stays linked under #137 as Leo-authored framework hardening.

Relates to #137, #192.

🤖 Generated with Claude Code

[LeoBuron]

Milestone 1 complete (2026-06-15)

The requant primitive + Quantization-layer foundation is merged into develop:

• refactor(arithmetic): rename roundHTE to roundHalfAway — name was the bug (#188) #211 — rename roundHTE→roundHalfAway (the name was the bug, arithmetic: roundHTE implements round-half-away-from-zero, not half-to-even — rename it #188 closed)
• fix(tensor): conversionMatrix hygiene — BOOL OOB, NULL-fn guard, fail-fast unsupported, bit-packed same-type copy #212 — conversionMatrix hygiene (tensor: conversionMatrix hygiene — BOOL OOB, missing NULL guard, silent unsupported conversions, wrong same-type byte count #199 closed): [6][6] + static-assert, NULL-fn guard, fail-fast unsupported, bit-packed same-type copy
• fix(loss): fail-fast dtype dispatch for CrossEntropy forward — SYM mantissas were silently reinterpreted as float bits #213 — CrossEntropy forward dtype guard (loss: CrossEntropy forward has no dtype dispatch — SYM_INT32 mantissas silently reinterpreted as float bits #200 closed)
• ci: UBSan job (signed-integer-overflow, float-cast-overflow) for the SYM integer paths #214 — UBSan CI job (ci: add UBSan job (signed-integer-overflow, float-cast-overflow) #204 closed) — caught a real pre-existing NaN-cast UB on day 1 (bug(ubsan): float-cast-overflow in roundHTE — -nan cast to int (UnitTestMSE) #215, fixed in fix: remove NaN-UB source in MSE SYM backward test + dead dequant pass (#215) #216)
• feat(tensor): SYM_INT32 requant primitives — dynamic absmax + fixed-scale, in-place capable (#192 PR-1) #217 — SYM_INT32 requant primitives: dynamic requantSymInt32Tensor (wired into the conversionMatrix diagonal) + fixed-scale requantSymInt32TensorToScale; shared test/unit/goldgen module
• refactor(layer)+docs: int32 bias-grad accumulator + correct Deutel attribution (refs #218) #219 — int32 bias-grad accumulator (no int64 in SYM paths) + corrected the Deutel attribution
• feat(layer): Quantization layer — generic dtype-boundary + SYM requant, factory, opt-in model validator, full-SYM chain test (#192 PR-2) #220 — the generic Quantization layer (kernel + factory + opt-in model validator) and the full-SYM Linear → Quant → LayerNorm → Quant → Linear chain training-step acceptance test (62/62 green). feat: Quantization layer — SYM→SYM requantization + full layer wiring #192 closed.

New findings during execution

• research: quantized gradient-accumulation scheme — fixed-scale vs strategy A, and a Deutel attribution correction #218 — research: the bias-grad fixed-scale accumulation was mis-attributed to Deutel (verified against arXiv:2407.10734v2: the paper has no bias path and does float-space SGD); scheme choice + the init-scale resolution limit tracked here.
• quant/framework: backwardQ is not honored by the training loop's grad-tensor allocation #221 — backwardQ is not honored by the training loop's grad-tensor allocation (works for uniform-quant configs; per-layer backward-dtype override is a framework gap).

Next (per the roadmap)

M2 = Conv1d/ConvT1d SYM (#45) — needs its own plan, and the #189 rescale-overflow helper first. Then M3 (full-SYM ECG example, #207), M4 (pools SYM, #205), M5 (CrossEntropy SYM + accuracy, #206). Parallel: test infra (#208), docs (#209). Bugs filed: #201, #202, #203. Serialize cases deferred to #186.