Skip to content

Add Qwen3 component harness: torch reference, equivalence helper, microbench helper#265

Closed
chhwang wants to merge 1 commit into
mainfrom
qwen3-q3-harness
Closed

Add Qwen3 component harness: torch reference, equivalence helper, microbench helper#265
chhwang wants to merge 1 commit into
mainfrom
qwen3-q3-harness

Conversation

@chhwang

@chhwang chhwang commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Add Qwen3 component harness: torch reference, equivalence helper, microbench helper

Shared scaffolding under examples/qwen3/ for Q4–Q8 component work.

Torch reference (qwen3_ref.py): pure-torch Qwen3-8B forward —
RMSNorm, RoPE (θ=1e6), QK-norm, GQA (32q/8kv, head_dim 128), SwiGLU MLP
(intermediate 12288), TransformerBlock, full model. Random-weight init with
fixed seed, fp16. No ARK dependency.

Config (qwen3_config.py): parameterized Qwen3Config dataclass with
8B defaults. All fields overridable for future 32B stretch.

Equivalence helper (equiv.py): assert_close() wrapping
torch.testing.assert_close with shape, max abs/rel error, and per-tensor
statistics on mismatch.

Microbench helper (microbench.py): L2-flush buffer (128 MB),
CUDA-graph capture, pilot-driven iteration tuning, torch.profiler timing.
Returns {"mean_us", "std_us", "n_iters"}.

Tests (test_harness.py): 9 tests, all gated with
@skipif(not torch.cuda.is_available()). Coverage: reference forward shape,
RMSNorm unit-RMS check, causal mask validation, RoPE non-identity,
equivalence pass/fail/shape-mismatch, microbench eager + CUDA-graph.

Files are non-overlapping with Q1 (bench/) and Q2 (PROFILE.md).

…ence-test helper, and microbench helper under examples/qwen3/
@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.70%. Comparing base (c257202) to head (32261fc).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #265   +/-   ##
=======================================
  Coverage   85.70%   85.70%           
=======================================
  Files         129      129           
  Lines        6457     6457           
=======================================
  Hits         5534     5534           
  Misses        923      923           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chhwang chhwang changed the title ark-dev: Add Qwen3 component harness: torch reference module, equivalence-test helper, and microbench helper under examples/qwen3/ Add Qwen3 component harness: torch reference, equivalence helper, microbench helper Jun 12, 2026
@chhwang

chhwang commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Closing without merging. The harness code (torch reference, equivalence + microbench helpers under examples/qwen3/) is retained on branch qwen3-q3-harness and carried by the component PRs that build on it; not landing it as a standalone PR.

@chhwang chhwang closed this Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant