Context
PR #29 added the benchmark harnesses and multiple CPU contraction optimizations. The latest full benchmark refresh on that PR shows the network scenarios are faster than the one-thread Julia baseline, but several binary benchmark scenarios remain substantially slower.
Full benchmark refresh command set:
make bench-binary
make bench-network
JULIA_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 make bench-julia
python3 benchmarks/compare.py
make check
Current binary rows from the PR #29 full benchmark refresh:
| Benchmark | Scenario | Julia (ns) | Rust (ns) | Julia/Rust | Drop vs Julia |
|------------|----------------------------------|--------------|--------------|------------|---------------|
| binary | batched_matmul_8x8_batch_4 | 3000 | 26247 | 0.114 | 774.90% |
| binary | high_d_12x12_contract_4_batch_4 | 8709 | 42167 | 0.207 | 384.18% |
| binary | high_d_12x12_contract_6 | 22875 | 54340 | 0.421 | 137.55% |
| binary | high_d_15x15_contract_5_batch_5 | 52958 | 119670 | 0.443 | 125.97% |
| binary | high_d_15x15_contract_7 | 364417 | 375286 | 0.971 | 2.98% |
| binary | high_d_18x18_contract_8 | 10882416 | 10720317 | 1.015 | 0.00% |
| binary | high_d_20x20_contract_9 | 84399125 | 78107208 | 1.081 | 0.00% |
| binary | matmul_10x10 | 4333 | 35552 | 0.122 | 720.50% |
Problem
The largest remaining gaps are in small, batched, and materialization-heavy binary cases:
batched_matmul_8x8_batch_4: Rust is +774.90% slower.
matmul_10x10: Rust is +720.50% slower.
high_d_12x12_contract_4_batch_4: Rust is +384.18% slower.
The larger pure-contraction cases are now competitive or faster, so the remaining cost is likely not the large GEMM kernel itself.
Suggested investigation
- Add or reuse a focused binary profiling harness similar to
profile_network so setup cost and Criterion overhead do not obscure hotspots.
- Profile
matmul_10x10 and batched_matmul_8x8_batch_4 first.
- Check whether remaining time is dominated by contraction setup, layout analysis, small-GEMM dispatch overhead, permutation/materialization, or tensor allocation.
- Decide whether small Standard contractions need a separate fast path from the large faer-backed path.
Acceptance criteria
- Record a focused profile for at least
matmul_10x10 and batched_matmul_8x8_batch_4.
- Identify the dominant cost bucket for each.
- Either implement a targeted optimization with before/after benchmark numbers, or document why the current behavior is acceptable.
Context
PR #29 added the benchmark harnesses and multiple CPU contraction optimizations. The latest full benchmark refresh on that PR shows the network scenarios are faster than the one-thread Julia baseline, but several binary benchmark scenarios remain substantially slower.
Full benchmark refresh command set:
Current binary rows from the PR #29 full benchmark refresh:
Problem
The largest remaining gaps are in small, batched, and materialization-heavy binary cases:
batched_matmul_8x8_batch_4: Rust is+774.90%slower.matmul_10x10: Rust is+720.50%slower.high_d_12x12_contract_4_batch_4: Rust is+384.18%slower.The larger pure-contraction cases are now competitive or faster, so the remaining cost is likely not the large GEMM kernel itself.
Suggested investigation
profile_networkso setup cost and Criterion overhead do not obscure hotspots.matmul_10x10andbatched_matmul_8x8_batch_4first.Acceptance criteria
matmul_10x10andbatched_matmul_8x8_batch_4.