Acer box: fixture test plan + first real-model checkpoint

[rhCat]

Next step: the real checkpoint is on the acer box

Weights: /home/rhmaomao/hosting/deepseek_v4_flash_0731

Pull the latest (includes the converter + packed-pool engine path):

cd ds4f-disk && git pull

Step 1 — discovery (paste the output back here):

python3 tools/convert-ds4f.py inspect /home/rhmaomao/hosting/deepseek_v4_flash_0731

This prints: shard count, config key mapping, routed-expert classification
(uniformity check = fixed-rate pool feasibility), dense per-layer bytes,
unclassified tensors (embed/norm/lm_head), and size estimates. The
converter refuses on missing config keys, non-uniform expert payloads, or
unmatched naming — if inspect shows surprises (e.g. a latent-MoE naming
scheme), the regexes get adjusted before any conversion runs.

Step 2 — convert (after inspect is confirmed):

python3 tools/convert-ds4f.py convert /home/rhmaomao/hosting/deepseek_v4_flash_0731 \
    --out ~/ds4f-model

Expect this to be the slow part: it copies the expert pool as-is (no
quantization, no dtype conversion) — at FP8 that's on the order of 250 GB
of reads+writes. Progress prints every 2000 experts. Verify step at the
end checks file sizes against the manifest.

Step 3 — structural run (the memory gate is the star):

./ds4f ~/ds4f-model --trunk ~/ds4f-model/trunk.bin \
    --offsets ~/ds4f-model/trunk.offsets --pool ~/ds4f-model/pool.bin \
    --preset laptop

On a small-RAM box this will REFUSE with the memory plan printed — that's
the gate working. Dial --cache-gb / --trunk-gb / --pin-layers until
it passes, then paste the run report. The engine currently validates the
memory/I-O structure at real scale (no kernels yet — those are roadmap).

Known naming assumptions in the converter (from the V3 family):
model.layers.{i}.mlp.experts.{e}.* for routed experts,
model.layers.{i}. prefix for dense layer tensors. inspect will show
whether V4-Flash matches.

[rhCat]

Acer box results — matches the Mac reference exactly.

• gate suite: 8/8 passed
• 20 fetch threads (nproc = 20), 0.04 s/token, peak RSS 0.26 GB @ 0.25 GB cache
• hit rates identical to reference: 54.6% @ 0.25 GB → 56.8% saturation at the ~288 MB working set
• replay table identical: LRU 56.13% vs Belady 56.60% at the tight edge (953 slots), ceiling 56.60% beyond

Interpretation: the fixture is deterministic (seed 7), so an identical trace + replay table across two machines is the determinism check passing — the pointer map, 3-phase cache, and trace path reproduce byte-for-byte. The box runs the full stack correctly.

Note: at this scale the fixture is not I/O-bound (290 MB read / 20 tokens), so sec/token differences are noise; the read-speed regime only shows at real-model scale.

Next milestone per roadmap: real DeepSeek-V4-Flash-0731 conversion — dense tensors → pack-trunk, routed experts as e.{layer}.{expert}, config.json with the no-defaults keys.

[rhCat]

Already up to date. then re-run inspect -- it now digs.

What the empty result meant: the converter only looked at the top level. The updated inspect handles both usual causes:

1. HF snapshot layout — git clone of a HF repo puts the real files under snapshots/<commit-hash>/. The tool now walks down (bounded) and re-points at the repo-like subdir automatically.
2. LFS clone without checkout — the files aren't there until git lfs pull. The tool now prints a tree of what is there and says so.

Please re-run:

cd ds4f-disk && git pull
python3 tools/convert-ds4f.py inspect /home/rhmaomao/hosting/deepseek_v4_flash_0731

and paste back the output (tree summary + mapping + classification, or the lfs hint).

[rhCat]

V4 naming confirmed different from V3 -- no model. prefix, ffn not mlp, per-tensor scales, hc_* hyper-connection tensors. The converter now accepts both styles, and inspect prints a skeleton histogram (digits collapsed to N) over all 72,317 tensors, so the expert leaf pattern shows itself without dumping names.

git pull and re-run:

python3 tools/convert-ds4f.py inspect /home/rhmaomao/hosting/deepseek_v4_flash_0731

Paste the routed experts block + the name skeletons section. Config already maps clean (43 layers, 256 experts, topk 6, hidden 4096, latent 1024) -- the histogram is the last unknown before conversion can run.

[rhCat]

The histogram settled it — V4 experts are layers.N.ffn.experts.N.wN.{weight,scale} (3 weights + 3 scales, uniform ~12.45 MB each, 11,008 experts = 43 x 256). The converter now matches that exactly, plus:

• shared_experts (plural) classified as resident
• mtp.* (multi-token prediction head) detected, reported, excluded — main layers only
• sample leaves show wq_a.weight-style names instead of bare weight

git pull, then run the conversion (this is the slow step — ~137 GB pool copy, progress every 2000 experts):

python3 tools/convert-ds4f.py convert /home/rhmaomao/hosting/deepseek_v4_flash_0731 --out ~/ds4f-model

Expect: config.json written (expert_nbytes=13045960)-ish, trunk 43 layers / 6.27 GB, pool.bin ~137 GB. Make sure ~160 GB free on disk. Then:

./ds4f ~/ds4f-model --trunk ~/ds4f-model/trunk.bin --offsets ~/ds4f-model/trunk.offsets --pool ~/ds4f-model/pool.bin --preset laptop

The memory gate will likely refuse on the first try — that is the gate working. Paste the refusal + the memory plan, or the run report if it starts.

[rhCat]

Two fixes, both pushed:

1. convert now resolves the repo root like inspect does — it refused because it was looking at the wrapper dir instead of model/. It now finds .../model itself and prints where.

2. Your ds4f binary was stale — the usage dump had no --pool line. Rebuild after pulling:

cd ds4f-disk && git pull && make

Then run the conversion (same command as before — it will now find the repo and start the ~137 GB pool copy, progress every 2000 experts):

python3 tools/convert-ds4f.py convert /home/rhmaomao/hosting/deepseek_v4_flash_0731 --out ~/ds4f-model

Then the structural run with the fresh binary:

./ds4f ~/ds4f-model --trunk ~/ds4f-model/trunk.bin \
    --offsets ~/ds4f-model/trunk.offsets --pool ~/ds4f-model/pool.bin \
    --preset laptop

Paste the convert summary and whatever the run prints (memory gate refusal or run report).

[rhCat]

Already up to date.
make: Nothing to be done for `all'. (the build warning was a set-but-unused variable — now printed in the run report as the pool source line). Then run the conversion:

found repo-like content under: None

It will print found repo-like content under: .../model and start the pool copy. Paste the convert summary.

[rhCat]

Conversion complete on the acer box:

• config.json: expert_nbytes = 13,369,344 (12.75 MB, uniform across all 11,008 experts)
• trunk.bin: 43 layers, 5.26 GB (layers alternate ~115/135 MB; the 6.27 GB estimate from the pre-fix inspect included the ~1 GB of shared_experts misclassified as dense — the fixed regex excludes them correctly)
• pool.bin: 137.06 GB, 11008 experts x 12.75 MB, layer-major fixed-rate
• verify: file sizes matched the manifest (no VERIFY FAIL)

The 304B checkpoint is now in ds4f-disk's layout. Structural run:

./ds4f ~/ds4f-model --trunk ~/ds4f-model/trunk.bin \
    --offsets ~/ds4f-model/trunk.offsets --pool ~/ds4f-model/pool.bin \
    --preset laptop --gen 5 --trace ~/ds4f-model/trace.csv

Expect: memory plan printed first (~13 GB: 8 GB cache + ~4 GB trunk pin + ring + shared estimate); either a gate refusal (if the box has less than ~14 GB available — that is the gate working) or the run report. Paste the whole output. If it runs, the trace also feeds the policy replay:

python3 tools/trace_replay.py ~/ds4f-model/trace.csv --caps 2,4,8,16

[rhCat]

First real-scale run: structure proven on the 304B checkpoint.

MEMORY PLAN  12.5 GB vs 125.2 GB available  -> gate passed
pin 28/43 layers, ring 2 x 142 MB, cache 598 slots (7.6 GB)
5 tokens in 39.6 s (7.91 s/token)
GB read per token: 3.76 (trunk 0.38 + experts 3.2)
cache: 1290 requests, 34 hits (2.6%), 0 dropped
PEAK RSS 11.92 GB (forecast was 12.5 -> within 5%)

Reading the numbers honestly:

1. The gate math validates. 12.5 forecast vs 11.92 measured — invariant 5 works at real scale.
2. GB/token = 3.76 is the real structural signal, and it is routing-bound, not cache-bound: the fixture router has no temporal locality, so 2.6% hits and every token re-reads ~3.2 GB of experts. Real routing kernels (roadmap) would collapse this toward the working set.
3. 7.91 s/token is NOT disk-bound — it is sink-bound. The skeleton's compute sink is a single-threaded FNV checksum over every byte read (~18 GB over the run). That's a placeholder for the real kernels; don't read speed into it. Two levers to see the difference:

# 1. more parallel preads — the box has 20 cores, default was 4
./ds4f ~/ds4f-model --trunk ~/ds4f-model/trunk.bin \
    --offsets ~/ds4f-model/trunk.offsets --pool ~/ds4f-model/pool.bin \
    --preset laptop --gen 5 --threads 20

# 2. the real request-pattern policy table from this run's trace
python3 tools/trace_replay.py ~/ds4f-model/trace.csv --caps 2,4,8,16

Optional third: --preset server (64 GB cache, 48 GB trunk budget) pins the whole 5.26 GB trunk -> trunk reads go to zero, leaving pure expert-read behavior.

Paste whichever you run. The most useful next number is the threads sweep (4/8/16/20) showing how the expert-read path scales with queue depth.

[rhCat]

Threads sweep (gen 3, real checkpoint):

threads  4: 7.53 s/token   GB/token 4.06   RSS 11.92 GB
threads  8: 7.31 s/token   GB/token 4.06   RSS 11.92 GB
threads 16: 7.25 s/token   GB/token 4.06   RSS 11.92 GB
threads 20: 7.30 s/token   GB/token 4.06   RSS 11.92 GB

Flat across 4 -> 20 threads. The expert-read path is NOT queue-depth
limited here — 5x the pread parallelism bought 4%. The wall is the
skeleton's synchronous loop + single-threaded compute sink: reads and
checksum don't overlap, and the sink is the placeholder for real kernels.
Conclusion: on this hardware the I/O path at 4 threads is already
sufficient; the next real speedup is pipelined kernels (roadmap), not
more read threads.

The replay table is the real result — the actual request pattern of
the 304B run (1290 requests from the gen-5 trace):

trace: 1290 requests, 1237 distinct experts
compulsory misses: 1237 -- ceiling for ANY policy at ANY size: 4.11% hit
 CACHE-GB  SLOTS    LRU   BELADY  PIN+LRU
        2    149   0.00%   4.11%   0.62%
        4    299   1.47%   4.11%   1.78%
        8    598   2.64%   4.11%   3.49%
       16   1196   4.11%   4.11%   4.11%

This is the kimi-k3 lesson reproduced on the real checkpoint:

• The stream is 96% first-touches (fixture router, no locality), so the
  best ANY policy can do is 4.11% — the compulsory ceiling.
• LRU at 2 GB: 0.00%. The pathological zero, on a near-cyclic stream.
• PIN+LRU at 2 GB: 0.62% — pinning the most-frequent experts captures
  most of the possible reuse with a tiny cache.
• Belady at 2 GB: 4.11% — the ceiling is reachable with 2 GB; LRU
  needs 16 GB to get there.

"The lever is the policy, not the size" — measured on the real model's
fetch pattern. Structural validation complete: pointer map, trunk
streaming, 3-phase cache, memory gate (forecast 12.5 vs measured 11.92),
and the policy tooling all proven on the 304B checkpoint.

Roadmap next: mxfp4 kernels (real math replaces the sink, pipelined so
reads/compute overlap) + real routing (creates the reuse that makes the
cache and the policy curve meaningful). The DSpark draft-pass-as-prefetch
idea gets interesting once routing is real.