Skip to content

Track latest tensor4all-rs (ae655a9), expand default sweeps, add the mac-m1-8gb profile - #1

Merged
lingrui96 merged 9 commits into
mainfrom
track-ae655a9-expanded-sweeps
Aug 7, 2026
Merged

Track latest tensor4all-rs (ae655a9), expand default sweeps, add the mac-m1-8gb profile#1
lingrui96 merged 9 commits into
mainfrom
track-ae655a9-expanded-sweeps

Conversation

@lingrui96

@lingrui96 lingrui96 commented Aug 7, 2026

Copy link
Copy Markdown
Member

What this does

  • Moves the pin from 7cfec22 to ae655a9, the current origin/main tip of tensor4all-rs, whose one new commit is tensor4all-rs#575 (treetci early stop on max_bond_dim saturation).
  • Expands the default sweeps: cases 2 and 3 now run BENCH_RS=6,8,10,12,14, case 1 adds K=128. A full run_all.sh takes about 8 minutes on the machine below, on an otherwise idle machine.
  • Splits result profiles per machine: your result/mac-cpu sweep stays exactly as you committed it, and the new numbers land in result/mac-m1-8gb (an 8 GB Apple M1 MacBook Pro). Nothing of yours is overwritten, and README's Latest results section lists both profiles with their pins.
  • run.yaml now records chip and memory, and no longer records the hostname: on a public repository a DHCP name leaks the operator's institution and location over time. The branch history was rebuilt once before review for the same reason. Your historical run.yaml files on main are untouched, that is your call.
  • Refreshes every quoted number in the README and the runner doc comments from the mac-m1-8gb sweep, and adds known issue 9: the naive arms at r >= 10 are memory bound on 8 GB, so their wall times are only comparable within one profile and official sweeps want an idle machine.

Review follow-ups

All four requested cleanups are in, no rerun needed:

  1. Title updated to name the mac-m1-8gb profile instead of a mac-cpu rerun.
  2. Duration reconciled to about 8 minutes. The 14 minute figure was measured before a reboot, on a machine deep in swap; the committed sweep took 7.8 minutes from the first raw record to the rendered reports, which is what the README says.
  3. The provisional AGENTS.md Pushing section is removed. It can return as accepted maintainer policy in its own change.
  4. The spec's status is now implemented, and its open question list is replaced by your answers, so it reads as a record.

Thank you for the answers on the open questions. Recorded in the spec, and the construction benchmark, option (a), is the direction for the next case.

🤖 Generated with Claude Code

lingrui96 and others added 7 commits August 7, 2026 14:54
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pin moves from 7cfec22 to ae655a9, picking up the treetci early stop
on max_bond_dim saturation (#575). Default sweeps grow: cases 2 and 3 run
BENCH_RS=6,8,10,12,14, case 1 adds K=128. Probed on this machine, the
dominant cost is case 2 naive at about 100 s per run at r = 12 and 14; the
old quote of 12.6 s at r = 12 was stale. See
docs/superpowers/specs/2026-08-07-latest-rev-expanded-sweep-design.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…verride

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A DHCP hostname in a public repository leaks the operator's institution
and network location over time, and identifies nothing the profile name,
the machine label and the chip and memory fields do not already carry.
BENCH_MACHINE overrides the label, defaulting to the profile name.
Hostnames already in git history are left as they are.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… pin

result/mac-cpu stays frozen as the maintainer's machine's record at the
previous pin. The expanded-sweep numbers for the current pin land in
result/mac-m1-8gb (committed separately with the sweep). README case
descriptions, cost notes and known issue 9 now quote the new pin's sweep
and name the machine; AGENTS.md gains the one-profile-per-machine rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lingrui96
lingrui96 force-pushed the track-ae655a9-expanded-sweeps branch from 2ac74d5 to 266ff4c Compare August 7, 2026 12:56
Run post reboot on an otherwise idle machine, all gates green. run.yaml
carries the machine label, chip and memory and no hostname; repo_rev is
the clean docs commit that preceded the run. Known issue 9 now cites the
measured pressure effect directly: the case-2 naive point at r = 10 cost
about 28 s per run with swap nearly full and 16 s after a reboot, same
code, same errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lingrui96
lingrui96 requested a review from shinaoka August 7, 2026 13:32
@shinaoka

shinaoka commented Aug 7, 2026

Copy link
Copy Markdown
Member

Thanks for the thorough update. I checked the branch locally:

  • cargo test --release: 19 passed
  • smoke runs for all three binaries: passed
  • all 64 raw JSON files parse successfully
  • the Markdown reports reproduce exactly from the committed raw data
  • the tensor4all-rs pin ae655a9 matches the current upstream main

The benchmark and generated results look sound. I recommend merging after four small documentation/governance fixes; no rerun is needed:

  1. Update the PR title: it says rerun mac-cpu, but this PR adds the mac-m1-8gb profile.
  2. Reconcile the full-run duration: the PR body says about 14 minutes, while the README says roughly 8 minutes.
  3. Remove the provisional AGENTS.md Pushing section from this PR, or replace it only after the maintainer has explicitly accepted the policy. AGENTS.md should contain active instructions, not a proposal awaiting veto.
  4. Remove the internal design spec or change its stale pending execution status to implemented.

My recommendations on the open questions:

  • Next case: benchmark TCI/quantics construction first, since #575 directly changes that path and the existing cases exclude construction time.
  • #571 remains open with no assignee or milestone; keep the simplett fit arm excluded.
  • I found no public issue or plan for a simplett tensor-train elementwise product; track it separately only when it blocks planned work.
  • Keep mac-m1-8gb as a machine-specific profile, but do not use its memory-bound naive timings as the cross-machine headline.
  • threads: default is acceptable for this profile. For a future official cross-machine sweep, pin a numeric thread count.
  • Keep BENCH_BOX_L=6 for now. If higher-accuracy comparisons are needed, use a finite-box analytic reference rather than only enlarging the box at fixed R.
  • The per-machine profiles, chip/memory fields, and hostname removal are good.
  • The proposed branch/PR/human-merge policy is reasonable, but should be added as an accepted maintainer policy in a separate change.

With those small cleanups, this is ready to approve.

…spec

The Pushing section was written as a proposal awaiting veto, which is not
what AGENTS.md is for: it holds active instructions. It comes out of this
change and can return as accepted maintainer policy in its own change.
The spec's status moves to implemented and its question list is replaced
by the maintainer's answers, so it reads as a record rather than an open
ask.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lingrui96 lingrui96 changed the title Track latest tensor4all-rs (ae655a9), expand default sweeps, rerun mac-cpu Track latest tensor4all-rs (ae655a9), expand default sweeps, add the mac-m1-8gb profile Aug 7, 2026
@lingrui96
lingrui96 merged commit c5ca08e into main Aug 7, 2026
1 check passed
shinaoka added a commit that referenced this pull request Aug 10, 2026
…, and the chi_out-driven case definitions

Resolution policy:
- Cargo.toml keeps the newer pin 1b9a517, which strictly contains the
  ae655a9 of PR #1 (verified as an ancestor), so it carries #574, #575 and
  #591. Cargo.lock regenerated against it.
- Case definitions and knobs keep the chi_out-driven semantics of this
  branch: BENCH_CONTRACT_TOL, the ACI scale-relative tolerance, the
  degenerate-product guard, case 4 and the arm names.
- The expanded default sweeps of PR #1 are adopted in full, BENCH_KS up to
  128 and BENCH_RS = 6, 8, 10, 12, 14, because the whole mac-cpu sweep
  stays inside five minutes at these per-cell costs on this machine.
- scripts/run_all.sh takes the hostname removal and the hardware fields of
  PR #1 and keeps the -dirty provenance logic of this branch.
- result/mac-m1-8gb is kept exactly as committed in PR #1: it is another
  machine's record at the revision its own run.yaml states, and it is not
  regenerated here.
- Known issues are unified and renumbered: the ACI item is now 9, the
  machine-bound naive timings of PR #1 are 10, and both sides' retirement
  of the ACI-pending item is kept once, as resolved at the current pin.
- CI keeps the case-3 and case-4 smokes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shinaoka added a commit that referenced this pull request Aug 10, 2026
… docs

The sweep runs the expanded defaults adopted from PR #1 (BENCH_KS up to 128,
BENCH_RS = 6, 8, 10, 12, 14) at the merged chi_out-driven case definitions, and
finishes in 263 s including the reports. run.yaml stamps the merge commit
90fbd82 as repo_rev, with the hardware fields the merged run_all.sh now writes.

What moved against the numbers quoted before the merge:

- Case 2 naive is 16 to 17 s per run at R = 12 and 14, the two points the
  expanded default sweep adds, and the naive and fit error range now runs to
  3.6e-9 with the zipup pair at 2.0e-5 to 1.1e-4.
- Case 3 reaches 1.7e-8 at R = 12 and its zipup_treetn arm spans 2.3e-1 to
  7.9e-1 over the wider sweep, still order one at every point.
- Case 4 lands on chi_in of 78, 101, 117, 140, fitting to N^0.27 against the
  0.28 of the previous sweep, so the sublinear conclusion is unchanged and the
  spread across reruns is now quoted. Its zipup_treetn arm passes 1 at N = 64,
  at 1.11, which widens the known issue 8 range to 2e-1 through 1.1.
- Case 1 zipup at the new K = 128 point measures 3.9e-6 against a 1e-5 gate,
  noted in the README so a future extension of BENCH_KS is read as the gate
  reporting truncation growth rather than as a regression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants