Skip to content

Make trainer catch up on v1 self-play data#166

Merged
k15z merged 1 commit into
mainfrom
trainer-v1-catchup-resume
May 9, 2026
Merged

Make trainer catch up on v1 self-play data#166
k15z merged 1 commit into
mainfrom
trainer-v1-catchup-resume

Conversation

@k15z

@k15z k15z commented May 9, 2026

Copy link
Copy Markdown
Owner

Summary

This changes the trainer startup/resume path for the existing v1 model + accumulated self-play data case.

  • Train v1 catch-up against the full self-play corpus until v2 exists, instead of applying the sublinear replay window immediately.
  • Use a v1 catch-up token bucket sized for catchup_passes_over_existing_selfplay passes over existing self-play, without the normal one-report seed/token caps.
  • Force the v2 candidate export once catch-up passes are complete, while deferring ordinary fresh-position candidate exports during full-corpus mode.
  • After v2, return to the normal rolling replay window and token bucket behavior.
  • Persist a resumable trainer checkpoint under state/trainer/resume_checkpoint.pt with model weights, optimizer state, train bucket state, SWA buffer, BN refresh batches, and catch-up counters.
  • Rename trainer-side model turnover helpers from promotion language to candidate publishing. Evaluation service promotion/approval is unchanged.
  • Reduce promote_every_new_positions from 2.5M to 1M for normal candidate exports after v1 catch-up.

Why

We already have a v1 model and around 1.1M paid-for self-play samples. The old restart behavior could apply the sliding replay window and normal token-bucket seed cap immediately, meaning a restarted trainer could ignore a large share of existing v1 self-play before ever producing v2. This PR makes v1 catch-up explicit: deploy it, train across all existing self-play, publish v2, then continue v3/v4/etc. on the normal rolling cadence.

The trainer summary feature is intentionally retained as observability/reporting cadence. It is not used as the candidate boundary.

Deployment notes

  • Existing v1 latest.meta.json without positions_at_promote is treated as watermark 0 so legacy v1 artifacts can start catch-up.
  • For v2+, missing positions_at_promote still fails because the watermark should be explicit.
  • models/checkpoint.pt remains a last-published-model alias, not a live trainer snapshot. The new resume checkpoint is state/trainer/resume_checkpoint.pt.

Validation

  • uv run pytest -q -> 186 passed, 1 skipped
  • git diff --check
  • codex review --uncommitted -> no discrete correctness issues

@k15z k15z merged commit d14c2d2 into main May 9, 2026
9 checks passed
@k15z k15z deleted the trainer-v1-catchup-resume branch May 9, 2026 23:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant