Make trainer catch up on v1 self-play data#166
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This changes the trainer startup/resume path for the existing v1 model + accumulated self-play data case.
catchup_passes_over_existing_selfplaypasses over existing self-play, without the normal one-report seed/token caps.state/trainer/resume_checkpoint.ptwith model weights, optimizer state, train bucket state, SWA buffer, BN refresh batches, and catch-up counters.promote_every_new_positionsfrom 2.5M to 1M for normal candidate exports after v1 catch-up.Why
We already have a v1 model and around 1.1M paid-for self-play samples. The old restart behavior could apply the sliding replay window and normal token-bucket seed cap immediately, meaning a restarted trainer could ignore a large share of existing v1 self-play before ever producing v2. This PR makes v1 catch-up explicit: deploy it, train across all existing self-play, publish v2, then continue v3/v4/etc. on the normal rolling cadence.
The trainer summary feature is intentionally retained as observability/reporting cadence. It is not used as the candidate boundary.
Deployment notes
latest.meta.jsonwithoutpositions_at_promoteis treated as watermark 0 so legacy v1 artifacts can start catch-up.positions_at_promotestill fails because the watermark should be explicit.models/checkpoint.ptremains a last-published-model alias, not a live trainer snapshot. The new resume checkpoint isstate/trainer/resume_checkpoint.pt.Validation
uv run pytest -q-> 186 passed, 1 skippedgit diff --checkcodex review --uncommitted-> no discrete correctness issues