Skip to content

Improve training observability and next-run defaults#161

Merged
k15z merged 1 commit into
mainfrom
codex/postmortem-observability-next-run
Apr 23, 2026
Merged

Improve training observability and next-run defaults#161
k15z merged 1 commit into
mainfrom
codex/postmortem-observability-next-run

Conversation

@k15z

@k15z k15z commented Apr 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • add an April 2026 postmortem documenting the failed v1->v5 run and the evidence behind the main failure hypotheses
  • persist immutable trainer export metadata and version-scoped promotion summaries, and keep trainer JSONL logs durable in S3
  • switch the next run to 1200-sim / 10x192 / full-search-only self-play, keep bootstrap imitation, but remove imitation from replay after bootstrap
  • make evaluation collection fixed-budget so candidate comparisons stay comparable even after an early provisional SPRT result
  • update tests that were pinned to the old defaults

Details

  • write models/versions/{N}.meta.json and state/runs/v{N}/promotion_summary.json for each export
  • include metadata links in latest.meta.json and approved.meta.json
  • continue collecting gate and benchmark games until the fixed budgets are complete before final promote/reject decisions
  • add trainer artifact tests for immutable metadata and promotion summaries
  • document the April run in postmortems/POSTMORTEM_APRIL_2026.md

Validation

  • uv run pytest training
  • cargo test
  • git diff --check

@k15z k15z merged commit 80fbb25 into main Apr 23, 2026
9 checks passed
@k15z k15z deleted the codex/postmortem-observability-next-run branch April 23, 2026 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant