v0.6.0

First feature release since the initial public release (v0.5.0).

Added

• New benchmark tasks & datasets:
  • GSM8K — 8-shot base-model task (#1) and DeepSeek-Math-aligned 0-shot chat-model task (#29).
  • TheoremQA — k-shot base-model task (#3).
  • HumanEval — 0-shot base-model task (#6).
  • CMMLU — few-shot base-model task (#10).
  • MBPP — few-shot base-model task (#12).
  • IFBench — few-shot base-model task (#13).
  • LiveCodeBench — few-shot base-model code-generation task (#14).
  • OpenBookQA — k-shot generative task (#19).
  • AIME 2026 and HMMT Feb 2026 — MathArena-aligned (#16); HMMT Feb 2025 and IMO-AnswerBench (#22).
  • CLP — eval mode and naming category (#23).
• SglangGenModel — echoed-input logprobs via the SGLang /generate endpoint (#21).
• stratified_sample dataset op (#7).
• Dataset source integrity: pinned HF revisions and checksummed URL datasets, enforced in preflight (#8).
• --resume now tolerates throughput-only (scheduling) config diffs (#4).
• GitHub Actions CI pipeline and import-time dependency hardening (#2).

Fixed

• IMO-AnswerBench: normalize during answer extraction, verbatim grader; promoted to stable (#28).
• Pass the gold answer first to math_verify.verify (#18, #20).
• Dataset integrity check compares on-the-wire bytes so gzipped responses are not falsely flagged (#17).

Changed

• Renamed the select dataset op to slice (#7).
• Sanitize CI check now detects hardcoded absolute paths and scans only tracked files (#5).

Full Changelog: v0.5.0...v0.6.0