fix(ppo): preserve raw KL so rollout/kl logging is correct#2114

Open

EazyReal wants to merge 4 commits into

EazyReal:fix/ppo-kl-inplace-metric

Commits on Jun 24, 2026

fix(ppo): stop corrupting the logged rollout/kl metric

EazyReal
and
claude
committed

Commits on Jun 25, 2026

fix(ppo): make KL reward tensor explicit
EazyReal
committed

Commits on Jun 29, 2026