fix(ppo): preserve raw KL metric tensor

[EazyReal]

Port of THUDM/slime#2114 for miles' refactored advantage helper.

Summary:

• build PPO token-level rewards out of place instead of mutating the kl tensor list
• preserve the raw approximate KL tensor used by rollout/kl logging
• add a CPU test directly on compute_advantages

Validation:

• uv run --with pytest --with torch --with numpy pytest --confcutdir=tests/fast/backends/training_utils/loss tests/fast/backends/training_utils/loss/test_ppo_kl_metric.py -q -> 1 passed

[gemini-code-assist]

Code Review

This pull request prevents the in-place mutation of the input kl list elements in compute_advantages by creating a new token_level_rewards variable instead of modifying the elements in-place. Additionally, a unit test is added to verify that the raw KL metric is preserved. I have no feedback to provide as there are no review comments.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.