Preserve per-turn trace payloads

[SunnySoldier357]

Summary

• Preserve payload-derived metadata for each assistant turn when remote traces are collapsed to the longest conversation.
• Keep existing top-level payload fields as the final completion payload for compatibility.
• Add unit coverage for per-turn logprobs and router replay payloads.

Test plan

• conda run -n eval python -m pytest tests/pytest/test_tracing_utils.py -q

Made with Cursor

---

Note

Low Risk
Scoped to eval-protocol trace preprocessing when include_payloads is enabled; no auth or production inference path changes.

Overview
When remote Fireworks traces are preprocessed to a single longest conversation row, payload mode now merges each shorter trace’s completion metadata onto the matching assistant turn instead of keeping only the final turn’s data.

The default loader’s preprocess_fn still runs filter_longest_conversation, then (if include_payloads) calls _merge_payloads_into_longest_row: it copies missing logprobs onto assistant messages by turn index and builds execution_metadata.extra["assistant_turn_payloads"] from keys like completion_logprobs, routing_matrices, and routing_metadata. Top-level extra on the longest row still reflects the last completion for backward compatibility.

Unit tests in test_tracing_utils.py cover multi-turn logprobs and per-turn router replay fields.

^{Reviewed by Cursor Bugbot for commit c3690ca. Bugbot is set up for automated code reviews on this repo. Configure here.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c3690caad5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".