sync(slime #2014..#2125): 3-way merge [WIP — 45 conflict files need review]

[aoshen02]

slime → vime sync: window #2014..#2125

Syncs all 48 PRs merged in slime after cutoff 44d29ee5 (slime #2013) up to slime #2125.
Follows SOP: vllm-agent-infra/knowledge/rl/slime-to-vime-sync-sop.md

Scope: full mirror, 153 changed files. Window extended #2118→#2125 to fold in the
coding-agent harness follow-ups (#2124/#2125) to #2005 + non-float reward fix (#2121).

---

Structure: one PR, two commits

Commit 1 (current): 3-way merge, conflict markers preserved

• ours = vime@main · base = translate(slime@cutoff #2013) · theirs = translate(slime@tip #2125)
• 41 clean merges · 45 files with <<<<<<< conflict markers · 62 new-to-vime · 5 deletions

Commit 2 (pending): resolve all conflict blocks — the only commit that needs review.

---

Status: DRAFT — waiting on conflict resolution

Conflict markers are intentionally preserved as the review queue. The agent maintains
agent_run/reports/QUESTIONS.md (43 questions) + an interactive decision tracker.

Decisions needed (3 genuine + 6 confirm):

Tier	Items
Decide	README new chapters (Q26) · coding_agent_rl env-var + harness (Q37) · tau-bench user sim (Q43)
Confirm	Dockerfile base (keep ours) · vllm.patch (keep ours 22L) · top_p.patch (delete) · GHA workflows (delete) · review skill (delete) · version.txt (accept)
Settled	agent adapters = slime #2005 TrajectoryManager refactor (accept + fix vLLM endpoints) · vllm.srt.constants not in vLLM (delete imports) · flag/API diffs keep ours

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #33343856 | 📁 Comparing 193d2b8 against latest (9a2427b)

  🔍 Preview build  

40 files changed · + 13 added · ± 27 modified

+ Added

• advanced/delta-weight-sync.html
• advanced/external-rollout-engines.html
• advanced/low-precision.html
• advanced/observability.html
• examples/deepseek-r1.html
• examples/glm4-9B.html
• examples/glm4.7-30B-A3B.html
• examples/glm4.7-355B-A32B.html
• examples/glm5.2-744B-A40B.html
• examples/qwen3-4b-base-openhermes.html
• and 3 more...

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/fault-tolerance.html
• advanced/megatron-config.html
• advanced/pd-disaggregation.html
• advanced/reproducibility.html
• advanced/speculative-decoding.html
• advanced/vllm-config.html
• and 17 more...

[gemini-code-assist]

Code Review

This pull request migrates the codebase from the vime framework (using vLLM) to the slime framework (using SGLang as the default rollout backend), updating all documentation, Docker configurations, example scripts, and tests accordingly. It also introduces advanced features such as delta weight sync, external rollout engines, and new model recipes (e.g., GLM-5.2 744B-A40B). The review feedback highlights several critical bugs and improvement opportunities across the codebase, including a falsy check in the placement group layout calculation when rollout_num_gpus is zero, a JSON parsing issue in the retool example caused by unsafe newline replacement, a Dockerfile anti-pattern of split apt update and apt install commands, a logic error in the retool reward calculation that could penalize correct tool usage, a missing abort status handler in the multi-agent system, a potential TypeError when accessing non-dictionary labels in the search-r1 example, and potential data loss and status finalization issues in the multi-turn VLM rollout.

[gemini-code-assist]

When rollout_num_gpus is explicitly set to 0, the placement group should not allocate any GPUs for rollout (i.e., the expected layout should be (16, 0)). The current expectation of (16, 16) suggests a bug in _get_placement_group_layout where args.rollout_num_gpus is checked using a falsy check (e.g., args.rollout_num_gpus or default) instead of an explicit is None check, causing 0 to incorrectly fall back to the default value.

Suggested change

	pytest.param({"rollout_num_gpus": 0}, (16, 16), id="zero_rollout_gpus"),
	pytest.param({"rollout_num_gpus": 0}, (16, 0), id="zero_rollout_gpus"),

[gemini-code-assist]

Replacing all newlines with \\n via json_str.replace("\n", "\\n") is highly problematic because it replaces formatting newlines outside of string literals (e.g., between keys and values in pretty-printed JSON), which makes the JSON invalid and causes json.loads to fail. To safely allow unescaped newlines and other control characters inside string values, use json.loads(json_str, strict=False) instead.

Suggested change

	# Replace newlines in string values with \n
	json_str = json_str.replace("\n", "\\n")
	tool_call_data = json.loads(json_str)
	tool_call_data = json.loads(json_str, strict=False)

[gemini-code-assist]

Running apt update and apt install in separate RUN instructions is an anti-pattern in Dockerfiles because of how layer caching works. If the apt update layer is cached, a subsequent build may not fetch the latest package lists, potentially causing the apt install step to fail or install outdated packages. They should be combined into a single RUN instruction, and the package lists should be cleaned up afterwards to keep the image size small.

RUN apt-get update && apt-get install -y \
    nvtop \
    rsync \
    dnsutils \
    prometheus \
 && rm -rf /var/lib/apt/lists/*

[gemini-code-assist]

There is a logic error in the reward calculation when tool_call_reward is positive but the original result["score"] is already better than -0.6 (e.g., -0.5). In this case, min(-0.6, result["score"] + tool_call_reward) will actually reduce the score to -0.6, penalizing the model for calling tools. The reward should only be capped at -0.6 if it is an improvement, or we should ensure we don't decrease a score that is already better than -0.6.

Suggested change

	if result["score"] < 0:
	tool_call_reward = (num_turns - 2) / 2 * 0.1
	result["score"] = min(-0.6, result["score"] + tool_call_reward)
	if result["score"] < 0:
	tool_call_reward = (num_turns - 2) / 2 * 0.1
	if tool_call_reward > 0:
	result["score"] = min(-0.6, result["score"] + tool_call_reward)
	else:
	result["score"] = result["score"] + tool_call_reward

[gemini-code-assist]

The case "abort" is currently commented out. If the inference engine finishes with an "abort" reason, the sample's status will not be updated to ABORTED, which could cause the training pipeline to process incomplete or empty trajectories. This case should be uncommented.

Suggested change

	# case "abort":
	# sample.status = Sample.Status.ABORTED
	case "abort":
	sample.status = Sample.Status.ABORTED

[gemini-code-assist]

Accessing sample.label["ground_truth"] directly assumes that sample.label is always a dictionary. However, in many datasets (and as seen in other rollout files), sample.label can be a raw string. To prevent a TypeError at runtime, use a defensive check to handle both dictionary and string labels.

Suggested change

	ground_truth=sample.label["ground_truth"],
	ground_truth=sample.label.get("ground_truth", sample.label) if isinstance(sample.label, dict) else sample.label,

[gemini-code-assist]

In _update_multimodal_state, if a key in obs_multimodal_inputs is not already present in sample.multimodal_inputs, it is silently ignored. This can lead to losing new multimodal input keys (such as "videos" or other custom keys) introduced in subsequent turns. We should initialize the key in sample.multimodal_inputs if it is missing.

Suggested change

	if (
	key in sample.multimodal_inputs
	and isinstance(sample.multimodal_inputs[key], list)
	and isinstance(val, list)
	):
	sample.multimodal_inputs[key].extend(val)
	if (
	key in sample.multimodal_inputs
	and isinstance(sample.multimodal_inputs[key], list)
	and isinstance(val, list)
	):
	sample.multimodal_inputs[key].extend(val)
	elif key not in sample.multimodal_inputs:
	sample.multimodal_inputs[key] = list(val) if isinstance(val, list) else val

[gemini-code-assist]

Checking if sample.status is None: to finalize the sample status to COMPLETED might miss cases where sample.status is initialized to Sample.Status.PENDING (which is the default for a new Sample). To be safe and ensure the sample is correctly marked as completed, check for both None and PENDING.

Suggested change

	if sample.status is None:
	if sample.status in (None, Sample.Status.PENDING):

[aoshen02]