feat(models): run_tool_loop shared turn-loop primitive#26
Draft
pradeepvrd wants to merge 1 commit into
Draft
Conversation
pradeepvrd
force-pushed
the
submit/1-models-loop
branch
2 times, most recently
from
88c9939 to
2827366
Compare
The agent turn-loop used to be inline in `pkg/agents/runner/api/api.py` (the `run_api_agent` loop, coupled to MCP/skill tools); this extracts it into a shared, provider-agnostic `devops_bench/models/loop.py` — `run_tool_loop` over a neutral tool-dispatch seam — reused by the API agent and the chaos agent. No new deps. **Behavior changes** - An explicit `max_turns` cap is enforced (and logged when hit) instead of looping until the model stops on its own. - Tool-dispatch errors propagate to the caller through the dispatcher seam instead of being swallowed and appended as error strings inside the loop. - Returns a typed `LoopResult` (response, contents, final_text, latency, tools_used) instead of a dict; latency uses `time.monotonic()`. **Bugs fixed** - The previous loop had no turn cap and could run unbounded against a misbehaving model. - The model's final summary was dropped when a tool call landed on the last turn; the text is now captured every turn.
pradeepvrd
force-pushed
the
submit/1-models-loop
branch
from
2827366 to
49f0968
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The agent turn-loop used to be inline in
pkg/agents/runner/api/api.py(therun_api_agentloop, coupled to MCP/skill tools); this extracts it into a shared, provider-agnosticdevops_bench/models/loop.py—run_tool_loopover a neutral tool-dispatch seam — reused by the API agent and the chaos agent. No new deps.Behavior changes
max_turnscap is enforced (and logged when hit) instead of looping until the model stops on its own.LoopResult(response, contents, final_text, latency, tools_used) instead of a dict; latency usestime.monotonic().Bugs fixed