feat(agents): API agent (MCP + skills) on capabilities + models/loop#30
Draft
pradeepvrd wants to merge 1 commit into
Draft
feat(agents): API agent (MCP + skills) on capabilities + models/loop#30pradeepvrd wants to merge 1 commit into
pradeepvrd wants to merge 1 commit into
Conversation
pradeepvrd
commented
Jun 20, 2026
pradeepvrd
force-pushed
the
submit/4-agents-base-cli
branch
from
d789dcd to
c4a1093
Compare
pradeepvrd
force-pushed
the
submit/5-agents-api
branch
3 times, most recently
from
ddf3ce9 to
307bb21
Compare
pradeepvrd
force-pushed
the
submit/4-agents-base-cli
branch
from
43c5d8f to
1962e0d
Compare
pradeepvrd
force-pushed
the
submit/5-agents-api
branch
from
307bb21 to
915b68f
Compare
pradeepvrd
force-pushed
the
submit/4-agents-base-cli
branch
from
1962e0d to
09c113b
Compare
pradeepvrd
force-pushed
the
submit/5-agents-api
branch
from
915b68f to
885f8b8
Compare
pradeepvrd
force-pushed
the
submit/4-agents-base-cli
branch
from
09c113b to
c785a87
Compare
pradeepvrd
force-pushed
the
submit/5-agents-api
branch
from
885f8b8 to
90261d8
Compare
pradeepvrd
force-pushed
the
submit/4-agents-base-cli
branch
from
c785a87 to
4585a19
Compare
The API/MCP agent used to live in `pkg/agents/runner/api/` as a monolithic loop with the MCP client, skill discovery, and LLM-adapter dispatch intermingled; this refactors it into `devops_bench/agents/api/` (`agent.py` harness, `mcp.py`, `skills.py`) built on the shared `run_tool_loop` and the `models` layer, with MCP/skills/rules and provider/model resolved from `AgentConfig` rather than env reads. **Behavior changes** - MCP and skills are now independent gates: skills-only, MCP-only, both, or neither — previously they were coupled. - Capability bindings and provider/model flow from `AgentConfig.capabilities` instead of env reads (`AGENT_TARGET`, `BENCH_USE_MCP`); the orchestrator wires them explicitly. - The agent reuses the shared `run_tool_loop` (provider-agnostic; the caller pre-formats tools) rather than carrying its own loop. - Optional provider/MCP SDKs are imported lazily. **Bugs fixed** - MCP server commands with arguments (e.g. `uv run mcp-server`) are parsed with `shlex.split` instead of being treated as a single executable. - Token usage is mapped across the Gemini/Claude/Ollama response shapes (totals computed when absent) instead of assuming one provider's field names. - Skill files are read on a worker thread instead of blocking the async event loop.
pradeepvrd
force-pushed
the
submit/5-agents-api
branch
from
90261d8 to
073719a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The API/MCP agent used to live in
pkg/agents/runner/api/as a monolithic loop with the MCP client, skill discovery, and LLM-adapter dispatch intermingled; this refactors it intodevops_bench/agents/api/(agent.pyharness,mcp.py,skills.py) built on the sharedrun_tool_loopand themodelslayer, with MCP/skills/rules and provider/model resolved fromAgentConfigrather than env reads.Behavior changes
AgentConfig.capabilitiesinstead of env reads (AGENT_TARGET,BENCH_USE_MCP); the orchestrator wires them explicitly.run_tool_loop(provider-agnostic; the caller pre-formats tools) rather than carrying its own loop.Bugs fixed
uv run mcp-server) are parsed withshlex.splitinstead of being treated as a single executable.