feat(agents): capability bindings/protocols/mixins#28
Draft
pradeepvrd wants to merge 1 commit into
Draft
Conversation
pradeepvrd
commented
Jun 20, 2026
pradeepvrd
force-pushed
the
submit/2-verification
branch
from
6592b55 to
a510df6
Compare
pradeepvrd
force-pushed
the
submit/3-agents-capabilities
branch
from
822d901 to
51db598
Compare
pradeepvrd
changed the title
pradeepvrd
force-pushed
the
submit/2-verification
branch
from
a510df6 to
9500fba
Compare
pradeepvrd
force-pushed
the
submit/3-agents-capabilities
branch
from
51db598 to
887c755
Compare
pradeepvrd
force-pushed
the
submit/2-verification
branch
from
9500fba to
2da6431
Compare
pradeepvrd
force-pushed
the
submit/3-agents-capabilities
branch
from
887c755 to
6e9a5ac
Compare
pradeepvrd
force-pushed
the
submit/2-verification
branch
from
2da6431 to
415e316
Compare
Agent capabilities used to be scattered across the `evaluate.py` path: the system instruction in `pkg/evaluator/evaluate.py` (`SYSTEM_INSTRUCTION` / `SYSTEM_INSTRUCTION_NO_MCP`), MCP server paths + hard-coded tool lists in `pkg/agents/runner/gcli.py`, and skill discovery in `pkg/agents/runner/api/`. This consolidates the three axes into `devops_bench/agents/capabilities/` as typed bindings (`AgentRules`, `McpBinding`, `SkillBinding`, `AllCapabilities`) plus structural Protocols (`SupportsRules`/`SupportsMcp`/`SupportsSkills`). No new deps. **Behavior changes** - Rules, MCP servers, and skills are now data the orchestrator binds, not strings/lists hard-coded in runner code; an agent's allowed-tools are derived from the bindings. - The three axes are independent: an agent can be granted skills without MCP (and vice versa); previously skills/MCP were entangled in the runner. - Capability support is expressed structurally (`isinstance(agent, SupportsMcp)`), so the orchestrator grants only what an agent actually consumes.
pradeepvrd
force-pushed
the
submit/3-agents-capabilities
branch
from
6e9a5ac to
1f3c053
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent capabilities used to be scattered across the
evaluate.pypath: the system instruction inpkg/evaluator/evaluate.py(SYSTEM_INSTRUCTION/SYSTEM_INSTRUCTION_NO_MCP), MCP server paths + hard-coded tool lists inpkg/agents/runner/gcli.py, and skill discovery inpkg/agents/runner/api/. This consolidates the three axes intodevops_bench/agents/capabilities/as typed bindings (AgentRules,McpBinding,SkillBinding,AllCapabilities) plus structural Protocols (SupportsRules/SupportsMcp/SupportsSkills). No new deps.Behavior changes
isinstance(agent, SupportsMcp)), so the orchestrator grants only what an agent actually consumes.