fix: align skill parameters with actual MCP tool schemas

[luis5tb]

Summary

Skills documented wrong parameter names that don't exist in the MCP tool schemas. The LLM followed skill instructions, sent invalid params (e.g., severity="Critical"), and the MCP server rejected them — causing the Error calling tool 'get_cves' failures in production.

Root cause: parameter names in skills were guessed, not verified against the actual MCP tool schemas. Production logs with AGENT_LOGGING_DETAIL=detailed revealed the actual call args, and the schema debug logging confirmed the real parameter names.

Parameter corrections

Tool	Skill had (wrong)	MCP schema has (correct)
vulnerability__get_cves	severity (Critical/Important)	impact (numeric: 7=Critical, 5=Important, 4=Moderate, 2=Low)
vulnerability__get_cves	affecting (boolean)	advisory_available (string "true")
vulnerability__get_cves	known_exploit (boolean)	known_exploit (string "true"/"false")
inventory__list_hosts	limit / offset	per_page (default 10) / page (starts at 1)
inventory__list_hosts	operating_system	does not exist
vulnerability__get_system_cves	severity, status, remediation	only system_uuid, filter_, limit, offset, sort

Changes

Skills modified (5):

• tool-invocation-rules v1.6 — correct parameter names, types, enum values; recommended default argument sets for common queries; string-typed boolean guidance; pushy skill description for reliable loading
• multi-step-workflows v1.2 — updated workflows to use impact, advisory_available, system_uuid
• response-formatting v1.1 — added Synopsis + CVSS Score to CVE tables, sort hosts by display_name
• efficient-counting v1.1 — corrected per_page/page for inventory counting
• error-handling v1.2 — corrected retry example to use impact="7" and advisory_available="true"

Debug logging added:

• schema_sanitizer.py — logs all tool parameter types and $ref usage at startup
• agent.py — logs each skill name and version at startup for deployment verification

Eval workspace:

• 3 eval cases (IDs 4-6): multi-severity CVEs, advisor recommendations, system listing
• iteration-5 results: 16/17 (94.1%) vs baseline 11/17 (64.7%) = +29.4%
• 5 discriminating assertions, 0 regressions

Test plan

• Deploy to staging and test: "Show me the top critical and important CVEs affecting my systems"
• Verify single vulnerability__get_cves call with sort="-cvss_score", advisory_available="true", impact="5,7" (no severity or affecting param)
• Test "List my registered systems" — verify per_page, sorted by order_by="display_name"
• Test "What are the top advisor recommendations?" — verify impacting="true" (string, not boolean), sort="-total_risk"

🤖 Generated with Claude Code

[dmartinol]

/LGTM

[dmartinol]

Review: parameter name corrections look good — but 4 issues remain

Great direction on this PR. The core fixes (severity→impact, affecting→advisory_available, limit/offset→per_page/page) are all confirmed correct against the live MCP schema. However, there are 4 bugs that need addressing before merge.

---

Bug 1 — tool-invocation-rules/SKILL.md: wrong impact numeric values for Moderate and Low

The skill documents "3" as Moderate and "1" as Low, but the actual MCP schema for vulnerability__get_cves maps:

(0, 'NotSet'), (1, 'None'), (2, 'Low'), (3, 'Medium'), (4, 'Moderate'),
(5, 'Important'), (6, 'High'), (7, 'Critical')

The default value "1,2,4,5,7" in the schema confirms this: Moderate=4, Low=2. If the agent follows the skill and sends impact="3" for "Moderate", it gets Medium CVEs instead.

Fix — in tool-invocation-rules/SKILL.md, update the impact description:

-`impact` (string — comma-separated numeric impact IDs: `"7"` for Critical,
-`"5"` for Important, `"3"` for Moderate, `"1"` for Low; combine as `"5,7"`
-for Important+Critical),
+`impact` (string — comma-separated numeric impact IDs: `"7"` for Critical,
+`"5"` for Important, `"4"` for Moderate, `"2"` for Low; combine as `"5,7"`
+for Important+Critical),

Same fix in the "Multi-impact queries" section at the bottom of the same file:

-For example, to get both Critical and Important CVEs: `impact="5,7"`.
+For example, to get both Critical and Important CVEs: `impact="5,7"`.
+For Moderate and above: `impact="4,5,7"`. For all actionable severities: `impact="2,4,5,7"`.

---

Bug 2 — evals.json Eval ID 4: wrong impact values in expected_output

The new eval asserts the correct call uses '3,4' for Important+Critical, but per the schema Important=5, Critical=7.

Fix — in adk-skills-workspace/evals/evals.json, Eval ID 4:

-"expected_output": "Should use vulnerability__get_cves with the correct MCP parameter names: impact (numeric string like '3,4' for Important+Critical), sort=-cvss_score, advisory_available='true'. ..."
+"expected_output": "Should use vulnerability__get_cves with the correct MCP parameter names: impact (numeric string like '5,7' for Important+Critical), sort=-cvss_score, advisory_available='true'. ..."

And the corresponding assertion description:

-{"name": "uses_impact_not_severity", "description": "Uses the 'impact' parameter (numeric string like '3,4') instead of the nonexistent 'severity' parameter..."}
+{"name": "uses_impact_not_severity", "description": "Uses the 'impact' parameter (numeric string like '5,7' for Important+Critical, or '7' for Critical-only) instead of the nonexistent 'severity' parameter..."}

---

Bug 3 — error-handling/SKILL.md: severity=Critical still used as example

The oversized-response retry section was updated to v1.1 but still contains the old wrong parameter:

"Add filters: Apply severity, date range, status, or other filters to narrow the result set (e.g., severity=Critical, status=Applicable)."

severity is the exact wrong parameter this PR removes everywhere else.

Fix — in src/lightspeed_agent/core/skills/error-handling/SKILL.md:

-2. **Add filters**: Apply severity, date range, status, or other filters to narrow
-   the result set (e.g., `severity=Critical`, `status=Applicable`).
+2. **Add filters**: Apply impact level, date range, or other filters to narrow
+   the result set (e.g., `impact="7"` for Critical-only, `advisory_available="true"`).

---

Bug 4 — PR test plan: uses affecting=true (the old wrong parameter)

In the PR description's test checklist:

"Verify single vulnerability__get_cves call with sort=-cvss_score, affecting=true (no severity param)"

affecting does not exist on get_cves — this is the parameter name the PR is fixing. The test plan should verify the correct form.

Fix — update the test plan item in the PR description:

-- [ ]  Verify single `vulnerability__get_cves` call with `sort=-cvss_score, affecting=true` (no severity param)
+- [ ]  Verify single `vulnerability__get_cves` call with `sort="-cvss_score", advisory_available="true"` (no `severity` or `affecting` param)

---

The impact value confusion (bugs 1 & 2) is the most impactful — it would cause the agent to silently return wrong severity-tier results without any tool error, making it hard to catch in testing.

[dmartinol]

/lgtm all issues fixed and by pre-injecting STRICT skills into the system prompt at startup you addressed the root cause: Gemini was consistently skipping load_skill for tool-invocation-rules, meaning the correct parameter names were never in context at all.
👍