Problem
The skill auto-extraction system has three compounding defects that cause low-quality skills to accumulate and be injected into agent context without user review:
1. Extraction gate fires on trivially short sessions
The gate in `routes/chat_helpers.py` triggers when `agent_rounds >= 2 OR agent_tool_calls >= 2`. Nearly every agent interaction qualifies — a task that reads a file and writes it back (2 tool calls, 1 round) is enough. The corresponding gate in `services/memory/skill_extractor.py` uses the same OR logic. This is not selective enough to identify genuinely reusable procedures.
2. Confidence threshold mismatch creates zombie skills
`MIN_CONFIDENCE = 0.6` in `skill_extractor.py` saves skills to disk. The injection gate in `agent_loop.py` defaults to `skill_min_confidence = 0.85`. Skills with confidence 0.60–0.84 are saved but never surfaced to the agent — they accumulate as dead weight in `data/skills/`.
3. `auto_approve_skills` defaults to `True`
Extracted skills are auto-published without user review. A skill extracted from a failed or one-off session immediately becomes part of the agent's injected context on the next turn. There is no gate for human judgment before a skill goes live.
Observed result: Low-quality skills (libvirt XML config procedures, etc.) have already appeared in `data/skills/` from routine agent tasks. Each extracted skill adds to the skill index, increasing prompt token usage on every subsequent agent request.
Proposed fix
`services/memory/skill_extractor.py`:
- Raise `MIN_CONFIDENCE = 0.6` → `0.85` (align with injection floor; stop saving skills that will never be injected)
- Change gate condition: `if round_count < 2 and tool_count < 2` → `if round_count < 2 or tool_count < 3` (require rounds ≥ 2 AND tools ≥ 3 — both conditions, not either)
- Change `auto_approve_skills` pref default: `True` → `False` (skills land as drafts; user publishes from Brain > Skills)
`routes/chat_helpers.py`:
- Change outer gate: `(agent_rounds >= 2 or agent_tool_calls >= 2)` → `(agent_rounds >= 2 and agent_tool_calls >= 3)` (match the new AND logic)
Tests
New file `tests/test_skill_extraction_gate.py` — 5 async unit tests using monkeypatch (pattern: `test_skill_extractor_stray_brace.py`):
- rounds=1, tools=2 → skipped (below new combined threshold)
- rounds=2, tools=2 → skipped (tools below 3)
- rounds=2, tools=3 → proceeds to LLM call
- confidence=0.84 → dropped (below new MIN_CONFIDENCE)
- auto_approve default=False → skill status is "draft" not "published"
Upstream relevance
ROADMAP lists "Agent prompt/context bloat" as high priority. This fix reduces the volume of skills injected into context by raising the bar for what qualifies as extractable. Relates to #4520 (junk name rejection) and upstream issue #4466 (skill curation).
Branch: `fix/skill-extraction-threshold` (from `upstream-mirror`)
Upstream target: `pewdiepie-archdaemon/odysseus` dev branch
Problem
The skill auto-extraction system has three compounding defects that cause low-quality skills to accumulate and be injected into agent context without user review:
1. Extraction gate fires on trivially short sessions
The gate in `routes/chat_helpers.py` triggers when `agent_rounds >= 2 OR agent_tool_calls >= 2`. Nearly every agent interaction qualifies — a task that reads a file and writes it back (2 tool calls, 1 round) is enough. The corresponding gate in `services/memory/skill_extractor.py` uses the same OR logic. This is not selective enough to identify genuinely reusable procedures.
2. Confidence threshold mismatch creates zombie skills
`MIN_CONFIDENCE = 0.6` in `skill_extractor.py` saves skills to disk. The injection gate in `agent_loop.py` defaults to `skill_min_confidence = 0.85`. Skills with confidence 0.60–0.84 are saved but never surfaced to the agent — they accumulate as dead weight in `data/skills/`.
3. `auto_approve_skills` defaults to `True`
Extracted skills are auto-published without user review. A skill extracted from a failed or one-off session immediately becomes part of the agent's injected context on the next turn. There is no gate for human judgment before a skill goes live.
Observed result: Low-quality skills (libvirt XML config procedures, etc.) have already appeared in `data/skills/` from routine agent tasks. Each extracted skill adds to the skill index, increasing prompt token usage on every subsequent agent request.
Proposed fix
`services/memory/skill_extractor.py`:
`routes/chat_helpers.py`:
Tests
New file `tests/test_skill_extraction_gate.py` — 5 async unit tests using monkeypatch (pattern: `test_skill_extractor_stray_brace.py`):
Upstream relevance
ROADMAP lists "Agent prompt/context bloat" as high priority. This fix reduces the volume of skills injected into context by raising the bar for what qualifies as extractable. Relates to #4520 (junk name rejection) and upstream issue #4466 (skill curation).
Branch: `fix/skill-extraction-threshold` (from `upstream-mirror`)
Upstream target: `pewdiepie-archdaemon/odysseus` dev branch