Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
f149f47
feat: add testing-expert agent for test pyramid generation
FlySpot Apr 7, 2026
cef7f15
fix: address code quality issues in testing-expert agent
FlySpot Apr 7, 2026
edddd18
feat: extend /awos:tasks to generate test pyramid tasks per vertical …
FlySpot Apr 7, 2026
cfb2854
fix: split positive/negative test examples into separate tasks in tas…
FlySpot Apr 7, 2026
b4e12f5
feat: add /awos:qa optional full-audit command
FlySpot Apr 7, 2026
3411e5d
fix: restore testing-expert cross-reference and improve qa.md clarity
FlySpot Apr 7, 2026
be37c25
feat: add QA context templates for test registry and regression suite
FlySpot Apr 7, 2026
d2f9c07
docs: add QA pyramid agent implementation plan
FlySpot Apr 7, 2026
93ab40a
chore: remove unnecessary testing-expert command wrapper — agent is i…
FlySpot Apr 7, 2026
582c95a
docs: remove testing-expert wrapper entry from plan file table
FlySpot Apr 7, 2026
c97c91a
fix: resolve contradictions in testing-expert and qa commands
FlySpot Apr 7, 2026
3ff18fa
docs(qa): add TODO section with known limitations
FlySpot Apr 9, 2026
e44649f
Delete docs/superpowers/plans/2026-04-07-qa-pyramid-agent.md
FlySpot Apr 9, 2026
7921ccc
chore: fix prettier formatting across markdown files
FlySpot Apr 14, 2026
cc4ffb7
fix(tasks): sequential numbering, opt-out tests, ignore docs/superpowers
FlySpot Apr 14, 2026
0a3eb4e
fix(tasks): sequential numbering and opt-out test generation
FlySpot Apr 14, 2026
f913646
fix(testing-expert): move from commands/ to plugins/awos/agents/
FlySpot Apr 20, 2026
92a2d4f
fix(tasks): replace vague 'planning mode' with explicit Agent tool in…
FlySpot Apr 20, 2026
637566b
fix(tasks): clarify Agent tool context-passing and move opt-out check…
FlySpot Apr 20, 2026
d03f776
fix(tasks): rebalance examples to show layer judgment and consistent …
FlySpot Apr 20, 2026
6f110e5
fix(testing-expert): replace caller-based mode detection with conditi…
FlySpot Apr 20, 2026
729d192
fix(tasks): add missing verify sub-tasks to JWT and avatar Slice 1 ex…
FlySpot Apr 20, 2026
6b15fb1
fix(qa): remove stale testing-expert cross-reference, clarify e2e-tes…
FlySpot Apr 20, 2026
de71e7c
fix: clean up garbled e2e-tester TODO and rename mode headings to mat…
FlySpot Apr 20, 2026
5d649b7
fix: update stale 'execution mode' reference and restore step 4 forma…
FlySpot Apr 20, 2026
902d3ed
fix(testing-expert): remove remaining caller-specific references from…
FlySpot Apr 20, 2026
5222fc7
fix(tasks): use Task tool instead of Agent tool for testing-expert in…
FlySpot Apr 20, 2026
06973be
fix(installer): deploy testing-expert agent to .claude/agents/ during…
FlySpot Apr 20, 2026
cd693f7
fix(tasks): move Verify step after test sub-tasks in Slice 2 avatar e…
FlySpot Apr 20, 2026
669a1b0
fix(testing-expert): normalize positive/negative suffix format across…
FlySpot Apr 20, 2026
8e2493b
fix(template): clarify test registry maintenance attribution
FlySpot Apr 20, 2026
57d08f5
fix: delegate qa gap tests to testing-expert, fix code fence, update …
FlySpot Apr 20, 2026
144094c
feat(tasks): replace per-slice testing with Feature Testing & Regress…
FlySpot Apr 24, 2026
ca7c3c5
feat(qa): remove /awos:qa slash command — preserved as future plugin …
FlySpot Apr 24, 2026
d05f640
feat(regression): add /awos:regression command with dedup, confirmati…
FlySpot Apr 24, 2026
f442811
fix(regression): fix extraction logic, fully-processed check, and con…
FlySpot Apr 24, 2026
9482da2
feat(regression): add claude/commands wrapper for /awos:regression
FlySpot Apr 24, 2026
ad53cb0
fix(regression): strip wrapper to standard pattern — remove prose and…
FlySpot Apr 24, 2026
033553d
fix(installer): remove testing-expert auto-deploy — now hired via awo…
FlySpot Apr 24, 2026
f9b6708
fix(verify): enforce mandatory Step 3 with fallback chain — prevent s…
FlySpot Apr 24, 2026
c5ba8be
feat(template): update regression-suite with layered format; fix stal…
FlySpot Apr 24, 2026
d58b877
feat(testing-expert): remove from awos core — now lives in awos-recru…
FlySpot Apr 25, 2026
9925d0f
feat(hire): add QA complement rule — auto-suggest testing-expert alon…
FlySpot Apr 25, 2026
127851c
fix(audit): update SDD-07 to recognize Feature Testing & Regression s…
FlySpot Apr 25, 2026
1608ada
chore(tasks): remove regression files, comment out /awos:regression s…
FlySpot May 7, 2026
019e678
feat(tasks): add artifact cleanup sub-task after each implementation …
FlySpot May 7, 2026
65157bd
chore: fix prettier formatting
FlySpot May 7, 2026
2ef8817
fix: address CodeRabbit review comments
FlySpot May 11, 2026
d0b1f79
chore: fix prettier formatting
FlySpot May 11, 2026
191a9b2
fix: address review comments on hire.md and qa.md
FlySpot May 25, 2026
5245b66
remove: delete commands/qa.md
FlySpot May 25, 2026
395caa4
feat(tasks): add --no-tests / skip tests flag to suppress verificatio…
FlySpot May 25, 2026
a357178
chore: merge main into feat/qa-pyramid-agent — resolve conflicts
FlySpot May 25, 2026
4145b62
chore: fix prettier formatting in tasks.md
FlySpot May 25, 2026
ba3a38b
fix: resolve 3 consistency issues found in PR review
FlySpot May 25, 2026
2e0ed54
chore: merge main into feat/qa-pyramid-agent; apply Option A QA-agent…
May 27, 2026
3d9c043
fix(tasks): trim Feature Testing & Regression slice noise
May 29, 2026
ba6e46c
fix(verify): require real UI rendering + screenshots in docs/screensh…
May 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,3 +144,4 @@ Non-obvious rules for this repo:
- **Prefer the built-in `Explore` and `Plan` subagents** for read-heavy context-gathering. Don't have an orchestrator command read the whole codebase in its own context.
- **Skip ceremonial preambles** like "Great!", "I will now…", "All done!" — modern models trim them naturally and AWOS prompts shouldn't fight that.
- **`AskUserQuestion` belongs in core `commands/*.md`** under an `# INTERACTION` section. AWOS targets Claude Code only, so the tool is a framework default rather than a host-specific customization — don't duplicate it into the `claude/commands/*.md` wrappers.
- **Do not hard-wrap markdown prose at 80 columns.** Let paragraphs and list items flow as a single line per logical unit. Markdown renderers reflow soft-wrapped text, and 80-col wrapping inflates diffs when prose is edited. Wrapping is fine only where the line is semantically a single token (a URL, a code identifier) or inside a fenced block whose literal line breaks matter.
12 changes: 12 additions & 0 deletions commands/hire.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,18 @@ Follow this process precisely.
| `python-backend` | `fastapi-expert` | — | — |
| `aws-infra` | `terraform-pro`, `aws-deploy` | `aws-mcp` | `aws-infra-expert` |

**QA Complement Rule:**

For each primary tech role identified above, search the registry for a complementary QA/testing agent in the same pass — query with the primary technology plus terms like "testing", "QA", or "acceptance" (e.g. `"React TypeScript testing acceptance"`). The intent is to surface any specialist that can write or run tests for that stack.

Pick **one** QA agent per primary role, in this order of preference:

1. A technology-specific tester from the registry or already in `.claude/agents/` (e.g. an agent dedicated to the project's actual testing stack — pytest-focused, React-component-focused, etc.).
2. The generic `testing-expert` from the `awos-recruitment` registry if no technology-specific tester is found.
3. Otherwise, no QA agent — record the gap in the Step 7 warning table.

Do **not** hardcode tool names or runners (Playwright, Cypress, WebdriverIO, Vitest, pytest…) into the proposal. Pick a runner only after the project's actual stack is known — by reading `technical-considerations.md`, the package manifest, or any existing test configuration — and prefer whatever is already configured before suggesting a new one. Optimize for the project's testing efficiency and developer wall-clock time, not for a fixed default.

## Step 5: Install Found Components

Detect the project's package runner: prefer `bunx` if a `bun.lockb` or `bun.lock` is present in the project root, otherwise use `npx`. The commands below show both; pick one.
Expand Down
68 changes: 48 additions & 20 deletions commands/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Follow this process precisely.

1. Analyze `<user_prompt>`. If it clearly references a spec by name or index, identify the corresponding directory in `context/spec/`.
2. If the prompt is empty or ambiguous, list the spec directories that contain both `functional-spec.md` and `technical-considerations.md` and ask the user to choose. Do not proceed until a valid spec is selected.
3. **Interpret the prompt's intent on testing.** Read `<user_prompt>` and decide whether the user wants to skip generated tests (e.g. wording like "skip tests", "no tests", "prototype", "throwaway", or an explicit `--no-tests` argument). Use natural-language understanding — substring matching alone would false-positive on phrases like "don't skip tests". When uncertain, ask the user via `AskUserQuestion` before continuing. Set `SKIP_TESTS = true` only when the intent is clear. Strip any explicit `--no-tests` / `skip tests` token from the prompt before further processing.

## Step 2: Gather and Synthesize Context

Expand All @@ -53,28 +54,48 @@ Follow this process precisely.
- A runnable slice means that after the work under it is done the application can be started and used without errors, and a small piece of new functionality is visible or testable.
- Avoid horizontal, layer-based slices (e.g., "Do all database work" then "Do all API work").
- Create vertical slices — the smallest end-to-end pieces of functionality.
- A slice is valid only if its functionality is verified by the agent using real tools (curl, shell, or a browser-automation MCP if the project has one configured).
- A slice is valid only if its functionality is verified by the agent using whatever verification tool best fits the slice (curl/shell, a browser-automation MCP or CLI if the project has one configured, a unit/integration test runner, etc.). Pick by efficiency for the slice and wall-clock time — don't hardcode a tool order.
- Check that the project has the MCPs, services, and dependencies needed for testing each slice. If something is missing, instruct the user to install it.
- If a slice cannot be tested, explain why and get user approval before proceeding.
- A slice is not complete unless it is tested or the user has explicitly approved skipping the test.
- **Verification artifacts are ephemeral.** Inline an artifact cleanup step into each Verify task — screenshots, recorded videos, generated e2e scripts and any other ephemeral files produced during verification get deleted at the end of the Verify task itself. Do **not** delete artifacts from the Feature Testing & Regression slice — those are intentionally kept for the regression suite.

- **Your Thought Process for Generating the Plan:**
1. First, identify the absolute smallest piece of user-visible value from the spec. This is your **Slice 1**.
1. Identify the absolute smallest piece of user-visible value from the spec. This is **Slice 1**.
2. Create a high-level checklist item for that slice (e.g., `- [ ] **Slice 1: View existing avatar (or placeholder)**`).
3. Under that slice, create the nested tasks (database, backend, frontend) needed to implement and verify **only that slice**.
4. **For each task, assign the appropriate subagent:**
- Identify the technology or domain the task involves
4. Assign a subagent to every task:
- Identify the technology or domain the task involves.
- Enumerate the universe of available specialist subagents by inspecting the `Agent` tool's description block in your own system prompt. This is an introspection step — no tool call is required, but it is mandatory. Both kinds of agents are listed there: project-local ones (declared as files under `.claude/agents/*.md`) and plugin-provided ones. Tell them apart by the `plugin-name:` prefix on `subagent_type` — plugin-provided agents carry it (e.g. `python-development:python-pro`); project-local agents do not. The always-available built-in `general-purpose` is your fallback when no specialist matches.
- Match the task to a subagent based on:
- Technology keywords
- Task intent
- Tech stack identified in technical-considerations.md
- Append the subagent assignment using format: `**[Agent: agent-name]**` at the end of the task description
- Use `general-purpose` when no specialist clearly matches — track these for the Recommendations table
5. Next, identify the second-smallest piece of value that builds on the first. This is **Slice 2**.
6. Create a high-level checklist item and its nested tasks with subagent assignments.
7. Repeat this process until all requirements from the specification are covered.
8. For each slice's verification task, identify required MCPs/services (browser MCP, curl, database access, etc.) and note any that may be missing.
- Match the task to a subagent based on technology keywords, task intent, and the tech stack identified in `technical-considerations.md`.
- Append the assignment as `**[Agent: agent-name]**` at the end of the task description.
- Use `general-purpose` only when no specialist matches — track these for the Recommendations table.
5. Within the same slice, after the implementation tasks, add a Verify task that exercises the slice end-to-end and deletes its own verification artifacts before completing. Skip the Verify task if `SKIP_TESTS = true`.
6. Repeat steps 1-5 for each subsequent slice until all spec requirements are covered.
7. Append the **Feature Testing & Regression** slice as the final slice (skip this step entirely if `SKIP_TESTS = true`). See **Step 3a** below for how to select the QA agent and emit the slice — do not invent your own wording.
8. For each slice's Verify task, identify required MCPs/services (browser MCP, curl, database access, etc.) and note any that may be missing for the Recommendations table in Step 4.

## Step 3a: Select the QA Agent and Emit the Feature Testing & Regression Slice

Skip this step if `SKIP_TESTS = true`.

1. **Search for a QA-coded subagent** by introspecting the `Agent` tool's description block from Step 3.4. Pick the best fit using this order, but do not hardcode names — match on responsibility:
- A project-specific tester for the actual stack (e.g. `react-testing`, `pytest-tester`, a custom `acceptance-tester` in `.claude/agents/`).
- A general AWOS testing agent if installed (e.g. `testing-expert` from the `awos-recruitment` registry).
- The built-in `general-purpose` agent as the last resort.
2. **If no project-specific tester or AWOS testing agent is found,** stop and ask the user via `AskUserQuestion`. Present exactly three options:
1. **Install a testing agent now** — run `/awos:hire` to add `testing-expert` (or a more specific tester) from the registry, then re-run `/awos:tasks`.
2. **Generate the slice with `general-purpose`** — proceed and produce the Feature Testing & Regression slice, marking its tasks `**[Agent: general-purpose]**`. Flag this in the Recommendations table.
3. **Skip the Feature Testing & Regression slice** — set `SKIP_TESTS = true` for this run only; the user can re-run `/awos:tasks` later once a tester is hired.
3. **Emit the slice** using the template below. Substitute `{qa-agent}` with the agent name selected above. Substitute `N` with the next slice number. Keep the wording — downstream automations depend on this exact structure.

```md
- [ ] **Slice N: Feature Testing & Regression**

> Verifies the whole feature end-to-end against functional-spec.md, run after all implementation slices are complete.
- [ ] Read functional-spec.md acceptance criteria in full. Generate acceptance-level tests that verify the entire feature as a whole — not individual slices. Cover applicable layers (unit for pure logic, integration for service interactions, e2e for user flows) based on the project's testing stack. Write tests with RED validation (must fail before implementation is confirmed done). Annotate each test with `@spec: [spec-directory]` and `@regression` if suitable for long-term regression. **[Agent: {qa-agent}]**
- [ ] Run all generated tests. All must pass. Fix any failures before proceeding. **[Agent: {qa-agent}]**
```

- **Example of applying the rule for "User Profile Picture Upload":**
- **Bad, Horizontal Plan (DO NOT DO THIS):**
Expand All @@ -85,24 +106,31 @@ Follow this process precisely.
- `[ ] **Slice 1: Display a placeholder avatar on the profile page**`
- `[ ] Task: Add a non-functional 'ProfileAvatar' UI component that shows a static placeholder image. **[Agent: react-expert]**`
- `[ ] Task: Place the component on the profile page. **[Agent: react-expert]**`
- `[ ] Verify: Start the app, open the profile page, confirm the placeholder avatar renders, then delete any screenshots or recordings produced during the check. **[Agent: manual-qa-expert]**`
- `[ ] **Slice 2: Display the user's actual avatar if it exists**`
- `[ ] Task: Add avatar_url column to the users table via a migration. **[Agent: python-expert]**`
- `[ ] Task: Update the user API endpoint to return the avatar_url. **[Agent: python-expert]**`
- `[ ] Task: Update the 'ProfileAvatar' component to fetch and display the user's avatar_url, falling back to the placeholder if null. **[Agent: react-expert]**`
- `[ ] Task: Run the application. Use chrome MCP to connect the page in Browser. Verify that the profile page shows the correct avatar or placeholder. **[Agent: manual-qa-expert]**`
- `[ ] Verify: Run the application, drive the profile page through the available browser-automation tool (whichever the project ships — playwright-cli, cypress, the chrome MCP, etc.), confirm the correct avatar or placeholder is shown, and delete any screenshots or recordings produced during the check. **[Agent: manual-qa-expert]**`
- `[ ] **Slice 3: Feature Testing & Regression**`
> Verifies the whole feature end-to-end against functional-spec.md, run after all implementation slices are complete.
- `[ ] Read functional-spec.md acceptance criteria in full. Generate acceptance-level tests that verify the entire feature as a whole — not individual slices. Cover applicable layers (unit for pure logic, integration for service interactions, e2e for user flows) based on the project's testing stack. Write tests with RED validation (must fail before implementation is confirmed done). Annotate each test with @spec: [spec-directory] and @regression if suitable for long-term regression. **[Agent: testing-expert]**`
- `[ ] Run all generated tests. All must pass. Fix any failures before proceeding. **[Agent: testing-expert]**`

## Step 4: Present Draft and Refine

- Present the complete, vertically sliced plan with subagent assignments to the user and ask for feedback.
- Iterate until the user is satisfied (adjust, split, merge slices or tasks, or reassign subagents as needed).
- If any tasks were assigned to `general-purpose` (because no specialist exists) or verification cannot be performed (missing MCPs/services), present a table:

| Task/Slice | Issue | Recommendation |
| --------------------- | -------------------------------------------------------- | ---------------------------------------------------- |
| Slice 2: Task 3 | Assigned to `general-purpose` — no TypeScript specialist | Install `typescript-pro` agent for proper delegation |
| Slice 3: Verification | Browser MCP not available | Install browser MCP to enable UI verification |
| Task/Slice | Issue | Recommendation |
| --------------------- | ----------------------------------------------------------------------------------- | ---------------------------------------------------- |
| Slice 2: Task 3 | Assigned to `general-purpose` — no TypeScript specialist | Install `typescript-pro` agent for proper delegation |
| Slice N (QA) | Feature Testing & Regression slice uses `general-purpose` — no QA-coded agent hired | Run `/awos:hire` to install `testing-expert` |
| Slice 3: Verification | Browser MCP not available | Install browser MCP to enable UI verification |

## Step 5: File Generation

1. Write the final slice/task list to `tasks.md` in the chosen spec directory.
2. Report the saved path and the next command: `/awos:implement`.
2. If `SKIP_TESTS = true`, record a one-line note at the top of the generated `tasks.md` so that downstream commands (e.g. `/awos:verify`) can detect the choice: `<!-- skip-tests: true -->`.
3. Report the saved path and the next command: `/awos:implement`.
Loading
Loading