From 59f9fceeed81b0df5d41e140816550018e73a338 Mon Sep 17 00:00:00 2001 From: oleksii-shevchuk Date: Wed, 8 Jul 2026 02:00:37 +0300 Subject: [PATCH] feat(spec): raise completeness and quality of /awos:spec output MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Layer four improvements onto the write-first structure from #141: - Step 3: elicit the boundary/error behavior the user sees, and require at least one acceptance criterion covering the failure path. - Step 4: add a Language and Ambiguity self-review that turns vague, unmeasurable wording ("fast", "user-friendly") into user-perceivable terms or a [NEEDS CLARIFICATION] marker, plus a Definition of Done sub-block (no vague wording remains; every requirement carries a criterion) — self-checks only, with no user turn before the write. - Step 6: when resolving each [NEEDS CLARIFICATION] marker post-save, offer the assumption the model would otherwise make as the recommended first AskUserQuestion option. The file is still written first (Step 5), and an unattended run leaves the [NEEDS CLARIFICATION] marker in place per main's deliverable-write contract — no **Assumption:** substitution before saving. Cross-section references use bold names instead of the ambiguous § symbol. Add three Layer-1 lint tests pinning the Definition of Done, the ambiguity check, and the boundary/error rules. Co-Authored-By: Claude Opus 4.8 Co-Authored-By: Alex Malakhov Co-Authored-By: Pavlo Samoilov --- commands/spec.md | 16 ++++++--- tests/lint-prompts.test.js | 73 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 85 insertions(+), 4 deletions(-) diff --git a/commands/spec.md b/commands/spec.md index b0359689..37c16b68 100644 --- a/commands/spec.md +++ b/commands/spec.md @@ -88,6 +88,7 @@ Your first goal is to determine the **topic** - the single, specific feature or 2. **Functional Requirements (The "What"):** - Ask the user to describe what needs to be done from a user's perspective. - For every piece of information the user gives you, think like a tester and clarify ambiguities. If the user answers in technical terms, rewrite the information into plain, user-facing language before including it in the spec. + - Where applicable, capture the boundary and error behavior the user sees: what error message appears, what limits exist (file size, format, count), and what happens when the action fails. - If the user says: "The user needs to be able to upload a profile picture." - Probe with clarifying questions like: "Great. Let's break that down. What file formats should be allowed (e.g., JPG, PNG)? Is there a maximum file size? What should happen after the upload is successful? What specific error message should the user see if it fails?" — for any that stay unanswered, leave a `[NEEDS CLARIFICATION: …]` marker rather than stopping. - If information is missing, mark every unresolved detail with `[NEEDS CLARIFICATION: your specific question]` directly in the draft. Example: "The user should see an error message. [NEEDS CLARIFICATION: What should the exact text of the error message be?]" @@ -96,8 +97,9 @@ Your first goal is to determine the **topic** - the single, specific feature or - After clarifying a requirement, turn it into a concrete, testable acceptance criterion. - Acceptance criteria must read as manual QA test scripts that a non-developer could execute. Describe only what is visible on screen and what the user does — never reference internal system behavior. - Each acceptance criterion follows the same three-part shape as the example below: a precondition (Given), a user action (When), and a visible outcome (Then). Include Given only when the precondition affects the outcome. - - If any `[NEEDS CLARIFICATION: …]` markers remain on the parent requirement in §Functional Requirements, ask clarifying questions and resolve the markers before writing acceptance criteria. - - If a clarifying answer reveals a constraint or detail that belongs to the parent requirement (not just the acceptance criterion), update the requirement statement in §Functional Requirements before continuing. The requirement and its acceptance criteria must agree on level of detail. + - If any `[NEEDS CLARIFICATION: …]` markers remain on the parent requirement in **Functional Requirements**, ask clarifying questions and resolve the markers before writing acceptance criteria. + - If a clarifying answer reveals a constraint or detail that belongs to the parent requirement (not just the acceptance criterion), update the requirement statement in **Functional Requirements** before continuing. The requirement and its acceptance criteria must agree on level of detail. + - For requirements that capture boundary or error behavior, include at least one acceptance criterion covering the failure path (e.g., "When the user uploads a file larger than 5MB, then they see: 'File too large. Maximum size is 5MB.'"). - Example Statement: "Okay, I've captured that. So a clear acceptance criterion would be: 'Given the user is on their profile page, when they upload a PNG file smaller than 5MB, then the new picture appears on their profile and a 'Success' message is shown.' Is that correct?" 4. **Scope and Boundaries:** @@ -106,9 +108,15 @@ Your first goal is to determine the **topic** - the single, specific feature or - Focus only on clarifying boundaries within the current **topic** itself. - Example: "To keep this focused on [your topic], what related aspects should we explicitly not include? For example, should we include [specific feature within this topic]?" -### Step 4: Self-Review (Language Check) +### Step 4: Self-Review (Language and Ambiguity Check) - Before presenting to the user, re-read the entire draft end-to-end. For every sentence, ask: "Would this make sense to someone who has never seen the codebase?" Replace any developer-facing language with plain, non-technical wording in the same language the user is using. Remove any references to internal system behavior, code, or architecture that slipped in. +- Then re-read **Functional Requirements** — both the requirement statements and their acceptance criteria — for vague or unmeasurable wording: words like "fast", "user-friendly", or "as appropriate" that a tester could not verify. Make each one concrete in user-perceivable terms (e.g., "the search feels fast" becomes "search results appear within 2 seconds"), or, if the user has not decided the specific value yet, replace it with a `[NEEDS CLARIFICATION: …]` marker so it is resolved in Step 6, after the spec is saved. Quantify only where the user would notice the difference — do not force a number onto every sentence; narrative sections like **Overview and Rationale** may stay qualitative. + +**Definition of Done.** A self-review, not an approval gate — the file is still written at the end of the process. Confirm the draft meets both: + +1. **No vague wording remains in requirements or acceptance criteria.** The check above already resolved each vague term — made it concrete in user-perceivable terms or converted it to a `[NEEDS CLARIFICATION: …]` marker; confirm none slipped through, and if one did, resolve it per the check above. Any term converted to a marker is resolved with the user in Step 6, or left in place in an unattended run. +2. **Every requirement has at least one acceptance criterion.** Confirm that each functional requirement in **Functional Requirements** carries at least one acceptance criterion in the When/Then shape (Given optional). If a requirement has none, write one for it before saving. ### Step 5: File Generation @@ -118,5 +126,5 @@ Your first goal is to determine the **topic** - the single, specific feature or ### Step 6: Final Review and Recommend Next Step -1. Present the saved specification and ask the user to review it for inaccuracies or missing details. Resolve any `[NEEDS CLARIFICATION: …]` markers with them, apply edits, and re-save. If no answer comes (e.g. an unattended `claude -p` run), leave the markers in place; the user — or `/awos:tech` — can resolve them later. +1. Present the saved specification and ask the user to review it for inaccuracies or missing details. Resolve each `[NEEDS CLARIFICATION: …]` marker with them via `AskUserQuestion`, offering the assumption you would otherwise make as the recommended first option (with a free-text option for open-ended markers); fold each answer back into the relevant requirement and its acceptance criteria, then re-save. If no answer comes (e.g. an unattended `claude -p` run), leave the markers in place; the user — or `/awos:tech` — can resolve them later. 2. Report the saved path and the next command: `/awos:tech`. diff --git a/tests/lint-prompts.test.js b/tests/lint-prompts.test.js index 85f217d2..36f2df2b 100644 --- a/tests/lint-prompts.test.js +++ b/tests/lint-prompts.test.js @@ -946,3 +946,76 @@ test('context/ references in prompts are internally consistent', () => { ); } }); + +test('commands/spec.md has a pre-write Definition of Done checklist', () => { + // The spec command runs a Definition of Done self-review inside Step 4, + // before the file is written: confirm no vague wording remains and every + // requirement carries an acceptance criterion. It is a self-review, not an + // approval gate — the file is still written. Any `[NEEDS CLARIFICATION]` + // marker is resolved with the user post-save in Step 6 (offering the + // assumption as the recommended first option), or left in place in an + // unattended run; the Definition of Done itself never asks the user a + // question. The behavioral proof — no raw markers in the produced + // functional-spec.md, every requirement carries a criterion — lives in + // awos-qa. This lint only pins that the instruction text is present, so + // the contract can't be silently dropped from the prompt later. + const body = readUtf8(path.join(commandsDir, 'spec.md')); + assert.ok( + /Definition of Done/i.test(body), + 'commands/spec.md must declare a "Definition of Done" pre-write checklist' + ); + assert.ok( + body.includes('Every requirement has at least one acceptance criterion'), + 'commands/spec.md Definition of Done must require every functional requirement to carry at least one acceptance criterion before saving' + ); + assert.ok( + /No vague wording remains/i.test(body), + 'commands/spec.md Definition of Done must gate on no vague wording remaining in requirements or acceptance criteria' + ); + assert.ok( + body.includes( + 'offering the assumption you would otherwise make as the recommended first option' + ), + 'commands/spec.md Step 6 must resolve each [NEEDS CLARIFICATION] marker post-save via AskUserQuestion, offering the assumption you would otherwise make as the recommended first option' + ); +}); + +test('commands/spec.md self-review checks for vague, unmeasurable wording', () => { + // The Step 4 self-review must hunt weasel words ("fast", "user-friendly", + // "as appropriate") in requirements and acceptance criteria, and either + // make them concrete in user-perceivable terms or convert them to a + // [NEEDS CLARIFICATION] marker that Step 6 resolves with the user + // post-save (or leaves in place in an unattended run). The behavioral + // proof — no unverifiable wording in the produced functional-spec.md — + // lives in awos-qa. This lint only pins that the instruction text is + // present, so the contract can't be silently dropped from the prompt later. + const body = readUtf8(path.join(commandsDir, 'spec.md')); + assert.ok( + /vague or unmeasurable wording/i.test(body), + 'commands/spec.md self-review must scan requirements and acceptance criteria for vague or unmeasurable wording' + ); + assert.ok( + /"user-friendly"/.test(body), + 'commands/spec.md self-review must name concrete weasel-word examples (e.g. "user-friendly") so the model knows what to hunt' + ); + assert.ok( + body.includes('Make each one concrete in user-perceivable terms'), + 'commands/spec.md Step 4 self-review rule must instruct making each vague term concrete in user-perceivable terms (not technical metrics), or converting it to a [NEEDS CLARIFICATION] marker' + ); +}); + +test('spec.md captures boundary/error behavior as rules in items 2 and 3', () => { + // both sentences must be present — item 2 tells the model what to + // elicit; item 3 closes the loop by requiring failure-path criteria for them. + const body = readUtf8(path.join(commandsDir, 'spec.md')); + assert.ok( + body.includes('boundary and error behavior the user sees'), + 'spec.md Step 3 item 2 must contain the boundary/error elicitation rule' + ); + assert.ok( + body.includes( + 'at least one acceptance criterion covering the failure path' + ), + 'spec.md Step 3 item 3 must require failure-path criteria for boundary/error requirements' + ); +});