diff --git a/CHANGELOG.md b/CHANGELOG.md index 4125433..9686a3f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,18 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/). +## [0.66.0] - 2026-06-12 + +LLM 性能監査(#121)P3 の最終弾(#116)。skills の一律強制を縮小し内容の鮮度を更新した。 + +### Changed +- **tdd-workflow(#116)**: 「全作業(バグ修正・リファクタ含む)に TDD + 80% カバレッジを一律強制」を「TDD の明示要求時、またはカバレッジ要件のある新機能開発」に限定。日常的なバグ修正・リファクタは「変更した挙動に focused なテスト」(rules/testing.md と整合)。絶対表現(ALWAYS / not optional)をワークフロー内デフォルトに緩和、80% は「プロジェクト基準に合わせた目安」へ。npm 固定コマンドをスタック別例示化、E2E は Web UI 限定と明記、references は JS/TS 向けと注記 +- **verification-loop(#116)**: 6 観点とレポート様式は維持しつつ、固定コマンド + `| tail -20` 等の出力打ち切り指定を例示に降格・削除。Phase 5 の `grep "sk-"` 式 secret 検査を「gitleaks/trufflehog → security-review skill → 最低限 diff 追加行の確認」に置換(リポジトリ全体 grep で PASS を出さない)。実行不能な「15 分ごとに検証」「mental checkpoint」の Continuous Mode セクションを削除(定期実行は hook / /loop の領分と明記) +- **eval-harness(#116)**: 対話セッションでは測定不能な pass@k / pass^k 指標の手書き記録と Model-Based Grader の 1-5 自己採点を削除。「PASS (pass@2)」は「PASS (attempts: 2)」に改名(再試行回数として有用)。Model Grader はサブエージェントによる YES/NO チェックリスト + PASS/FAIL に置換。「本当に pass@k を測るなら headless 自動実行スクリプトで」と注記。成功条件の事前定義・Code-Based Grader・Eval Storage は維持 +- **prompt-patterns(#116)**: 旧世代の「ultrathink」キーワード詠唱機構の説明を「Deep Reasoning」(Plan Mode / harness の thinking・effort 制御 / 自然言語で深さを促す)に置換。Phase-wise Gated Plan の「Estimated context usage」(モデルに実測手段がなかった時代の項目)を「Rollback plan if verification fails」に差し替え +- **frontend-patterns(#116)**: メモ化 3 行(useMemo/useCallback/React.memo)を React Compiler 未採用プロジェクト条件付きに変更 + Compiler 採用時の注記。手書き useQuery フック(onSuccess/onError 形・依存配列の無限再フェッチ footgun 含む)を TanStack Query / SWR の最小例に置換。Framer Motion → motion 改称(`import from 'motion/react'`) +- **project-guidelines-example(#116)**: 例示コードの `claude-sonnet-4-5-20250514`(存在しない疑いのある日付サフィックス付き旧 ID)を `claude-opus-4-8` + 「最新 ID は公式 docs で確認」コメントに修正 + ## [0.65.0] - 2026-06-12 LLM 性能監査(#121)P3 の第 2 弾(#115)。commands の低優先掃除(ネイティブ重複・呪文・ハードコードの除去)。 diff --git a/agents/tdd-guide.md b/agents/tdd-guide.md index a739b07..b4fce24 100644 --- a/agents/tdd-guide.md +++ b/agents/tdd-guide.md @@ -1,6 +1,6 @@ --- name: tdd-guide -description: Test-driven development guide. Use when adding behavior, fixing bugs, or refactoring code where tests should describe the expected contract. +description: Test-driven development guide. Use when the user explicitly requests TDD or tests-first development, or when building new behavior where test coverage is a stated requirement. tools: Read, Write, Edit, Bash, Grep model: sonnet --- diff --git a/skills/eval-harness/SKILL.md b/skills/eval-harness/SKILL.md index 09a8fd3..9c203db 100644 --- a/skills/eval-harness/SKILL.md +++ b/skills/eval-harness/SKILL.md @@ -14,7 +14,6 @@ Eval-Driven Development treats evals as the "unit tests of AI development": - Define expected behavior BEFORE implementation - Run evals continuously during development - Track regressions with each change -- Use pass@k metrics for reliability measurement ## Eval Types @@ -58,16 +57,16 @@ npm run build && echo "PASS" || echo "FAIL" ``` ### 2. Model-Based Grader -Use Claude to evaluate open-ended outputs: +Have a separate instance (subagent) review open-ended outputs against a checklist: ```markdown [MODEL GRADER PROMPT] -Evaluate the following code change: +Review the following code change and answer each question with YES/NO plus evidence: 1. Does it solve the stated problem? 2. Is it well-structured? 3. Are edge cases handled? 4. Is error handling appropriate? -Score: 1-5 (1=poor, 5=excellent) +Verdict: PASS/FAIL Reasoning: [explanation] ``` @@ -80,19 +79,7 @@ Reason: Why human review is needed Risk Level: LOW/MEDIUM/HIGH ``` -## Metrics - -### pass@k -"At least one success in k attempts" -- pass@1: First attempt success rate -- pass@3: Success within 3 attempts -- Typical target: pass@3 > 90% - -### pass^k -"All k trials succeed" -- Higher bar for reliability -- pass^3: 3 consecutive successes -- Use for critical paths +Note: statistical metrics like pass@k require running the same task k times independently. If you truly need them, implement an automated script (e.g., a headless `claude -p` loop), not manual bookkeeping in an interactive session. ## Eval Workflow @@ -109,10 +96,6 @@ Risk Level: LOW/MEDIUM/HIGH 1. Existing login still works 2. Session management unchanged 3. Logout flow intact - -### Success Metrics -- pass@3 > 90% for capability evals -- pass^3 = 100% for regression evals ``` ### 2. Implement @@ -135,9 +118,9 @@ EVAL REPORT: feature-xyz ======================== Capability Evals: - create-user: PASS (pass@1) - validate-email: PASS (pass@2) - hash-password: PASS (pass@1) + create-user: PASS (attempts: 1) + validate-email: PASS (attempts: 2) + hash-password: PASS (attempts: 1) Overall: 3/3 passed Regression Evals: @@ -146,10 +129,6 @@ Regression Evals: logout-flow: PASS Overall: 3/3 passed -Metrics: - pass@1: 67% (2/3) - pass@3: 100% (3/3) - Status: READY FOR REVIEW ``` @@ -188,11 +167,10 @@ Store evals in project: 1. **Define evals BEFORE coding** - Forces clear thinking about success criteria 2. **Run evals frequently** - Catch regressions early -3. **Track pass@k over time** - Monitor reliability trends -4. **Use code graders when possible** - Deterministic > probabilistic -5. **Human review for security** - Never fully automate security checks -6. **Keep evals fast** - Slow evals don't get run -7. **Version evals with code** - Evals are first-class artifacts +3. **Use code graders when possible** - Deterministic > probabilistic +4. **Human review for security** - Never fully automate security checks +5. **Keep evals fast** - Slow evals don't get run +6. **Version evals with code** - Evals are first-class artifacts ## Example: Adding Authentication @@ -221,7 +199,7 @@ Run: /eval check add-authentication ### Phase 4: Report EVAL REPORT: add-authentication ============================== -Capability: 5/5 passed (pass@3: 100%) -Regression: 3/3 passed (pass^3: 100%) +Capability: 5/5 passed +Regression: 3/3 passed Status: SHIP IT ``` diff --git a/skills/frontend-patterns/SKILL.md b/skills/frontend-patterns/SKILL.md index 7e7450c..bc4f263 100644 --- a/skills/frontend-patterns/SKILL.md +++ b/skills/frontend-patterns/SKILL.md @@ -15,7 +15,7 @@ Composition over inheritance, compound components, render props. See [references/component-patterns.md](references/component-patterns.md) for implementations. ### Custom Hooks -Reusable hooks: useToggle, useQuery, useDebounce. +Reusable hooks: useToggle, useDebounce; server state via TanStack Query / SWR. See [references/hooks-patterns.md](references/hooks-patterns.md) for implementations. ### State Management & Performance @@ -23,7 +23,7 @@ Context + Reducer pattern, memoization, code splitting, virtualization. See [references/state-performance.md](references/state-performance.md) for implementations. ### Forms, Error Boundaries, Animation & Accessibility -Controlled forms with validation, ErrorBoundary, Framer Motion, keyboard navigation, focus management. +Controlled forms with validation, ErrorBoundary, motion (formerly Framer Motion), keyboard navigation, focus management. See [references/forms-errors-a11y.md](references/forms-errors-a11y.md) for implementations. ## Quick Decision Guide @@ -31,13 +31,15 @@ See [references/forms-errors-a11y.md](references/forms-errors-a11y.md) for imple | Need | Pattern | |------|---------| | Shared state across siblings | Context + Reducer | -| Expensive computation | `useMemo` | -| Stable callback reference | `useCallback` | -| Prevent unnecessary re-renders | `React.memo` | +| Expensive computation (no React Compiler) | `useMemo` | +| Stable callback reference (no React Compiler) | `useCallback` | +| Prevent unnecessary re-renders (no React Compiler) | `React.memo` | | Large list rendering | Virtualization (`@tanstack/react-virtual`) | | Heavy component | `lazy()` + `Suspense` | | Form validation | Controlled form + error state | | Graceful error handling | ErrorBoundary class component | | Keyboard accessible UI | `role`, `aria-*`, `onKeyDown` handlers | +**Note**: On projects using React Compiler, manual memoization (`useMemo`/`useCallback`/`React.memo`) is generally unnecessary; add it only when profiling shows a real problem. + **Remember**: Choose patterns that fit your project complexity. Not every project needs every pattern. diff --git a/skills/frontend-patterns/references/forms-errors-a11y.md b/skills/frontend-patterns/references/forms-errors-a11y.md index 46d79ee..4edfd9d 100644 --- a/skills/frontend-patterns/references/forms-errors-a11y.md +++ b/skills/frontend-patterns/references/forms-errors-a11y.md @@ -88,10 +88,10 @@ export class ErrorBoundary extends React.Component< } ``` -## Framer Motion Animations +## motion Animations (formerly Framer Motion) ```typescript -import { motion, AnimatePresence } from 'framer-motion' +import { motion, AnimatePresence } from 'motion/react' export function AnimatedList({ items }: { items: Item[] }) { return ( diff --git a/skills/frontend-patterns/references/hooks-patterns.md b/skills/frontend-patterns/references/hooks-patterns.md index a729968..24155ba 100644 --- a/skills/frontend-patterns/references/hooks-patterns.md +++ b/skills/frontend-patterns/references/hooks-patterns.md @@ -10,46 +10,31 @@ export function useToggle(initialValue = false): [boolean, () => void] { } ``` -## Async Data Fetching Hook +## Server State (Data Fetching) + +Do not hand-roll data-fetching hooks. Use a dedicated server-state library — TanStack Query or SWR — which handles caching, deduplication, retries, and revalidation correctly. ```typescript -interface UseQueryOptions { - onSuccess?: (data: T) => void - onError?: (error: Error) => void - enabled?: boolean +// TanStack Query (v5: isPending; v4 used isLoading) +import { useQuery } from '@tanstack/react-query' + +export function UserProfile({ userId }: { userId: string }) { + const { data, error, isPending } = useQuery({ + queryKey: ['user', userId], + queryFn: () => fetchUser(userId), + }) + + if (isPending) return + if (error) return + return } +``` -export function useQuery( - key: string, - fetcher: () => Promise, - options?: UseQueryOptions -) { - const [data, setData] = useState(null) - const [error, setError] = useState(null) - const [loading, setLoading] = useState(false) - - const refetch = useCallback(async () => { - setLoading(true) - setError(null) - try { - const result = await fetcher() - setData(result) - options?.onSuccess?.(result) - } catch (err) { - const error = err as Error - setError(error) - options?.onError?.(error) - } finally { - setLoading(false) - } - }, [fetcher, options]) - - useEffect(() => { - if (options?.enabled !== false) refetch() - }, [key, refetch, options?.enabled]) +```typescript +// SWR (2.x: isLoading) +import useSWR from 'swr' - return { data, error, loading, refetch } -} +const { data, error, isLoading } = useSWR(`/api/users/${userId}`, fetcher) ``` ## Debounce Hook diff --git a/skills/frontend-patterns/references/state-performance.md b/skills/frontend-patterns/references/state-performance.md index 7ac26f4..9f0f398 100644 --- a/skills/frontend-patterns/references/state-performance.md +++ b/skills/frontend-patterns/references/state-performance.md @@ -49,6 +49,8 @@ export function MarketProvider({ children }: { children: React.ReactNode }) { ## Memoization +For projects **not** using React Compiler. With the Compiler enabled, manual memoization is generally unnecessary — add it only when profiling shows a real problem. + ```typescript // useMemo for expensive computations const sortedMarkets = useMemo(() => { diff --git a/skills/project-guidelines-example/references/code-patterns.md b/skills/project-guidelines-example/references/code-patterns.md index 4b7d67a..a697d69 100644 --- a/skills/project-guidelines-example/references/code-patterns.md +++ b/skills/project-guidelines-example/references/code-patterns.md @@ -67,7 +67,7 @@ class AnalysisResult(BaseModel): async def analyze_with_claude(content: str) -> AnalysisResult: client = Anthropic() response = client.messages.create( - model="claude-sonnet-4-5-20250514", + model="claude-opus-4-8", # Check the official docs (Models Overview) for the latest model IDs max_tokens=1024, messages=[{"role": "user", "content": content}], tools=[{ diff --git a/skills/prompt-patterns/SKILL.md b/skills/prompt-patterns/SKILL.md index 6ba5568..a633895 100644 --- a/skills/prompt-patterns/SKILL.md +++ b/skills/prompt-patterns/SKILL.md @@ -65,12 +65,14 @@ Process these 5 files in parallel using subagents. Each subagent handles one fil Collect all results and present a unified summary. ``` -### Ultrathink -For complex reasoning tasks, include the keyword: +### Deep Reasoning +For complex design and architecture decisions: +- Use Plan Mode for work that benefits from an explicit reviewed plan before changes. +- Extended thinking is controlled by the harness (thinking toggle / effort settings), not by magic keywords in the prompt. +- In the prompt itself, ask for depth in natural language, e.g.: ``` -ultrathink about the best architecture for this system +Before implementing, compare alternative designs and their trade-offs. ``` -Triggers extended thinking mode for higher effort reasoning. ## Debugging Patterns @@ -93,7 +95,7 @@ Look at this screenshot. The layout is broken. Fix it. Create a phase-wise plan for this feature. Each phase must have: 1. Clear deliverable 2. Test criteria (unit, integration, or manual verification) -3. Estimated context usage +3. Rollback plan if verification fails Do not proceed to the next phase until the current phase passes all tests. ``` diff --git a/skills/tdd-workflow/SKILL.md b/skills/tdd-workflow/SKILL.md index a0775e9..860240a 100644 --- a/skills/tdd-workflow/SKILL.md +++ b/skills/tdd-workflow/SKILL.md @@ -1,31 +1,31 @@ --- name: tdd-workflow -description: Use this skill when writing new features, fixing bugs, or refactoring code. Enforces test-driven development with 80%+ coverage including unit, integration, and E2E tests. -when_to_use: Use when implementing features, fixing bugs, or refactoring with tests-first discipline and explicit coverage expectations. +description: Use this skill when the user explicitly asks for TDD or a tests-first workflow, or when developing a new feature with a test coverage requirement. Provides a structured red-green-refactor workflow with unit, integration, and E2E test guidance. +when_to_use: Use when the user explicitly requests TDD/tests-first development, or when building a new feature where test coverage is a stated requirement. --- # Test-Driven Development Workflow -This skill ensures all code development follows TDD principles with comprehensive test coverage. +This skill provides a structured TDD workflow for projects that adopt tests-first development. ## When to Activate -- Writing new features or functionality -- Fixing bugs or issues -- Refactoring existing code -- Adding API endpoints -- Creating new components +- The user explicitly asks for TDD or tests-first development +- New feature work where test coverage is a stated requirement +- Adding API endpoints or components under a coverage policy + +For routine bug fixes or refactors, add focused tests for the changed behavior instead of invoking this full workflow (the /tdd command and tdd-guide agent remain available when you want it explicitly). ## Core Principles ### 1. Tests BEFORE Code -ALWAYS write tests first, then implement code to make tests pass. +Within this workflow, the default is to write tests first, then implement code to make tests pass. -### 2. Coverage Requirements -- Minimum 80% coverage (unit + integration + E2E) -- All edge cases covered -- Error scenarios tested -- Boundary conditions verified +### 2. Coverage Guidance +- Coverage target follows the project's own standard (e.g., 80% as a common baseline); scale effort with the size of the change +- Cover relevant edge cases +- Test error scenarios +- Verify boundary conditions ### 3. Test Types @@ -33,7 +33,7 @@ ALWAYS write tests first, then implement code to make tests pass. **Integration Tests** - API endpoints, database operations, service interactions, external API calls. -**E2E Tests (Playwright)** - Critical user flows, complete workflows, browser automation, UI interactions. +**E2E Tests (Playwright)** - Critical user flows, complete workflows, browser automation, UI interactions. Applies to web UI projects only. ## TDD Workflow Steps @@ -50,8 +50,11 @@ so that I can find relevant markets even without exact keywords. For each user journey, create comprehensive test cases covering happy paths, edge cases, fallback behavior, and sorting/filtering logic. ### Step 3: Run Tests (They Should Fail) +Use the project's test runner. Examples: ```bash -npm test +npm test # Node.js +pytest # Python +bash tests/run-unit-tests.sh # shell projects # Tests should fail - we haven't implemented yet ``` @@ -59,6 +62,7 @@ npm test Write minimal code to make tests pass. ### Step 5: Run Tests Again +Re-run the same test command (e.g., `npm test`, `pytest`): ```bash npm test # Tests should now pass @@ -72,9 +76,11 @@ Improve code quality while keeping tests green: - Enhance readability ### Step 7: Verify Coverage +Use the project's coverage tooling. Examples: ```bash -npm run test:coverage -# Verify 80%+ coverage achieved +npm run test:coverage # Node.js +pytest --cov # Python +# Verify coverage meets the project's target ``` ## Best Practices @@ -92,7 +98,7 @@ npm run test:coverage ## Success Metrics -- 80%+ code coverage achieved +- Coverage target met (per project standard, e.g., 80%) - All tests passing (green) - No skipped or disabled tests - Fast test execution (< 30s for unit tests) @@ -101,9 +107,11 @@ npm run test:coverage ## References -- `references/test-templates.md` - Unit, integration, E2E, and mocking code templates -- `references/testing-mistakes.md` - Common pitfalls, file organization, coverage config, CI setup +These templates target JavaScript/TypeScript stacks; adapt the structure for other languages. + +- `references/test-templates.md` - Unit, integration, E2E, and mocking code templates (JS/TS) +- `references/testing-mistakes.md` - Common pitfalls, file organization, coverage config, CI setup (JS/TS) --- -**Remember**: Tests are not optional. They are the safety net that enables confident refactoring, rapid development, and production reliability. +**Remember**: Give changed behavior a focused test. Tests are the safety net that enables confident refactoring, rapid development, and production reliability. diff --git a/skills/verification-loop/SKILL.md b/skills/verification-loop/SKILL.md index 1dfabf2..e5ea211 100644 --- a/skills/verification-loop/SKILL.md +++ b/skills/verification-loop/SKILL.md @@ -18,60 +18,37 @@ Invoke this skill: ## Verification Phases +Pick the right command for each phase from the project's own configuration (package.json scripts, Makefile, CI workflow, etc.). The commands below are examples, not prescriptions. Summarize long output instead of pasting it verbatim. + ### Phase 1: Build Verification -```bash -# Check if project builds -npm run build 2>&1 | tail -20 -# OR -pnpm build 2>&1 | tail -20 -``` +Check that the project builds. Examples: `npm run build`, `cargo check`, `go build ./...`, `shellcheck -S warning *.sh`. -If build fails, STOP and fix before continuing. +If the build fails, STOP and fix before continuing. ### Phase 2: Type Check -```bash -# TypeScript projects -npx tsc --noEmit 2>&1 | head -30 - -# Python projects -pyright . 2>&1 | head -30 -``` +Run the project's type checker if it has one. Examples: `npx tsc --noEmit` (TypeScript), `pyright .` or `mypy .` (Python). Report all type errors. Fix critical ones before continuing. ### Phase 3: Lint Check -```bash -# JavaScript/TypeScript -npm run lint 2>&1 | head -30 - -# Python -ruff check . 2>&1 | head -30 -``` +Run the project's linter. Examples: `npm run lint`, `ruff check .`, `shellcheck`. ### Phase 4: Test Suite -```bash -# Run tests with coverage -npm run test -- --coverage 2>&1 | tail -50 - -# Check coverage threshold -# Target: 80% minimum -``` +Run the project's test suite, with coverage if available. Examples: `npm test -- --coverage`, `pytest --cov`, `bash tests/run-unit-tests.sh`. Report: - Total tests: X - Passed: X - Failed: X -- Coverage: X% +- Coverage: X% (against the project's target) ### Phase 5: Security Scan -```bash -# Check for secrets -grep -rn "sk-" --include="*.ts" --include="*.js" . 2>/dev/null | head -10 -grep -rn "api_key" --include="*.ts" --include="*.js" . 2>/dev/null | head -10 +Check for leaked secrets in the changes: +- If a dedicated scanner is available (gitleaks, trufflehog), use it. +- Otherwise, follow the bundled security-review skill. +- At minimum, review the added lines of `git diff` for credentials, tokens, and keys. Do not report PASS based on a repository-wide grep. -# Check for console.log -grep -rn "console.log" --include="*.ts" --include="*.tsx" src/ 2>/dev/null | head -10 -``` +Also check for leftover debug output (e.g., `console.log`, `print`) in the changed files. ### Phase 6: Diff Review ```bash @@ -107,20 +84,8 @@ Issues to Fix: 2. ... ``` -## Continuous Mode - -For long sessions, run verification every 15 minutes or after major changes: - -```markdown -Set a mental checkpoint: -- After completing each function -- After finishing a component -- Before moving to next task - -Run: /verify -``` - ## Integration with Hooks This skill complements PostToolUse hooks but provides deeper verification. Hooks catch issues immediately; this skill provides comprehensive review. +If you need periodic automated re-verification, use a hook or the /loop command; instructions in this skill body cannot schedule themselves.