diff --git a/.claude/commands/assess-complexity.md b/.claude/commands/assess-complexity.md new file mode 100644 index 0000000..c3e652f --- /dev/null +++ b/.claude/commands/assess-complexity.md @@ -0,0 +1,146 @@ +--- +description: Create an EIP complexity assessment using the standard template. Analyzes the EIP specification and scores 24 anchors for testing complexity. +argument-hint: +model: opus +allowed-tools: Read, Write, Edit, Grep, Glob, Bash(gh:*), WebFetch +--- + +# EIP Complexity Assessment + +You are an expert Ethereum protocol analyst with deep knowledge of the EVM, consensus mechanisms, transaction types, system contracts, and the execution-specs testing framework. Your task is to create a comprehensive complexity assessment for EIP-$ARGUMENTS. + +## Important: Think Deeply + +This is a critical assessment that will guide testing efforts. Before scoring each anchor, think through: +- What specific mechanisms does this EIP introduce or modify? +- How does this interact with existing protocol components? +- What edge cases and boundary conditions exist? +- What is the testing burden for this change? + +Take your time to analyze thoroughly. Accuracy is more important than speed. + +## Step 1: Fetch the EIP Specification + +Use WebFetch to retrieve the EIP markdown from: +`https://raw.githubusercontent.com/ethereum/EIPs/master/EIPS/eip-$ARGUMENTS.md` + +Read the EIP carefully, noting: +- The abstract and motivation +- Specification details (new opcodes, precompiles, transaction types, etc.) +- Security considerations +- Backwards compatibility notes + +## Step 2: Check for Open PRs to this EIP + +Run this command to find open PRs that may modify this EIP: +```bash +gh api "search/issues?q=repo:ethereum/EIPs+is:pr+is:open+$ARGUMENTS+in:title" --jq '.items[] | "PR #\(.number): \(.title)"' +``` + +Note any open PRs for the final assessment. + +## Step 3: Read the Template and Examples + +Read the template: +- `Templates/EIP-Complexity-Assessment.md` + +Read 2-3 existing assessments for reference on scoring patterns and rationale style: +- `complexity_assessments/EIPs/EIP-7805.md` (medium complexity example) +- `complexity_assessments/EIPs/EIP-7708.md` (low complexity example) + +## Step 4: Create the Assessment + +For each of the 24 anchors in the checklist, analyze the EIP and determine: +1. **Score (0-3)**: Based on the anchor definitions in the template +2. **Rationale**: Brief justification referencing specific EIP mechanisms + +### Anchors to Score + +Score each anchor systematically: + +1. **EVM Gas rule changes** - New gas accounting rules? +2. **Blob gas accounting changes** - Blob-related gas changes? +3. **New EVM gas refund** - New refund mechanisms? +4. **Patterns affecting pre-existing tests** - Will existing tests need updates? +5. **Transition-tool interface changes** - New t8n fields/mechanisms? +6. **Cryptography** - New crypto primitives or algorithms? +7. **Edge/boundary conditions** - Boundary-prone mechanisms? +8. **Block syncing changes** - RLP validation changes? +9. **Engine API changes** - New fields or endpoints? +10. **Added system contracts** - New system contracts? +11. **Modified system contracts** - Changes to existing contracts? +12. **Added opcodes** - New EVM opcodes? +13. **Modified opcodes** - Changes to existing opcodes? +14. **Added precompiles** - New precompiled contracts? +15. **Modified precompiles** - Changes to existing precompiles? +16. **Encoding changes (RLP/SSZ)** - Encoding format changes? +17. **New transaction types** - New tx type introduced? +18. **New or modified transaction validity** - Validity rule changes? +19. **New block/header fields** - New header fields? +20. **New fork activation mechanism** - State changes at fork activation? +21. **Performance risks** - Performance validation needed? +22. **Security risks** - Security-sensitive mechanisms? +23. **Cross-EIP interactions** - Dependencies on other EIPs? + +### Scoring Guidelines + +- **0**: No impact in this category +- **1**: Minor/simple change, limited testing scope +- **2**: Moderate change, affects multiple components +- **3**: Major/complex change, extensive testing required +- **4**: Exceptional circumstances only (document why) + +Some anchors only allow 0 or 3 (binary impact): +- Modified opcodes +- Encoding changes +- New transaction types +- New block/header fields +- New fork activation mechanism + +## Step 5: Fill in the Specs Section + +Under "Execution Specs > Specs", write a brief summary of the key specification changes this EIP introduces. Focus on what changes in the protocol, not the full EIP text. + +## Step 6: Calculate Total and Determine Tier + +Sum all anchor scores for the total (max 72). + +Determine complexity tier: +- **< 10**: 🟢 Low Complexity +- **10-19**: 🟡 Medium Complexity +- **>= 20**: 🔴 High Complexity + +## Step 7: Write Special Considerations and Notes + +- **Special Considerations**: Factors that make this EIP particularly complex beyond the checklist +- **Notes**: Points for test authors that don't affect scoring + +## Step 8: Handle Open PRs + +If there are open PRs to this EIP, append a section at the end: + +```markdown +--- + +## Open PRs Notice + +At the time of writing this complexity assessment, EIP-$ARGUMENTS had the following open PRs to its specification: + +- [List the PRs found in Step 2] + +**Impact Assessment**: [Brief assessment of whether these PRs would materially change the complexity scores if merged] +``` + +## Step 9: Write the Assessment File + +Write the complete assessment to: +`complexity_assessments/EIPs/EIP-$ARGUMENTS.md` + +The file should follow the exact structure of the template, with all sections filled in. + +## Output + +After writing the file, provide a brief summary: +- Total score and complexity tier +- Key drivers of complexity (highest-scoring anchors) +- Any special considerations worth highlighting diff --git a/.claude/commands/review-complexity.md b/.claude/commands/review-complexity.md new file mode 100644 index 0000000..b0da56e --- /dev/null +++ b/.claude/commands/review-complexity.md @@ -0,0 +1,170 @@ +--- +description: Review a complexity assessment PR or local file. Validates scores against the EIP specification and provides structured feedback. +argument-hint: +model: opus +allowed-tools: Read, Grep, Glob, Bash(gh:*), WebFetch +--- + +# Complexity Assessment Review + +You are an expert Ethereum protocol analyst reviewing a complexity assessment. Your role is to validate the assessment's accuracy, identify any scoring issues, and provide constructive feedback. + +## Important: Use Extended Thinking + +Complexity assessments guide testing prioritization. An incorrect score could lead to: +- Under-testing of critical changes (if scored too low) +- Wasted resources on simple changes (if scored too high) + +For each scored anchor, think through: +- Does the EIP actually introduce/modify what this anchor measures? +- Is the score consistent with the rubric definition (0-3 scale)? +- Does the rationale accurately reflect the EIP specification? +- Are there mechanisms the assessor may have missed? + +Take your time to analyze thoroughly. Accuracy is more important than speed. + +## Step 1: Determine Input Type + +The argument is: `$ARGUMENTS` + +Determine if this is: +- **PR number**: A plain number (e.g., `27`, `123`) → fetch from GitHub PR +- **File path**: Contains `/` or `.md` (e.g., `complexity_assessments/EIPs/EIP-7971.md`) → read local file + +## Step 2: Fetch the Assessment + +### If PR number: + +Get the PR information: +```bash +gh pr view $ARGUMENTS --json title,body,author,files,additions,deletions +``` + +Get the PR diff to see the assessment content: +```bash +gh pr diff $ARGUMENTS +``` + +### If local file: + +Read the file directly using the Read tool with the provided path. + +If the path is relative, it's relative to the repository root. + +--- + +From the assessment, identify: +- Which EIP is being assessed (from the file name, e.g., `EIP-7971.md` → EIP-7971) +- What scores and rationales were provided in the checklist table + +## Step 3: Fetch the EIP Specification + +Extract the EIP number from the assessment file name (e.g., `EIP-1234.md` → 1234). + +Fetch the EIP using WebFetch: +``` +https://raw.githubusercontent.com/ethereum/EIPs/master/EIPS/eip-{number}.md +``` + +Read the EIP thoroughly to understand the actual specification. + +## Step 4: Read the Assessment Template + +Read `Templates/EIP-Complexity-Assessment.md` to understand: +- The scoring rubric for each anchor (0-3 scale) +- Which anchors are binary (0 or 3 only) +- What constitutes proper rationale + +## Step 5: Review Each Scored Anchor + +For each anchor in the assessment's checklist table: + +1. **Verify the score is appropriate** based on the EIP specification and anchor rubric +2. **Check the rationale** is sufficient and accurate +3. **Note any discrepancies** between the score and EIP content + +### Common Issues to Look For + +**Under-scoring (score too low):** +- EIP introduces mechanism but anchor scored 0 +- Complex mechanism scored as simple (1 instead of 2-3) +- Missing consideration of indirect effects + +**Over-scoring (score too high):** +- Score of 2-3 without clear justification +- Assuming complexity that isn't in the EIP +- Double-counting (same mechanism counted in multiple anchors) + +**Rationale issues:** +- Empty rationale for non-zero scores +- Rationale doesn't match the score given +- Vague rationale that doesn't reference specific EIP mechanisms + +**Binary anchor violations:** +- These anchors should only be 0 or 3: + - Modified opcodes + - Encoding changes (RLP/SSZ) + - New transaction types + - New block/header fields + - New fork activation mechanism + +## Step 6: Verify Totals and Tier + +1. Sum the individual scores - does it match the stated total? +2. Verify the complexity tier matches the total: + - < 10 → 🟢 Low Complexity + - 10-19 → 🟡 Medium Complexity + - >= 20 → 🔴 High Complexity + +## Step 7: Review Special Considerations and Notes + +- Are there complexity factors not captured in the anchors? +- Are the notes helpful for future test authors? +- Is anything missing that should be called out? + +## Step 8: Check Specs Section + +- Is the "Specs" section filled in with a meaningful summary? +- Does it accurately reflect the EIP's key changes? + +## Step 9: Provide Structured Feedback + +Present your review in this format: + +--- + +## Review: EIP-{number} Complexity Assessment + +**Source**: [PR #X | Local file: path/to/file.md] + +### Summary +- **Overall Assessment**: [Approve / Request Changes / Comment] +- **Total Score Accuracy**: [Correct / Incorrect (explain)] +- **Complexity Tier**: [Correct / Incorrect] + +### Scoring Review + +| Anchor | Submitted | Suggested | Issue | +|--------|-----------|-----------|-------| +| [Only list anchors with issues] | | | | + +### Detailed Feedback + +#### Scores to Adjust +[List any scores that should be changed, with reasoning] + +#### Missing Considerations +[Any factors not accounted for in the assessment] + +#### Rationale Improvements +[Suggestions for better justifying scores] + +### Recommendations + +[Actionable next steps for the author] + +--- + +## Output + +Provide your complete review following the format above. Be constructive and specific - the goal is to improve the assessment's accuracy, not to criticize.