Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions .claude/commands/assess-complexity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
---
description: Create an EIP complexity assessment using the standard template. Analyzes the EIP specification and scores 24 anchors for testing complexity.
argument-hint: <EIP-number>
model: opus
allowed-tools: Read, Write, Edit, Grep, Glob, Bash(gh:*), WebFetch
---

# EIP Complexity Assessment

You are an expert Ethereum protocol analyst with deep knowledge of the EVM, consensus mechanisms, transaction types, system contracts, and the execution-specs testing framework. Your task is to create a comprehensive complexity assessment for EIP-$ARGUMENTS.

## Important: Think Deeply

This is a critical assessment that will guide testing efforts. Before scoring each anchor, think through:
- What specific mechanisms does this EIP introduce or modify?
- How does this interact with existing protocol components?
- What edge cases and boundary conditions exist?
- What is the testing burden for this change?

Take your time to analyze thoroughly. Accuracy is more important than speed.

## Step 1: Fetch the EIP Specification

Use WebFetch to retrieve the EIP markdown from:
`https://raw.githubusercontent.com/ethereum/EIPs/master/EIPS/eip-$ARGUMENTS.md`

Read the EIP carefully, noting:
- The abstract and motivation
- Specification details (new opcodes, precompiles, transaction types, etc.)
- Security considerations
- Backwards compatibility notes

## Step 2: Check for Open PRs to this EIP

Run this command to find open PRs that may modify this EIP:
```bash
gh api "search/issues?q=repo:ethereum/EIPs+is:pr+is:open+$ARGUMENTS+in:title" --jq '.items[] | "PR #\(.number): \(.title)"'
```

Note any open PRs for the final assessment.

## Step 3: Read the Template and Examples

Read the template:
- `Templates/EIP-Complexity-Assessment.md`

Read 2-3 existing assessments for reference on scoring patterns and rationale style:
- `complexity_assessments/EIPs/EIP-7805.md` (medium complexity example)
- `complexity_assessments/EIPs/EIP-7708.md` (low complexity example)

## Step 4: Create the Assessment

For each of the 24 anchors in the checklist, analyze the EIP and determine:
1. **Score (0-3)**: Based on the anchor definitions in the template
2. **Rationale**: Brief justification referencing specific EIP mechanisms

### Anchors to Score

Score each anchor systematically:

1. **EVM Gas rule changes** - New gas accounting rules?
2. **Blob gas accounting changes** - Blob-related gas changes?
3. **New EVM gas refund** - New refund mechanisms?
4. **Patterns affecting pre-existing tests** - Will existing tests need updates?
5. **Transition-tool interface changes** - New t8n fields/mechanisms?
6. **Cryptography** - New crypto primitives or algorithms?
7. **Edge/boundary conditions** - Boundary-prone mechanisms?
8. **Block syncing changes** - RLP validation changes?
9. **Engine API changes** - New fields or endpoints?
10. **Added system contracts** - New system contracts?
11. **Modified system contracts** - Changes to existing contracts?
12. **Added opcodes** - New EVM opcodes?
13. **Modified opcodes** - Changes to existing opcodes?
14. **Added precompiles** - New precompiled contracts?
15. **Modified precompiles** - Changes to existing precompiles?
16. **Encoding changes (RLP/SSZ)** - Encoding format changes?
17. **New transaction types** - New tx type introduced?
18. **New or modified transaction validity** - Validity rule changes?
19. **New block/header fields** - New header fields?
20. **New fork activation mechanism** - State changes at fork activation?
21. **Performance risks** - Performance validation needed?
22. **Security risks** - Security-sensitive mechanisms?
23. **Cross-EIP interactions** - Dependencies on other EIPs?

### Scoring Guidelines

- **0**: No impact in this category
- **1**: Minor/simple change, limited testing scope
- **2**: Moderate change, affects multiple components
- **3**: Major/complex change, extensive testing required
- **4**: Exceptional circumstances only (document why)

Some anchors only allow 0 or 3 (binary impact):
- Modified opcodes
- Encoding changes
- New transaction types
- New block/header fields
- New fork activation mechanism

## Step 5: Fill in the Specs Section

Under "Execution Specs > Specs", write a brief summary of the key specification changes this EIP introduces. Focus on what changes in the protocol, not the full EIP text.

## Step 6: Calculate Total and Determine Tier

Sum all anchor scores for the total (max 72).

Determine complexity tier:
- **< 10**: 🟢 Low Complexity
- **10-19**: 🟡 Medium Complexity
- **>= 20**: 🔴 High Complexity

## Step 7: Write Special Considerations and Notes

- **Special Considerations**: Factors that make this EIP particularly complex beyond the checklist
- **Notes**: Points for test authors that don't affect scoring

## Step 8: Handle Open PRs

If there are open PRs to this EIP, append a section at the end:

```markdown
---

## Open PRs Notice

At the time of writing this complexity assessment, EIP-$ARGUMENTS had the following open PRs to its specification:

- [List the PRs found in Step 2]

**Impact Assessment**: [Brief assessment of whether these PRs would materially change the complexity scores if merged]
```

## Step 9: Write the Assessment File

Write the complete assessment to:
`complexity_assessments/EIPs/EIP-$ARGUMENTS.md`

The file should follow the exact structure of the template, with all sections filled in.

## Output

After writing the file, provide a brief summary:
- Total score and complexity tier
- Key drivers of complexity (highest-scoring anchors)
- Any special considerations worth highlighting
170 changes: 170 additions & 0 deletions .claude/commands/review-complexity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
---
description: Review a complexity assessment PR or local file. Validates scores against the EIP specification and provides structured feedback.
argument-hint: <PR-number or file-path>
model: opus
allowed-tools: Read, Grep, Glob, Bash(gh:*), WebFetch
---

# Complexity Assessment Review

You are an expert Ethereum protocol analyst reviewing a complexity assessment. Your role is to validate the assessment's accuracy, identify any scoring issues, and provide constructive feedback.

## Important: Use Extended Thinking

Complexity assessments guide testing prioritization. An incorrect score could lead to:
- Under-testing of critical changes (if scored too low)
- Wasted resources on simple changes (if scored too high)

For each scored anchor, think through:
- Does the EIP actually introduce/modify what this anchor measures?
- Is the score consistent with the rubric definition (0-3 scale)?
- Does the rationale accurately reflect the EIP specification?
- Are there mechanisms the assessor may have missed?

Take your time to analyze thoroughly. Accuracy is more important than speed.

## Step 1: Determine Input Type

The argument is: `$ARGUMENTS`

Determine if this is:
- **PR number**: A plain number (e.g., `27`, `123`) → fetch from GitHub PR
- **File path**: Contains `/` or `.md` (e.g., `complexity_assessments/EIPs/EIP-7971.md`) → read local file

## Step 2: Fetch the Assessment

### If PR number:

Get the PR information:
```bash
gh pr view $ARGUMENTS --json title,body,author,files,additions,deletions
```

Get the PR diff to see the assessment content:
```bash
gh pr diff $ARGUMENTS
```

### If local file:

Read the file directly using the Read tool with the provided path.

If the path is relative, it's relative to the repository root.

---

From the assessment, identify:
- Which EIP is being assessed (from the file name, e.g., `EIP-7971.md` → EIP-7971)
- What scores and rationales were provided in the checklist table

## Step 3: Fetch the EIP Specification

Extract the EIP number from the assessment file name (e.g., `EIP-1234.md` → 1234).

Fetch the EIP using WebFetch:
```
https://raw.githubusercontent.com/ethereum/EIPs/master/EIPS/eip-{number}.md
```

Read the EIP thoroughly to understand the actual specification.

## Step 4: Read the Assessment Template

Read `Templates/EIP-Complexity-Assessment.md` to understand:
- The scoring rubric for each anchor (0-3 scale)
- Which anchors are binary (0 or 3 only)
- What constitutes proper rationale

## Step 5: Review Each Scored Anchor

For each anchor in the assessment's checklist table:

1. **Verify the score is appropriate** based on the EIP specification and anchor rubric
2. **Check the rationale** is sufficient and accurate
3. **Note any discrepancies** between the score and EIP content

### Common Issues to Look For

**Under-scoring (score too low):**
- EIP introduces mechanism but anchor scored 0
- Complex mechanism scored as simple (1 instead of 2-3)
- Missing consideration of indirect effects

**Over-scoring (score too high):**
- Score of 2-3 without clear justification
- Assuming complexity that isn't in the EIP
- Double-counting (same mechanism counted in multiple anchors)

**Rationale issues:**
- Empty rationale for non-zero scores
- Rationale doesn't match the score given
- Vague rationale that doesn't reference specific EIP mechanisms

**Binary anchor violations:**
- These anchors should only be 0 or 3:
- Modified opcodes
- Encoding changes (RLP/SSZ)
- New transaction types
- New block/header fields
- New fork activation mechanism

## Step 6: Verify Totals and Tier

1. Sum the individual scores - does it match the stated total?
2. Verify the complexity tier matches the total:
- < 10 → 🟢 Low Complexity
- 10-19 → 🟡 Medium Complexity
- >= 20 → 🔴 High Complexity

## Step 7: Review Special Considerations and Notes

- Are there complexity factors not captured in the anchors?
- Are the notes helpful for future test authors?
- Is anything missing that should be called out?

## Step 8: Check Specs Section

- Is the "Specs" section filled in with a meaningful summary?
- Does it accurately reflect the EIP's key changes?

## Step 9: Provide Structured Feedback

Present your review in this format:

---

## Review: EIP-{number} Complexity Assessment

**Source**: [PR #X | Local file: path/to/file.md]

### Summary
- **Overall Assessment**: [Approve / Request Changes / Comment]
- **Total Score Accuracy**: [Correct / Incorrect (explain)]
- **Complexity Tier**: [Correct / Incorrect]

### Scoring Review

| Anchor | Submitted | Suggested | Issue |
|--------|-----------|-----------|-------|
| [Only list anchors with issues] | | | |

### Detailed Feedback

#### Scores to Adjust
[List any scores that should be changed, with reasoning]

#### Missing Considerations
[Any factors not accounted for in the assessment]

#### Rationale Improvements
[Suggestions for better justifying scores]

### Recommendations

[Actionable next steps for the author]

---

## Output

Provide your complete review following the format above. Be constructive and specific - the goal is to improve the assessment's accuracy, not to criticize.