feat: add daily CLAUDE.md audit command#185
Open
edspencer wants to merge 1 commit into
Open
Conversation
Add `/claude-md-audit-daily` meta-orchestrator that: - Runs `/audit-claude-md` via sub-agent on a daily schedule - Compares report against previous baseline for regressions - Creates a PR only when issue count increases - Follows established patterns from security-audit-daily and docs-audit-daily Update canon/autoclaude.md to mark all 4 steps as complete. Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughIntroduces a new meta-orchestrator for automated daily CLAUDE.md audits with a five-phase workflow including branch management, unattended execution, baseline comparison, and conditional PR creation. Updates project documentation to reference the new daily audit capability. Changes
Sequence DiagramsequenceDiagram
participant Scheduler as Scheduler/Cron
participant Orchestrator as Daily Audit<br/>Orchestrator
participant Git as Git/Branch<br/>Manager
participant SubAgent as Sub-Agent<br/>(Audit Runner)
participant Reports as Reports<br/>Directory
participant BaselineComp as Baseline<br/>Comparator
participant GitOps as Git Push/<br/>PR Handler
Scheduler->>Orchestrator: Trigger daily audit
Orchestrator->>Orchestrator: Phase 0: Preflight validation
Orchestrator->>Git: Phase 1: Create daily branch<br/>(claude-md/audit-YYYY-MM-DD)
Git-->>Orchestrator: Branch created/verified
Orchestrator->>SubAgent: Phase 2: Delegate audit task
SubAgent->>SubAgent: Run full CLAUDE.md audit
SubAgent-->>Orchestrator: Audit complete, return report
Orchestrator->>Reports: Write audit report<br/>(YYYY-MM-DD-audit.md)
Orchestrator->>Reports: Phase 3: Read previous baseline
Reports-->>Orchestrator: Previous audit data
Orchestrator->>BaselineComp: Compare NEW vs PREVIOUS totals
BaselineComp-->>Orchestrator: STATUS & ACTION<br/>(commit-only/create-pr/error)
Orchestrator->>GitOps: Phase 4: Commit changes
alt Regression Detected
GitOps->>GitOps: Create PR with delta details
end
GitOps->>Git: Push branch (or delete if clean)
Git-->>Orchestrator: Push complete
Orchestrator->>Orchestrator: Phase 5: Cleanup & report status
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly Related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Deploying herdctl with
|
| Latest commit: |
1874a8b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://78febcec.herdctl.pages.dev |
| Branch Preview URL: | https://feature-claude-md-daily-audi.herdctl.pages.dev |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.claude/commands/claude-md-audit-daily.md:
- Around line 142-161: The fenced code block that begins with "Use the Task tool
with:" (containing keys like subagent_type: "general-purpose",
run_in_background: false, prompt: "...") is untyped and triggers markdownlint
MD040; update the opening fence from ``` to include a language identifier (e.g.,
```text) so the block becomes a typed fenced code block and linting passes.
- Around line 199-223: The decision order makes CLEAN unreachable because the
NEW_TOTAL == 0 check comes after the numeric comparisons; update the branching
logic that sets STATUS and ACTION (using variables NEW_TOTAL, PREVIOUS_TOTAL,
STATUS, ACTION and the Phase 4/5 flow) so the zero-check runs before comparing
NEW_TOTAL to PREVIOUS_TOTAL (or implement a post-condition override that sets
STATUS="CLEAN" and ACTION="commit-only" when NEW_TOTAL==0). Ensure the
special-case for NEW_TOTAL=="ERROR" still short-circuits to STATUS="ERROR" and
skips Phase 4 as before.
- Around line 264-289: The heredoc is using a quoted delimiter (<<'PREOF') which
disables shell variable expansion so ${PREVIOUS_TOTAL}, ${NEW_TOTAL},
${ISSUE_DELTA}, and ${TODAY} remain literal; change the delimiter to an unquoted
form (<<PREOF) in the --body "$(cat <<'PREOF' ... PREOF )" block so those
variables are interpolated into the PR body.
Comment on lines
+142
to
+161
| ``` | ||
| Use the Task tool with: | ||
| - subagent_type: "general-purpose" | ||
| - run_in_background: false | ||
| - prompt: "IMPORTANT RULES: | ||
| (1) Do NOT create branches or run git checkout — stay on the current branch. | ||
| (2) Do NOT commit any changes — the orchestrator handles commits. | ||
|
|
||
| Run the /audit-claude-md command. Execute a full CLAUDE.md audit: | ||
| 1. Read the gold standard from canon/claude-md-gold-standard.md | ||
| 2. Discover all CLAUDE.md files and .claude/rules/*.md files | ||
| 3. Launch the 4 audit sub-agents (inventory+size, style+content, staleness, coverage) | ||
| 4. Assemble the full report | ||
| 5. Write the report to .reports/claude/{TODAY}-audit.md | ||
|
|
||
| When complete, report back with: | ||
| - Total issues found (number) | ||
| - Breakdown by dimension (inventory, size, style, content, staleness, coverage) | ||
| - The full remediation checklist" | ||
| ``` |
Contributor
There was a problem hiding this comment.
Add a language identifier to this fenced block.
This block is currently untyped; markdownlint MD040 is triggered. Use a language tag (likely text here since this is Task-tool instruction format).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/commands/claude-md-audit-daily.md around lines 142 - 161, The fenced
code block that begins with "Use the Task tool with:" (containing keys like
subagent_type: "general-purpose", run_in_background: false, prompt: "...") is
untyped and triggers markdownlint MD040; update the opening fence from ``` to
include a language identifier (e.g., ```text) so the block becomes a typed
fenced code block and linting passes.
Comment on lines
+199
to
+223
| If NEW_TOTAL is "ERROR": | ||
| STATUS="ERROR" | ||
| ACTION="error" | ||
| → Skip Phase 4, proceed to Phase 5 cleanup | ||
|
|
||
| If NEW_TOTAL > PREVIOUS_TOTAL: | ||
| STATUS="REGRESSION" | ||
| ACTION="create-pr" | ||
| → New issues found, create a PR for visibility | ||
|
|
||
| If NEW_TOTAL == PREVIOUS_TOTAL: | ||
| STATUS="STABLE" | ||
| ACTION="commit-only" | ||
| → No change, commit report to branch but no PR needed | ||
|
|
||
| If NEW_TOTAL < PREVIOUS_TOTAL: | ||
| STATUS="IMPROVED" | ||
| ACTION="commit-only" | ||
| → Issues decreased (someone fixed things), commit report as new baseline | ||
|
|
||
| If NEW_TOTAL == 0: | ||
| STATUS="CLEAN" | ||
| ACTION="commit-only" | ||
| → Perfect score, commit report as new baseline | ||
| ``` |
Contributor
There was a problem hiding this comment.
CLEAN status is unreachable in a common zero-issue case.
When NEW_TOTAL=0 and PREVIOUS_TOTAL>0, the < branch is matched first, so status becomes IMPROVED instead of CLEAN. Move the zero check before numeric comparisons (or derive CLEAN as a post-condition override).
Suggested decision-order fix
-If NEW_TOTAL > PREVIOUS_TOTAL:
+If NEW_TOTAL == 0:
+ STATUS="CLEAN"
+ ACTION="commit-only"
+ → Perfect score, commit report as new baseline
+
+If NEW_TOTAL > PREVIOUS_TOTAL:
STATUS="REGRESSION"
ACTION="create-pr"
→ New issues found, create a PR for visibility
@@
-If NEW_TOTAL == 0:
- STATUS="CLEAN"
- ACTION="commit-only"
- → Perfect score, commit report as new baseline🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/commands/claude-md-audit-daily.md around lines 199 - 223, The
decision order makes CLEAN unreachable because the NEW_TOTAL == 0 check comes
after the numeric comparisons; update the branching logic that sets STATUS and
ACTION (using variables NEW_TOTAL, PREVIOUS_TOTAL, STATUS, ACTION and the Phase
4/5 flow) so the zero-check runs before comparing NEW_TOTAL to PREVIOUS_TOTAL
(or implement a post-condition override that sets STATUS="CLEAN" and
ACTION="commit-only" when NEW_TOTAL==0). Ensure the special-case for
NEW_TOTAL=="ERROR" still short-circuits to STATUS="ERROR" and skips Phase 4 as
before.
Comment on lines
+264
to
+289
| --body "$(cat <<'PREOF' | ||
| ## CLAUDE.md Audit Regression | ||
|
|
||
| The daily CLAUDE.md audit found **more issues than the previous baseline**. | ||
|
|
||
| | Metric | Previous | Current | Delta | | ||
| |--------|----------|---------|-------| | ||
| | Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} | | ||
|
|
||
| ### What This Means | ||
|
|
||
| New CLAUDE.md quality issues were introduced since the last audit. Review the | ||
| audit report at `.reports/claude/${TODAY}-audit.md` for the full remediation | ||
| checklist. | ||
|
|
||
| ### Next Steps | ||
|
|
||
| 1. Review the remediation checklist in the audit report | ||
| 2. Fix the new issues (or confirm they are acceptable) | ||
| 3. Merge this PR to update the baseline | ||
|
|
||
| --- | ||
|
|
||
| *Generated by `/claude-md-audit-daily` on ${TODAY}* | ||
| PREOF | ||
| )" \ |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
fd -t f "claude-md-audit-daily.md"Repository: edspencer/herdctl
Length of output: 43
🏁 Script executed:
find . -type f -name "*claude*" | head -20Repository: edspencer/herdctl
Length of output: 237
🏁 Script executed:
wc -l ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 108
🏁 Script executed:
sed -n '264,289p' ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 769
🏁 Script executed:
# Let me check the context around this code to understand the intent
sed -n '250,290p' ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 1120
🏁 Script executed:
# Let me verify the heredoc behavior with variables in the context
cat > /tmp/test_heredoc.sh << 'EOF'
#!/bin/bash
PREVIOUS_TOTAL=3
NEW_TOTAL=7
ISSUE_DELTA=4
TODAY="2024-01-15"
echo "Test 1: Quoted heredoc (no expansion) - what the code currently does:"
result=$(cat <<'PREOF'
| Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} |
Generated on ${TODAY}
PREOF
)
echo "$result"
echo ""
echo "Test 2: Unquoted heredoc (with expansion) - what reviewer suggests:"
result=$(cat <<PREOF
| Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} |
Generated on ${TODAY}
PREOF
)
echo "$result"
EOF
bash /tmp/test_heredoc.shRepository: edspencer/herdctl
Length of output: 345
🏁 Script executed:
# Check if there are other instances of similar patterns in the file
grep -n "<<'" ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 90
PR body variables won't expand due to quoted heredoc delimiter.
Line 264 uses <<'PREOF', which prevents shell variable expansion. Variables like ${PREVIOUS_TOTAL}, ${NEW_TOTAL}, ${ISSUE_DELTA}, and ${TODAY} will appear literally in the PR body instead of being interpolated. Change to <<PREOF (unquoted delimiter) to enable expansion.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/commands/claude-md-audit-daily.md around lines 264 - 289, The
heredoc is using a quoted delimiter (<<'PREOF') which disables shell variable
expansion so ${PREVIOUS_TOTAL}, ${NEW_TOTAL}, ${ISSUE_DELTA}, and ${TODAY}
remain literal; change the delimiter to an unquoted form (<<PREOF) in the --body
"$(cat <<'PREOF' ... PREOF )" block so those variables are interpolated into the
PR body.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/claude-md-audit-dailymeta-orchestrator for automated daily CLAUDE.md quality auditingcanon/autoclaude.mdto mark all 4 steps of the AutoCLAUDE initiative as completeWhat it does
The daily audit command:
/audit-claude-mdauditFollows the established patterns from
security-audit-dailyanddocs-audit-daily.Test plan
claude "/claude-md-audit-daily"produces a report.reports/claude/*-audit.md🤖 Generated with Claude Code
Summary by CodeRabbit