Refactor: full-repository code unification & deduplication (with Docker redeploy verification)

[rafeekpro]

Context

ClaudeAutoPM v3.25.5 is a Node.js framework (~655 .js, ~727 .md, ~134 .sh) distributed via npm; autopm install copies .claude/ and supporting dirs into target projects. The codebase has a triplication problem: the same logic exists in /.claude/ (primary source), /autopm/.claude/ (install template), and /packages/plugin-*/ (plugin variants). Additionally there are parallel .sh/.js implementations of the same commands and three competing epic-sync versions.

---

Concrete duplication findings

1. Frontmatter parsing — 3-4 implementations

• lib/filter-engine.js::parseFrontmatter() (lines 111-145)
• lib/services/ContextService.js::_parseFrontmatter() (lines 190+)
• .claude/lib/frontmatter.js
• autopm/.claude/lib/frontmatter.js (copy)
  Each with slightly different regex and error handling.

2. Epic-sync command triplication → 9 files

• .claude/commands/pm:epic-sync.md (485 lines), pm:epic-sync-original.md (473), pm:epic-sync-modular.md (337)
• × 3 locations (main, autopm/.claude/, packages/plugin-pm-github/)
  Three competing implementations; "original"/"modular" suggest incomplete migration.

3. Parallel .sh + .js implementations — 14 command pairs

In .claude/scripts/pm/: blocked, epic-list, epic-show, epic-status, in-progress, init, next, prd-list, prd-status, search, standup, status, validate, help — each exists as both .js and .sh with observed divergence (e.g. blocked.js vs blocked.sh).

4. Shell utility triplication in scripts/lib/

datetime-utils.sh (253 lines), frontmatter-utils.sh (308), github-utils.sh (236), logging-utils.sh (198), validation-utils.sh (338) — each ×3 copies (main, autopm, plugin-core). A bug fix requires patching 3 locations; function signatures already inconsistent.

5. Provider architecture duplication

• lib/providers/GitHubProvider.js (475 lines, Octokit)
• lib/providers/AzureDevOpsProvider.js (575) + AzureDevOpsCliWrapper.js (370) + AzureDevOpsRestClient.js (424) + AzureDevOpsResourcesProvider.js (322)
  No shared BaseProvider; Azure has 4 variants with unclear responsibilities; auth patterns repeated.

6. Install logic hardcoded

install/install.js lines 37-53: hardcoded installItems array (.claude/agents, .claude/commands, …, lib). No manifest; adding a directory means editing install.js. Plus install/install.sh wrapper and install/merge-claude.js.

7. Sync command duplication

.claude/scripts/pm/sync.js, sync-batch.js, epic-sync.sh (orchestrator), and epic-sync/{create-epic-issue,create-task-issues,update-epic-file,update-references}.sh — bash is modularized, JS isn't; pattern unclear.

8. Rule files in both .md and .xml

tdd.enforcement, command-pipelines, agent-mandatory, context7, naming-conventions, github-operations — each has a .md AND .xml version with overlapping content (.xml referenced via @include in CLAUDE.md). Canonical unclear.

9. Core rules copied into 10+ plugin directories

/.claude/rules/ (24 files) ≈ /packages/plugin-core/rules/ (18 near-identical files) + plugin-ai, plugin-cloud, etc.

10. Six service classes without shared base

lib/services/{PRD,Epic,Task,Issue,Context,Agent}Service.js — each independently implements read/write/filter/search on markdown files with repeated error handling.

11. Three different CLI/SDK access styles

GitHubProvider uses Octokit SDK; AzureDevOpsCliWrapper shells out to az; pm scripts shell out to gh directly. No unified CLI abstraction.

12. Two batch processors

lib/batch-processor.js + lib/batch-processor-integration.js + separate utils/RateLimiter.js + utils/CircuitBreaker.js — unclear which to use when.

13. Mixed test frameworks

Jest tests (test/jest-tests/epic-sync-scripts-jest.test.js, azure-epic-sync-jest.test.js) coexist with Node native --test files.

---

Proposed target structure

lib/
├── core/{frontmatter.js, file-service.js, sync-engine.js}
├── providers/{base-provider.js (NEW), github-provider.js, azure-provider.js (4→1), cli-wrapper.js (NEW)}
├── services/{base-service.js (NEW) + 6 services extending it}
└── utils/{batch-processor.js (2→1), rate-limiter.js, circuit-breaker.js}

.claude/
├── scripts/lib/common.sh        # 5 util files → 1, with README of functions
├── scripts/pm/*.js              # .js canonical; .sh deprecated
└── commands/pm:epic-sync.md     # single version; -original/-modular deleted

.claude/MANIFEST.json            # NEW: install manifest replaces hardcoded array
packages/plugin-core/rules/      # reference core rules, don't copy

---

Distribution & release impact

This repo has no Docker — distribution = npm publish + autopm install/update copying files into user projects.

Current: installItems hardcoded in install.js (lines 37-53); users update via npm update claude-autopm + autopm update --merge (bin/autopm.js lines 58-78 → install/update.sh).

Post-refactor release procedure:

npm run test:all                  # full suite before publishing
npm run build:manifest            # NEW: generate MANIFEST.json from structure
npm run validate:duplicates       # NEW: CI gate — fails if dup functions/files reappear
npm version minor                 # 3.25.5 → 3.26.0 (breaking: removed -original/-modular commands)
npm publish --access public

# Users then:
npm update claude-autopm
autopm update --merge             # merges CLAUDE.md, resolves conflicts

Breaking-change note for CHANGELOG: pm:epic-sync-original and pm:epic-sync-modular removed (alias → pm:epic-sync via MANIFEST aliases map); deprecated .sh command scripts print pointer to .js versions.

---

Execution plan

Phase 1 — Consolidation

• lib/core/frontmatter.js single source; migrate filter-engine.js, ContextService.js, .claude/lib/frontmatter.js consumers; npm test
• Merge 5 shell util libs into .claude/scripts/lib/common.sh + README; update all .sh sources; run bash test suite
• Epic-sync: confirm current version canonical; delete -original and -modular from all 3 locations (9 files); consolidate epic-sync .js/.sh into one .js implementation
• For each of 14 .js/.sh pairs: diff, keep .js, deprecate .sh with pointer; npm run test:install
• npm run test:all && npm run test:install:scenarios

Phase 2 — Service layer

• lib/services/base-service.js (loadFiles/parseFile/writeFile/filter/search + uniform errors)
• Refactor PRD/Epic/Task/Issue/Context/Agent services to extend it; npm run test:unit after each
• lib/core/file-service.js shared markdown ops

Phase 3 — Providers

• lib/providers/base-provider.js abstract (authenticate/create/read/update/delete/search + retry + rate limiting)
• lib/providers/cli-wrapper.js unified gh/az abstraction
• Consolidate 4 Azure variants → single azure-provider.js; GitHubProvider extends base
• Merge 2 batch processors; compose with RateLimiter + CircuitBreaker
• npm run test:github:integration && npm run test:integration:azure

Phase 4 — Distribution

• Remove /autopm/.claude/ duplication (template generated at build, not hand-maintained)
• plugin-core rules import from core instead of copies
• Resolve .md/.xml rule duplication (keep .md, regenerate .xml bridge if @include requires it)
• Create .claude/MANIFEST.json; install.js reads manifest instead of hardcoded array
• scripts/build-manifest.js + scripts/validate-duplicates.js wired into CI
• npm run test:install:scenarios (lite, github, azure, docker, full, performance)

Phase 5 — Release

• npm run test:full && npm run test:security && npm run test:regression && npm run test:cli
• CHANGELOG entry (breaking changes listed above); lib/ARCHITECTURE.md
• npm version minor && npm publish --access public
• Upgrade test in a real project: npm update claude-autopm && autopm update --merge → verify /pm:help, /pm:status work and CLAUDE.md merged cleanly

---

Verification

• npm run test:all — zero regressions
• npm run validate:duplicates — 0 remaining (identical function names across modules, identical .sh files, duplicate command .md files)
• npm run test:performance — sync speed not regressed
• Fresh install smoke: npm i claude-autopm@3.26.0 && autopm install && autopm status
• Expected outcome: single frontmatter parser, single epic-sync, unified service+provider layers, 15-20% LOC reduction, shell utils 1333→300 lines

[rafeekpro]

🔍 Audit addendum — Security / Performance / Testing / Best practices

Extends the refactor to optimization, security, testing, best practices. This framework shells out to gh/az CLIs, so command-injection is the dominant risk class. file:line cited.

Security audit

1. HIGH — Command injection in AzureDevOpsCliWrapper.js lib/providers/AzureDevOpsCliWrapper.js:144,169,172,200,204,208,250-256,281-292: user-controlled filters.name/type/folder/tag interpolated raw into JMESPath/CLI args. $(...) / quote-break → arbitrary shell execution. Use the Azure REST client (already in repo) or shell-escape + strict charset validation.
2. HIGH — Azure PAT in execSync env AzureDevOpsCliWrapper.js:75-76: AZURE_DEVOPS_EXT_PAT in subprocess env appears in process listings / error output. Prefer REST (azure-devops-node-api) with HTTP auth.
3. HIGH — Missing input validation across wrapper methods (variableGroupList, getVariableGroup(id) no numeric check, etc.).
4. HIGH — Path-traversal surface in install install/install.js:299-375: copyDirectory()/installFramework() verify target is a dir but don't block symlinks/.. escaping the target. Add realpath containment check.
5. MED — No package-lock bundled (.npmignore:57): users install mutable transitive versions of @anthropic-ai/sdk, azure-devops-node-api, @octokit/rest. Supply-chain risk.
6. MED — Loose execSync error handling install/install.js:453: npm install failure caught loosely → silent partial install.
7. MED — 31/165 shell scripts lack set -euo pipefail (scripts/local-test-runner.sh, setup-githooks.sh, test.sh, +28).
8. LOW — .npmignore lacks .env.*.local/.secrets/credentials.json patterns (only .env excluded).

Performance optimization

1. MED — Sequential GitHub issue creation (.claude/scripts/pm-sync-upload-local.js): await octokit.issues.create() per item, no batching. Use the existing BatchProcessor → ~10× for 100+ issues.
2. MED — Repeated file reads of same .claude/ metadata across services (EpicService.js:1248, IssueService.js:1030 both load sync-map separately). Shared cache layer → ~30%.
3. MED — Duplicate rate-limit logic batch-processor.js:256-275 vs executeWithRetry():189 may double-wait. Consolidate.
4. MED — No pagination GitHubProvider.listIssues():116-143 (caps at 100, silent data loss on big repos). Use octokit.paginate().

Testing gaps

Framework split: Jest (194 files) + node --test. Coverage threshold 50% — too low, and collectCoverageFrom only covers autopm/.claude/scripts/**, missing lib/, install/, bin/.

• Raise threshold to 80% global, 100% for lib/providers/, 90% for install/
• Fix collectCoverageFrom to include lib/, install/, bin/
• AzureDevOpsCliWrapper — injection-prevention unit tests (every interpolation point), retry tests (currently no test file)
• batch-processor — concurrency stress (200 items, maxConcurrent=10), rate-limit edge cases
• GitHubProvider — rate-limit header handling, pagination
• Install — path-traversal attack tests
• Smoke tests for the 40+ untested .claude/scripts/pm/*.js

Best-practice remediations

1. Fix command injection (security build(deps): Bump actions/checkout from 4 to 5 #1) — highest priority; ties to provider unification (refactor finding feat: Establish PM and Azure systems as complementary solutions #5/Unify Git strategy and implement provider-based architecture #11: a single hardened cli-wrapper.js).
2. Add set -euo pipefail to the 31 unprotected scripts.
3. Replace 922 console.* calls with structured logging (pino/winston) → log levels + audit trail.
4. Centralize hardcoded .claude/ paths (15+ sites) into lib/config.js (ties to refactor finding on hardcoded install array → MANIFEST.json).
5. Add npm audit + security lint to the test pipeline.
   ✅ Verified OK: .env excluded from npm package; async/await consistency ~95% (only 4 .then() sites).

[rafeekpro]

Overlap analysis vs the just-completed epic #605 (all 7 sub-issues merged):

Covered by #605:

• Triplication of distribution copies (.claude / autopm/.claude / packages): now mechanically governed — plugin packages are the source of truth with sync:scripts / sync:commands + CI divergence gates ([chore][scripts] Enforce plugin-core as single source of truth with automated sync + CI divergence check #610/[refactor][prompts] Dedupe 44 duplicated commands and extract boilerplate via @include (≥20% token cut) #609), and root .claude/ is classified & refreshed as a consumer ([chore][repo] Refresh root .claude/ as consumer of improved framework #612, see .claude/README.md).

Still open — the real remaining scope of this issue:

1. Frontmatter parsing JS consolidation — 3–4 independent implementations (lib/filter-engine.js, lib/services/ContextService.js, .claude/lib/frontmatter.js + copy) — untouched by [refactor][core] Framework-wide optimization: prompts, token usage, code quality, security, tests #605.
2. epic-sync variant consolidation — the 3 competing command versions (epic-sync / -original / -modular) were synced across locations but not unified into one.
3. Parallel .sh/.js implementations of the same commands.
4. Docker redeploy verification.

Suggest rescoping this issue to those 4 deltas (its duplication-of-copies half is done). Happy to update the body and decompose if you want to proceed.