Self-host spectacles: build spectacles with spectacles

[norrietaylor]

Context

Spectacles is a complete GitHub-native SDD pipeline (sdd:spec → sdd:triage → sdd:ready → sdd:in-progress → sdd:review → sdd:done) but it was built conventionally — it has never built itself. ADR 0003 §3 deferred self-hosting until the pipeline had a clean run and a validator/reviewer safety net.

That net now exists: sdd-validate and sdd-review are live, spectacles-bot is installed on this repo for lock recompilation (ADR 0019), and auto-recompile-on-merge works. Goal: turn the pipeline inward permanently so future features (e.g. #229, #230) ship through spectacles itself.

Decisions: enable self-hosting (permanent), relax protected paths so sdd-execute can edit .github/ on this repo, first-target feature deferred until infra is in place.

Two structural facts that shape everything

1. Protected paths are enforced in prompts, not CI. No CI job fails a PR touching .github/. The constraint lives in three places:

   • shared/sdd-gates.md impl gate (protected-path change is a Blocker) — ~L175-176
   • .github/workflows/sdd-execute-haiku.md step 5 + Boundaries (~L412-425, L654), identical in -sonnet/-opus
   • shared/sdd-mcp-serena.md write-scope line (~L241)

   Relaxing protected paths is a fragment/source edit, which auto-recompiles via recompile-locks.yml on merge.

2. Locks cannot be correctly compiled on a feature branch. gh-aw resolves pinned @main imports from remote main only (ADR 0018). Any feature PR editing a shared/*.md fragment or .github/workflows/*.md source leaves the committed .lock.yml stale and reds the compile drift gate — the lock only regenerates after merge, by recompile-locks.yml. Central seam, addressed in Phase 4.

Phase 0 — Decision record (keystone, human PR)

Write ADR 0020: Self-hosting spectacles on its own repository (decisions/0020-self-hosting.md):

• Supersede ADR 0003 §3; record the safety-net rationale (validate+review live, App installed, auto-recompile working, every agent PR still human-reviewed).
• Redefine the protected set. Relaxing ≠ removing. Keep agent-off-limits: *.lock.yml (generated, ADR 0002/0004), recompile-locks.yml, branch ruleset 16496590, secrets/credentials, scripts/digest-snapshot.yml (generated). Make agent-writable on this repo: .github/workflows/*.md, .github/actions/**, wrappers/*.yml, shared/*.md, decisions/*.md, templates/**, docs/**. (.github/actions/** is required for Planning hardening: latent-cycle detection + spike primitive for unproven assumptions #229 Track A — the sdd-cycle-detect / sdd-route-triage composite actions — and was missing from the original set; the blanket protect_top_level_dot_folders guard must be relaxed for it too, not just .github/workflows/.)
• Record the Phase 4 lock-drift resolution; verification + rollback (revert Phase 2 wrapper installs).

Phase 1 — Provision operator infra (operator, needs repo owner)

App is installed but scoped to contents:write + workflows:write for recompile only. Self-hosting needs the full SDD surface:

Name	Type	Purpose	Status
APP_ID	var	App token minting	present
APP_PRIVATE_KEY	secret	App token minting	present
COPILOT_GITHUB_TOKEN	secret	agent code-gen engine	set
DISTILLERY_MCP_URL	var	retrieval endpoint	set
DISTILLERY_OAUTH_TOKEN	secret	retrieval auth	set
DISTILLERY_PROJECT	var	scope queries to this repo	set
SERENA_LANGUAGE_SERVERS	var	code intel (repo is md/py/yaml → likely empty)	decide
SDD_AUTO_MERGE	var	auto-merge green PRs	leave off initially
SDD_DISPATCH_MAX_PARALLEL	var	fan-out cap (default 5)	optional

• Expand App installation permissions on this repo to add issues: write + pull-requests: write (currently recompile-only). GitHub settings action — repo owner only.
• Keep SDD_AUTO_MERGE unset for the first inward runs.

Phase 2 — Install SDD wrappers into .github/workflows/

Wrappers live in wrappers/ as distributable artifacts; they are not active workflows here. Activate:

• Copy wrappers/*.yml → .github/workflows/ (what scripts/quick-setup.sh --suite sdd does for consumers), pinning uses: ref to @main so the repo self-references its own hosted locks.
• Sync labels (templates/.github/labels.yml) + issue templates via quick-setup.sh.
• Confirm self-referencing uses: norrietaylor/spectacles/.github/workflows/<agent>.lock.yml@main resolves.

Phase 3 — Relax protected-path enforcement (fragment/source edits)

Apply the ADR 0020 protected set to the three enforcement points (do not hand-touch any .lock.yml):

• shared/sdd-gates.md impl gate → Blocker only for narrowed set (locks, secrets, recompile/ruleset infra); workflow sources + wrappers become in-scope warnings.
• .github/workflows/sdd-execute-haiku.md step 5 + Boundaries (+ -sonnet/-opus, templated from same body) → narrowed list; keep needs-human escalation for the still-protected set.
• shared/sdd-mcp-serena.md write-scope line → same narrowing.
• These edits red compile until merge → recompile; land as a human PR.

Phase 4 — Lock-drift-on-feature-PR seam — ✅ LIVE

Resolved. The lock-drift handling is already in place: the compile drift gate is advisory on sdd/* branches and recompile-locks.yml regreens main post-merge. No further work in this phase; remaining phases can rely on it. Original plan retained below for the ADR 0020 record:

• Make the compile drift gate advisory (non-required) on sdd/* branches, relying on recompile-locks.yml to green main post-merge. Implement as a branch/event condition in lint.yml drift step; keep the gate required on human PRs.
• Document the post-merge sequence: agent PR edits source → merges (compile advisory) → recompile-locks.yml fires → locks regenerate → main green.
• Rejected alt: sdd-execute running gh aw compile in-branch — incorrect, @main imports resolve from remote main, inlining the old fragment.

Phase 5 — First inward run (target deferred, guarded)

• Open a tracking issue, apply sdd:spec.
• Recommended first target: a shared/-only or docs/-only change (lowest blast radius), or Planning hardening: latent-cycle detection + spike primitive for unproven assumptions #229 Track A (latent-cycle: shared/*.md + .github/actions/). Avoid Advance spec + arch status frontmatter through the SDD lifecycle #230 as first target (heavy .github/workflows/ surface).
• Run full pipeline; merge by hand (SDD_AUTO_MERGE off). This run supersedes ADR 0003's fixture-run verification.

Verification

End-to-end success = one real spectacles feature shipped by spectacles:

1. Tracking issue reaches sdd:done.
2. sdd-execute opens an sdd/* PR editing a previously protected path (proves Phase 3), passes sdd-validate (no false Blocker) + sdd-review.
3. On merge, main returns green via recompile-locks (proves Phase 4).
4. App authored comments/labels/PR as spectacles-bot[bot] (proves Phase 1).

Guardrails: low-blast-radius first target, SDD_AUTO_MERGE unset, every PR human-reviewed. Rollback = remove installed wrappers + revert ADR 0020 and the three fragment edits.

Needs repo owner

• App permission expansion + secret/variable creation (Phase 1).
• SERENA_LANGUAGE_SERVERS value (likely none for a md/py/yaml repo).