Skip to content

Latest commit

 

History

History
221 lines (175 loc) · 20.2 KB

File metadata and controls

221 lines (175 loc) · 20.2 KB

Backlog

A living list of work beyond the initial migration, organized into seven buckets. Each item carries a rough impact (1–5) and effort (1–5) score.

How to read the scores. Impact 5 = changes what users can do, not just how. Effort 5 = multi-week, touches several components, needs experimentation. Priority is not strictly impact÷effort — high-impact items are often worth doing even when expensive — but the ratio is a useful first filter.

Cross-references. Items tagged [cmp-E-N] were added or refined on 2026-06-03 after the comparative analysis against six adjacent AI-dev frameworks (comparative-analysis.md). Items tagged [hist-N] came from audit-archive/HISTORY.md § Between-cycle observations. Where multiple sources converge on the same idea, that's recorded inline.

Shipped

Completed backlog items are preserved here so the active backlog tables stay scannable.

Bucket # Item I E Shipped
A A1 Gemini CLI adapter 4 2 v0.2.0 · CHANGELOG
A A4 Pluggable adapter discovery 3 2 landed · CHANGELOG
A A6 Native Windows validation and support 2 2 landed · Node 22 windows-latest portability smoke
B B1 Accessibility audit stage 4 2 v0.2.0 · CHANGELOG
B B2 Performance budget stage 4 3 v0.6.0 · CHANGELOG
B B3 Deploy cost gate 4 2 landed in PR #221 · CHANGELOG
B B4 Observability gate 4 2 v0.2.0 · CHANGELOG
B B5 Migration safety stage 5 3 v0.4.0 · CHANGELOG
B B6 Documentation gate 3 2 landed in PR #225 · CHANGELOG
B B7 Multi-language QA 4 4 Unreleased · Phase 19 · PR #264
B B8 Cross-artifact consistency analyze [cmp-E-1] 4 2 v0.6.0 · CHANGELOG
B B9 Bounded workspace deltas [cmp-E-2] 4 3 v0.6.0 · CHANGELOG
B B10 Discover Standards preprocessing [cmp-E-5] 3 3 v0.6.0 · CHANGELOG
C C1 Filesystem-level allowedWrites enforcement 4 4 v0.6.0 · CHANGELOG
C C2 Secret scanning hook 4 1 v0.2.0 · CHANGELOG
C C3 License compatibility gate 3 1 v0.6.0 · CHANGELOG
C C4 Reproducible runs (recording half) 4 4 v0.4.0 · CHANGELOG
C C5 Capability-required permissions 3 2 v0.6.0 · CHANGELOG
C C6 Tamper-evident gate chain 3 3 v0.6.0 · CHANGELOG
C C7 eslint-plugin-security [hist-a] 3 1 v0.6.0 · CHANGELOG
C C8 CHANGELOG-per-PR fragments [hist-b] 3 2 v0.6.0 · CHANGELOG
D D1 OpenTelemetry tracing per stage 5 3 v0.2.0 · CHANGELOG
D D2 Gate-pass-rate dashboards 4 2 v0.2.0 · CHANGELOG
D D3 Lessons-learned across projects (org-shared) 5 4 v0.3.0 · CHANGELOG
D D4 Per-role per-model performance scores 5 3 v0.3.0 · CHANGELOG
D D5 Adaptive routing 5 3 v0.3.0 · CHANGELOG
D D6 Cost telemetry 4 2 v0.3.0 · CHANGELOG
D D7 Persistent project memory (embeddings index) 5 4 v0.2.0 · CHANGELOG
E E1 devteam status rich CLI output 3 1 v0.1.0 as devteam summary; Phase 11.1-11.3 updates · CHANGELOG
E E2 Web UI for pipeline runs 4 4 v0.2.0 · CHANGELOG
E E4 Live streaming output 3 2 landed in shared headless invoke helper · CHANGELOG
E E5 Pre-flight check (devteam doctor) 3 1 v0.1.0 plus Phase 14.2-14.3 updates · CHANGELOG
E E6 devteam replay <stage-id> 3 3 v0.4.0 · CHANGELOG
E E7 /goal integration for convergence-shaped stages 3 2 v0.6.0 · CHANGELOG
E E8 Codebase audit feature 5 3 v0.3.0 · CHANGELOG
E E10 Autonomous run watch mode 3 1 PR #268 · Phase 20
E E11 Prototype mode 4 2 Unreleased · pre-SDLC packet for fast learning, optional host-run builds, feedback, and explicit promotion into a normal delivery track. See Phase 23 plan.
F F1 GitHub PR integration 4 3 v0.2.0 · CHANGELOG
F F4 CI runner integration 4 3 v0.4.0 (GitHub Actions only) · CHANGELOG
G G1 Multi-model peer review v0.2.0 · CHANGELOG
G G2 Closed-loop acceptance criteria → exec spec → tests v0.4.0 · CHANGELOG
G G3 Production feedback loop landed · CHANGELOG
G G4 Red-team role between build and peer-review v0.3.0 · CHANGELOG
G G6 Stage shopping (AI-inferred tracks) v0.6.0 · CHANGELOG
G G7 Verification beyond tests v0.4.0 · CHANGELOG
G G8 Long-context architecture continuity v0.3.0 · CHANGELOG
G G10 Role tool budgets landed · CHANGELOG
G G11 devteam run --repair — bug-fix intent mode (ADR-009) complete (Phase 10) · CHANGELOG

A. Reach — more hosts, more deployment targets

# Idea I E Notes
A2 Cursor / Windsurf / Aider / Cline adapters 3 3 One per IDE-embedded agent. Each is an adapter, mostly install-payload work.
A3 Cloud-runner adapter (e.g. AWS Lambda + Bedrock, Replit Agent) 4 4 Host adapter that runs one workstream on a remote worker, not the user's laptop. Enables long-running stages (multi-hour audits, big test suites). Phase 21 plan proposed for review.
A5 API-direct adapter (no host CLI; talks to Anthropic / OpenAI / Google APIs directly) 3 3 For users who don't want to install claude-code or codex but still want orchestration. Lighter dependency footprint.
A7 Docker-based headless runner (#282) 4 2 ✅ Unreleased · hosts/docker/ packages Stagecraft into a non-root Docker runner for unattended local orchestration against a mounted project, with runtime-only credentials and conservative lock handling.
A8 Omnigent runtime adapter follow-through (#291) 4 4 Phase 24.1 ships a conservative omnigent host adapter. Follow-through tracks launch config (#292), prompt-file transport (#293), policy bridging (#294), session evidence (#295), and optional director-mode consolidation (#296). See Phase 24.

B. Pipeline depth — more/richer stages

No open items. B7 moved to Shipped in Phase 19.

C. Quality & safety — enforcement, sandboxing, scanning

# Idea I E Notes
C1 Filesystem-level allowedWrites enforcement 4 4 ✅ v0.6.0 · CHANGELOG
C2 Secret scanning hook 4 1 ✅ v0.2.0 · CHANGELOG
C3 License compatibility gate 3 1 ✅ v0.6.0 · CHANGELOG
C4 Reproducible runs (recording half) 4 4 ✅ v0.4.0 · CHANGELOG
C5 Capability-required permissions 3 2 ✅ v0.6.0 · CHANGELOG
C6 Tamper-evident gate chain 3 3 ✅ v0.6.0 · CHANGELOG
C7 eslint-plugin-security [hist-a] 3 1 ✅ v0.6.0 · CHANGELOG
C8 CHANGELOG-per-PR fragments [hist-b] 3 2 ✅ v0.6.0 · CHANGELOG
C9 Verify-before-promoting enforcement in audit skill [hist-c] 3 2 ✅ Unreleased · Phase 1/2 audit findings now require verified_by evidence, templates expose verification slots, and structural tests lock the contract.

D. Observability & learning — telemetry, metrics, persistent learning

# Idea I E Notes
D5 D5 maturation — continuous adaptive routing 5 3 Today D5 proposes role-level swaps; the mature form re-routes the next run based on the prior run's outcomes automatically. Phase 17 makes per-workstream dispatch history durable and privacy-bounded; the gate stays shut pending ≥5 dispatches per (role, host) pair across ≥2 real user projects and cost telemetry.

E. Developer experience

# Idea I E Notes
E3 VS Code extension 3 3 Sidebar with stage status, "run next stage" button, gate viewer.
E9 Conversational stage mode [cmp-E-4] 3 3 devteam stage requirements --interactive opens a conversation with the PM subagent to refine the brief through Q&A before producing the artifact. Useful specifically for upstream stages (requirements, design, clarification) where the artifact benefits from refinement before being rendered. Architecture supports it (adapters could expose streaming-conversation alongside one-shot render). Implement if user feedback indicates the gate-driven loop is too rigid for upstream stages. Related to E7 but different mechanism: E7 is host-loops-until-condition; E9 is stage-manager-converses-with-agent.
E11 Prototype mode 4 2 devteam prototype creates a lightweight pre-SDLC packet, can run the build prompt in a packet workspace, captures feedback, and records explicit discard/iterate/promote decisions. It is deliberately not a production gate track; promotion hands off into devteam run --feature-file ... --track <t>.

F. Integrations — where the team plugs in

# Idea I E Notes
F2 Jira/Linear ticket integration 3 2 devteam stage requirements --ticket FOO-123 pulls the ticket as the feature brief input. Gates link back to the ticket.
F3 Slack/Discord notifications 3 1 Pipeline events (stage start, fail, escalate) post to a channel. Triggers for human checkpoints.
F5 Pre-commit hook integration 3 1 Optional pre-commit hook that runs the relevant track for the change (nano if config-only, full if otherwise).

G. Innovation bets — speculative, future-oriented

These don't fit neatly in impact/effort because their value depends on how the field evolves. They are the items that would most meaningfully differentiate this tool.

G5. Multi-modal stages

Design specs include architecture diagrams (images). Stage 2 (design) and Stage 5 (review) accept image inputs. Principal can output a system diagram, not just prose. Visual reasoning is no longer a separate workflow.

G9. Self-modifying pipeline

Retrospective stage proposes changes to stages.js / roles/ / rules/ based on what worked. Proposals queue for human approval. The pipeline learns its own shape from operation.


Priority queue (2026-06-19 — Phase 19 closeout)

The full evidence, effort/risk ratings, dependencies, and PR sequence now live in the current audit backlog and roadmap.

Immediate and near-term

No ungated implementation item remains from the audit's immediate and targeted improvement batches. Phase 16 completed privacy-safe readiness/export, Phase 17 made dispatch evidence durable, Phase 18 added explicit accepted-resolution evidence for H3, Phase 19 shipped polyglot verification in PR #264, and Phase 20 implements the separable devteam run --watch operator UX without enabling active stall response. The next capability horizon is real collection followed by review, not calendar-driven activation. E9 conversational stage mode remains a discovery proposal until five real users report upstream rigidity.

Completed from this audit cycle: dashboard HTML safety and lifecycle (PR #235), native Windows CI evidence, support wording, and A6 promotion (PR #236), and bounded durable transcript streaming (PR #237). Current-truth reconciliation removed the remaining P1-3 ownership, vocabulary, comment, count, link, and provider drift. Stable-fact consistency now locks schema vocabulary, Node/platform support, and the absence of volatile test totals while leaving audit history untouched (audit P2-3). The autonomous driver decomposition is complete: characterization, dispatch/transient, and fix/ruling/merge transitions landed as three behavior-preserving slices while run() retained lock, loop, effect, and final-persistence ownership (audit P2-2).

Evidence-gated next horizon

  • P3-1 — evidence readiness and export. Phase 16 implements the approved privacy model, read-only local readiness, consented aggregate export, identity lifecycle, strict bundle validation, and explicit multi-project analysis. See plans/phase-16-evidence-readiness-and-export.md. Phase 17 adds allowlisted per-workstream dispatch events so D5 evidence accumulates during normal autonomous runs without reconstructing history from gate snapshots. See plans/phase-17-durable-evidence-instrumentation.md. Phase 18 adds explicit, hash-bound human acceptance for successful fix/retry resolutions so H3's derivability threshold can be measured without exporting recipe text. See plans/phase-18-accepted-resolution-evidence.md. This makes the gates below measurable; it does not open them.

  • D5 maturation — continuous adaptive routing. Today D5 proposes role-level swaps; the mature form re-routes the next run based on the prior run's outcomes automatically. Evidence baseline (2026-06-14, plans/adaptive-routing-evidence.md): zero real-run telemetry at review time. Phase 17 starts durable collection from real autonomous dispatches; it does not backfill old gates. Gate stays shut pending ≥5 durable dispatches per (role, host) pair across ≥2 real user projects and cost telemetry. ADR-007 Tier 1 (liveness heartbeat + observe-only stall probe) implemented in Phase 11.1; ADR-008 (advisory sweep + --fail-on-advisory) implemented in Phase 11.2; ADR-007 Tier 2 remains evidence-gated.

  • H3 — Recipe factory (escalation→recipe learning) (Phase 3 of ADR-003 · design). Persist resolved escalations as semantically-indexed fix-recipes via the existing core/memory/ embedding store (D7); computeFixSteps consults it on a FAIL signature before escalating, so recurring derivable failures resolve deterministically. Evidence review done (2026-06-14, plans/h3-ground-truth.md): zero real run logs/archives and no recurring unresolved class. Phase 18 makes explicit acceptance measurable under ADR-012. Gate stays shut pending ≥2 real projects each with ≥5 autonomous fix/retry runs, the same schema-bound failure accepted ≥3 times across both projects, and ≥80% derivability. Tracked by GitHub #142.

  • ADR-005 standing grants. Keep deferred until at least 10 repair runs across 2+ projects and consequence-ceiling halt data establish which grants operators routinely approve. Tracked by GitHub #144.

  • ADR-007 Tier 2 active stall response. Keep deferred until real stall-detected events calibrate frequency and threshold. Tracked by GitHub #145.

Consciously deprioritized

Five items that the comparative analysis or shifted context argues against investing in now:

  • E3 — VS Code extension. Stagecraft sits above the IDE, not inside it. Building an editor extension works against that positioning, and IDE-native tooling is a crowded category.
  • A2 — Cursor/Windsurf/Aider/Cline adapters. Supporting 30+ AI agents is a maintenance treadmill. Land A4 — Pluggable adapter discovery first and let the long tail be community-built.
  • F2 / F3 / F5 — Jira / Slack / pre-commit integration. None changes what Stagecraft can do. Accept community PRs but don't invest core time.
  • G9 — Self-modifying pipeline. Premature. Wait until multiple teams use the platform in different configurations before optimizing for any one signal.

Staying ahead of the curve — bets

Six positioning bets about where software development is heading.

1. Models keep getting smarter, cheaper, faster.

Design the contract assuming 10× capability in 2 years. The schema (gate JSON), the contract (per-workstream gates merged to stage), and the routing layer should outlive the specific models we route to today.

2. Diversity beats monoculture.

Single-model agentic systems are giving way to multi-model coordinated systems. For non-trivial tasks, the stronger outcome usually involves 2–3 different model families. The routing layer is already built for this; the next step is making diversity structurally load-bearing (G1 multi-model peer review, D4/D5 adaptive routing).

3. Evals are the rate-limit.

The pipeline produces structured gate JSON so evals can be built on top. Every refinement should make the gate richer and more measurable. D1/D2/D4 are all expressions of this bet.

4. Memory and persistence are the next frontier.

Today's pipeline is mostly stateless within a run; each new run starts fresh. Sustained coordination across projects, with continuous learning, requires the work in D3/D7/G8.

5. Tool depth beats raw intelligence.

An agent with a deep, well-composed tool stack outperforms one that only writes text. Role briefs are the place to encode tool budgets per role (G10). Skills are early steps; tool negotiation is the mature form.

6. Compliance and auditability are coming, fast.

EU AI Act, US executive orders, SOC 2 controls: all require reproducible runs, audit trails, and documented decision provenance. C4 (reproducible runs), C6 (tamper-evident gate chain), and D1 (tracing) address this directly.

The unit is the team, not the model.

A coordinated team of specialized agents — each with a role, a tool budget, a gate contract, and shared memory — outperforms a single model. The model is a substrate. The team is the product.