Skip to content

decomposer(feedback): execution outcomes back into the drafter #10096

Description

@JSONbored

Problem

Every executed issue is a labelled example — this phrasing, this sizing, this acceptance
detail produced this outcome — and the loop currently does not close. There is a post-merge
feedback path into the drafter, but nothing systematically feeds the quality score back.

Scope

  • Feed per-issue quality scores and their component signals back as drafting signal:
    which phrasings, sizings, and acceptance-detail patterns correlate with clean first-pass
    merges
  • Persist the corpus of (issue text → outcome) pairs in a form usable for evaluation and
    future training, with the same held-out-split discipline used elsewhere in the repo so
    improvements are measured rather than asserted
  • Report drafter performance over time against Epic: issue-quality measurement — score a scoped issue by how cleanly it executed #10076's score, per repo and per drafting
    method, so a regression is visible

Acceptance

Drafter changes are evaluated against held-out historical outcomes rather than judgment,
and the trend is visible.

Note

This is the compounding piece — decomposition quality improves from real execution rather
than synthetic data, and the corpus grows with every merged PR.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions