Audit missing GitHub change signals for agent feedback loops

[diegomarino]

Context

Batutas v2 is being designed as a daily workbench for collaborating with agents across ADEs such as Orca, Paseo, and Agent Orchestrator-style systems.

The motivating problem is that agents cannot be trusted to reliably communicate state to each other or continuously monitor GitHub themselves. gh-delta exists to provide a deterministic GitHub issue/PR change detector.

Recent comparison against AO and Paseo suggests the right boundary is:

• gh-delta detects that relevant GitHub state changed;
• consumers such as Batutas query GitHub for richer details when a delta matters;
• Batutas/ADE adapters decide attention, routing, alerting, and delivery policy.

So this issue is not asking gh-delta to classify agent routing or decide who should act.

Problem

Before Batutas v2 relies on gh-delta as its first SCM signal source, we should audit whether gh-delta misses any GitHub state transitions that are important for agent feedback loops.

Known current coverage includes, among others:

• PR and issue creation/closed/reopened/missing/reappeared lifecycle;
• PR merged;
• comment count increase;
• CI/check changes;
• review decision / latest review changes;
• unresolved review thread count changes;
• total review thread changes;
• draft -> ready;
• mergeability and mergeStateStatus transitions;
• issue label changes;
• optional baseline-state emission for pre-existing open items.

The question is whether any relevant GitHub change signals are absent, not whether wrappers need better routing policy.

Audit Questions

Evaluate at least these candidate gaps:

1. PR review comments edited/deleted when counts do not change.
2. Review threads resolved/reopened cases that may not change the current counters in a detectable way.
3. Issue or PR body/title edits: whether they are intentionally ignored, caught only as opaque updated, or should have explicit classes.
4. Requested reviewers / review request added or removed.
5. Assignee changes.
6. Milestone changes.
7. Base branch changes.
8. Head branch/ref changes beyond head SHA updates.
9. Branch deletion/restore after merge or close, if GitHub exposes it usefully.
10. Project/status fields, if accessible and relevant enough for the package scope.
11. More granular check-run/check-suite transitions, if current ci-changed detail is insufficient for consumers to decide when to query.
12. Issue comments edited/deleted where total count is unchanged.

Expected Output

Produce a short audit matrix:

Signal	Relevant to agent feedback?	Currently detected?	Current class/detail if yes	Proposed action

Actions should be one of:

• already covered;
• covered but docs/tests should clarify;
• intentionally out of scope;
• add new fingerprint field;
• add new delta class;
• add detail only;
• requires consumer follow-up query, no gh-delta change.

Constraints

• Keep gh-delta a detector, not a scheduler, queue, ADE adapter, or routing policy engine.
• Do not add agent-role, audience, attention, or routing concepts to the gh-delta contract.
• Prefer detecting neutral state transitions over interpreting business meaning.
• If a consumer can safely perform a follow-up GitHub query from an existing delta, prefer documenting that over bloating fingerprints.
• Avoid breaking report.schemaVersion === 1; additive classes/details/fields are acceptable only if they fit the existing compatibility policy.

Acceptance Criteria

• The audit identifies which AO/Paseo-style feedback-loop signals gh-delta already detects.
• Any proposed implementation work is limited to missing neutral GitHub change signals.
• If no implementation is needed, the issue can close with docs/test clarifications only.
• Batutas can use the result to decide whether slice 001 Doctor should validate gh-delta as an SCM signal source without requiring private wrapper assumptions.

[diegomarino]

Signal audit — missing GitHub change signals for agent feedback loops

Audit for #23. Method: source review of lib/gh.mjs (fetch surface), lib/fingerprint.mjs / lib/detect.mjs (compared fields and classifiers), lib/contract.mjs / docs/contract.md (class set, detail shapes), plus live API probes run on 2026-08-05 against this repo (merged PR #20; every probe was reverted, final state identical to pre-audit).

Detection model (drives every row below)

gh-delta observes an item through exactly two lenses, which puts every candidate signal into one of three tiers:

1. Fingerprinted field changes → a specific class (ci-changed, new-comments, …).
2. updatedAt bump with no fingerprinted field change → the opaque catch-all updated. Because updatedAt is itself a compared fingerprint field, anything GitHub timestamps is at least detectable — the consumer then does a follow-up gh query for the reason. This is the intended division of labor: gh-delta says "look here", the consumer fetches details.
3. No updatedAt bump and no fingerprinted field change → invisible. For OPEN items this tier is small (phase 1 re-fetches all open items in full every tick, so counters catch even un-timestamped changes); for CLOSED/MERGED items it covers everything, by design (dormant memory under the incremental contract).

Empirical probe results (2026-08-05, PR #20)

Mutation	Bumps parent updatedAt?	Other observable effect
Issue-style (conversation) comment added	yes (13:28:40)	totalCommentsCount +1
Conversation comment edited	yes (13:28:47)	none — count unchanged
Conversation comment deleted	yes (13:29:00)	count −1
Review thread resolved / unresolved	no (unchanged through both toggles)	isResolved / unresolved counter
Review-thread reply (review comment) added	no	totalCommentsCount +1
Review comment edited	no	none — count unchanged
Review comment deleted	no	count −1
Assignee added / removed	yes (13:29:39 / 13:29:43)	none fingerprinted
Head branch deleted (historical, at merge time)	yes (updatedAt == HeadRefDeletedEvent.createdAt)	headRef → null; headRefName retained

Also verified: PullRequest.totalCommentsCount includes review comments and review bodies, not just conversation comments (PR #20: totalCommentsCount: 3 with comments.totalCount: 0, 2 reviews, 1 thread).

Audit matrix

#	Signal	Relevant to agent feedback?	Currently detected?	Current class/detail if yes	Proposed action
1a	PR review comment added/deleted	yes	yes on OPEN PRs — count moves (verified: review comments are in totalCommentsCount); add → new-comments, delete → opaque updated. Invisible on MERGED/CLOSED (no updatedAt bump, verified)	new-comments; updated + comments detail	covered but docs/tests should clarify (optionally add a comments-removed class — classifier-only, id-stable)
1b	PR review comment edited, count unchanged	moderate	no — verified blind spot: no updatedAt bump, no counter change, even on OPEN PRs	—	intentionally out of scope, documented: detection needs per-comment timestamps/bodies (nested pagination + payload bloat, against detector policy)
2	Review threads resolved/reopened without net counter change	yes	mostly — resolve does not bump updatedAt (verified), but OPEN PRs are fully re-fetched so net counter moves are caught. Invisible: offsetting same-tick changes (1 resolved + 1 unresolved) and post-merge/close resolutions	unresolved-threads-added / -resolved / review-threads-changed	add new fingerprint field (optional, low priority): sha1 digest of per-thread (id, isResolved) — nodes already fetched, only id must join the query; else covered-but-docs-clarify
3	Issue/PR body or title edits	moderate	yes, opaque — both bump updatedAt (title also refreshes the delta's title metadata)	updated + updatedAt detail	covered but docs should clarify: intentionally no content fingerprinting; consumer follow-up query for the new content
4	Requested reviewers added/removed	yes — core feedback-loop signal	opaque only — bumps updatedAt → updated	updated	add new fingerprint field + delta class (review-requests-changed): sorted requested-reviewer logins, added/removed detail like relabeled. Small, neutral state; strongest gap found
5	Assignee changes	yes	opaque only (verified: bumps updatedAt)	updated	add new fingerprint field + delta class (assignees-changed, pr + issue): sorted logins, symmetric with labels
6	Milestone changes	low	yes, opaque — bumps updatedAt	updated	requires consumer follow-up query, no gh-delta change
7	Base branch changes	yes — invalidates prior CI/mergeability context	yes, opaque — bumps updatedAt; mergeable/mergeStateStatus churn usually co-fires	updated (+ mergeable/mergeStateStatus details)	add new fingerprint field + delta class (base-changed): single baseRefName string, cheap and unambiguous
8	Head branch/ref changes beyond head SHA (rename)	low	partially — headRefName rides every delta as non-compared metadata; a rename alone is likely invisible until the next delta	headRefName metadata	intentionally out of scope; docs clarify that headRefName is contextual, not compared
9	Branch deletion/restore after merge/close	low	yes, opaque — deletion bumps updatedAt (verified) → updated within the horizon window; the reason is not identified	updated + updatedAt detail	requires consumer follow-up query (headRef == null via GraphQL), no gh-delta change
10	Project/status fields (Projects v2)	situational	no — project-item field edits do not touch the issue/PR updatedAt; separate API surface and token scope	—	intentionally out of scope; consumers query the Projects API directly
11	Granular check-run/check-suite transitions	yes	yes — per-check {name, status, conclusion} rows persist in from.ciChecks / to.ciChecks, ci-changed details name the exact checks that changed, --summaries adds the fail-closed ciRollup verdict	ci-changed + ci detail; summary.ciRollup	already covered; docs could add a "diff from.ciChecks vs to.ciChecks" recipe for consumers
12	Issue comments edited/deleted, total count unchanged	moderate	yes, opaque — edit and delete each bump updatedAt (verified) → updated; delete additionally moves the count	updated + comments/updatedAt details	covered but docs should clarify (same optional comments-removed class as 1a)
+	(bonus) PR label changes	yes	opaque only — the PR query does not fetch labels, so relabeled is issue-only (contract already notes this)	updated	candidate: fetch PR labels and reuse relabeled — unifies the two families with an existing class

Recommended implementation shortlist (all additive, schemaVersion 1-safe)

Priority order, by feedback-loop relevance ÷ cost:

1. review-requests-changed (row 4) — the one signal an agent loop genuinely cannot afford to receive as opaque noise; sorted logins only, no profile payloads.
2. assignees-changed (row 5) — pr + issue, sorted logins.
3. base-changed (row 7) — one string field.
4. PR relabeled (bonus row) — existing class, new fetch field.
5. Optional: thread-state digest (row 2) and comments-removed (rows 1a/12).

Compatibility notes for any of the above:

• Each new compared field needs the mergeStateStatus-style pairwise upgrade suppression in fingerprintChanged so the first post-upgrade tick doesn't emit a fleet-wide updated burst.
• New compared fields enter to, so delta ids shift once per item on its first post-upgrade delta (ids hash the observed state — this is the documented id policy, not a regression); cross-monitor idempotency holds again immediately after.
• New classes are id-neutral (deltaIdentity excludes classes when to != null) and covered by the existing forward-compat rule ("unrecognized class = something changed, inspect").

Explicit non-goals confirmed by this audit

• Content fingerprinting (bodies, comment text) — stays out; updated + follow-up query is the contract.
• Review-comment edit detection (row 1b) — the only true blind spot found; cost (nested comment pagination or body hashing) is disproportionate to the signal. Should be documented as a known limitation in contract.md.
• Projects v2, milestones, branch-restore attribution — consumer follow-up territory.

Acceptance criteria mapping

• AO/Paseo-style signals already detected: everything CI, review-decision, review-thread-counter, merge-state, lifecycle, comment-count related — plus, opaquely via updated, every timestamped mutation (body/title, assignees, milestone, base branch, review requests, branch deletion). The only silent case is review-comment edits and offsetting same-tick thread-resolution churn.
• Proposed work is limited to neutral GitHub state transitions: four small compared fields/classes (reviewer requests, assignees, base ref, PR labels), zero routing/attention concepts.
• Docs-only closure is viable: none of the proposals block Batutas — every relevant transition is at least updated-visible on open items, so slice 001 Doctor can validate gh-delta as an SCM signal source today, with the shortlist as optional sharpening rather than a prerequisite.