Follow-up to #42, which measured the symptom precisely. This issue proposes the architectural response, as distinct from the four targeted reductions (#42 A–D).
Why the targeted fixes are necessary but not sufficient
#42's attribution is the important datum: 84% of Linear traffic is four per-issue re-fetch loops, and complexityRemaining sat at 2,999,817/3,000,000 — essentially untouched. That is not expensive queries. It is thousands of tiny ones.
That shape says the problem is not tuning. Each of those four loops exists because some code path needed a field it did not keep. Capping them individually — a TTL on the audit refresh, a per-tick budget on checkTrackerReplies — treats the same cause four times, and a fifth loop added next quarter restarts the cycle.
The daemon currently treats Linear as the source of truth and re-derives its own state from it on every tick.
Proposal: local projection + outbox
Invert it. Linear becomes an event source and a write target; the daemon owns a local projection it can read for free.
1. One bulk read per tick, not N per-issue reads.
FetchIssuesByStates already returns full issues with relations. The four per-issue loops exist because each wants a field the bulk fetch already carried and nobody stored. A local table keyed by issue ID, refreshed from the bulk poll, serves checkTrackerReplies, the input-required replay, the pending-resume path and the dependency audit from memory. That is the 84% collapsing at its source.
2. Batch the writes.
UpdateIssueState (internal/tracker/linear/client.go:255) costs 2 requests because it re-resolves the team's name → UUID map on every single transition — a map that is effectively static. Caching it (#42 suggestion A) halves every transition. Batching goes further: Linear's GraphQL API accepts multiple mutations per document, so N transitions in a tick become one request.
3. A write-ahead outbox is what actually breaks the feedback loop.
This is the part that matters most. Today, recovery requires a successful read — internal/orchestrator/event_loop.go:628 resumes a pending_input_resume entry only after FetchIssueDetail succeeds. So when the budget is exhausted the queue cannot drain, which keeps the backlog high, which keeps the budget exhausted. #42 documents entries sitting for 75+ minutes.
With an outbox, a state transition is recorded locally and durably first, then flushed. Failing reads can no longer block writes, because the write already happened locally. #42's suggestion E achieves this at the scheduler level (shed polling reads, admit writes); an outbox achieves it at the persistence level and additionally survives a restart.
Existing groundwork
The infrastructure pattern is already here: the automation queue and dependency audit are persisted through .itervox/ today, and the local tracker uses a per-repo SQLite database. This is not a new class of state for the daemon.
Costs, stated plainly
This is materially bigger than #42 A–D and should not be started under rate-limit pressure.
- Reconciliation. Local says
Done, Linear says In Progress — who wins, and when? Every projection design owes an answer.
- UI honesty. The dashboard would show state Linear has not yet accepted. That is a product decision, not just an implementation detail.
- Scope. A–D is roughly a week; this is a redesign of the tracker boundary.
Recommendation
Do #42 A and B first — the state-map cache and persisting the input-required detail cache are ~58% of traffic between them, both small and local — then decide on this with real headroom.
Skipping C and D in favour of the projection is defensible: a staleness TTL and a per-tick cap are precisely the band-aids the projection makes unnecessary. E is not optional either way — reads must never be able to starve the writes that let the queue drain.
Follow-up to #42, which measured the symptom precisely. This issue proposes the architectural response, as distinct from the four targeted reductions (#42 A–D).
Why the targeted fixes are necessary but not sufficient
#42's attribution is the important datum: 84% of Linear traffic is four per-issue re-fetch loops, and
complexityRemainingsat at 2,999,817/3,000,000 — essentially untouched. That is not expensive queries. It is thousands of tiny ones.That shape says the problem is not tuning. Each of those four loops exists because some code path needed a field it did not keep. Capping them individually — a TTL on the audit refresh, a per-tick budget on
checkTrackerReplies— treats the same cause four times, and a fifth loop added next quarter restarts the cycle.The daemon currently treats Linear as the source of truth and re-derives its own state from it on every tick.
Proposal: local projection + outbox
Invert it. Linear becomes an event source and a write target; the daemon owns a local projection it can read for free.
1. One bulk read per tick, not N per-issue reads.
FetchIssuesByStatesalready returns full issues with relations. The four per-issue loops exist because each wants a field the bulk fetch already carried and nobody stored. A local table keyed by issue ID, refreshed from the bulk poll, servescheckTrackerReplies, the input-required replay, the pending-resume path and the dependency audit from memory. That is the 84% collapsing at its source.2. Batch the writes.
UpdateIssueState(internal/tracker/linear/client.go:255) costs 2 requests because it re-resolves the team'sname → UUIDmap on every single transition — a map that is effectively static. Caching it (#42 suggestion A) halves every transition. Batching goes further: Linear's GraphQL API accepts multiple mutations per document, so N transitions in a tick become one request.3. A write-ahead outbox is what actually breaks the feedback loop.
This is the part that matters most. Today, recovery requires a successful read —
internal/orchestrator/event_loop.go:628resumes apending_input_resumeentry only afterFetchIssueDetailsucceeds. So when the budget is exhausted the queue cannot drain, which keeps the backlog high, which keeps the budget exhausted. #42 documents entries sitting for 75+ minutes.With an outbox, a state transition is recorded locally and durably first, then flushed. Failing reads can no longer block writes, because the write already happened locally. #42's suggestion E achieves this at the scheduler level (shed polling reads, admit writes); an outbox achieves it at the persistence level and additionally survives a restart.
Existing groundwork
The infrastructure pattern is already here: the automation queue and dependency audit are persisted through
.itervox/today, and the local tracker uses a per-repo SQLite database. This is not a new class of state for the daemon.Costs, stated plainly
This is materially bigger than #42 A–D and should not be started under rate-limit pressure.
Done, Linear saysIn Progress— who wins, and when? Every projection design owes an answer.Recommendation
Do #42 A and B first — the state-map cache and persisting the input-required detail cache are ~58% of traffic between them, both small and local — then decide on this with real headroom.
Skipping C and D in favour of the projection is defensible: a staleness TTL and a per-tick cap are precisely the band-aids the projection makes unnecessary. E is not optional either way — reads must never be able to starve the writes that let the queue drain.