Investigative / design umbrella. Customer (PXG team) wants better self-service tooling around Harper's throttling and event-loop backpressure so app developers can react to it earlier and customize behavior, rather than only seeing it as warnings or errors in logs.
Heart of the request (from Dawson)
- Capability to prioritize requests and relieve backpressure when throttling is a problem (e.g. deprio reads vs. writes).
- Earlier mechanisms to detect and deal with throttling.
- Better surfacing of problems in user code that cause throttling — long-running functions that aren't yielding.
- Warnings about read/write transaction queues before they become errors.
Mechanisms to consider
- Easier profiling workflows on top of the 4.7 profiling work.
- Expose queues and warnings as programmatic events / observables.
- A
backpressure hook/function the app dev can call to make app-side decisions (stop accepting new work, shut down a background loop, etc.).
- Expose a curated set of internal utilities (e.g.
server/throttle.ts — the throttle higher-order function) so app devs can use the same heuristics Harper itself uses, without re-implementing them incorrectly.
Why this is Not Ready
The ask is broad enough that it needs decomposition before scheduling:
- Which surface? A
utils global? Plugin-API additions on scope? Operations-API hooks?
- Which signals are stable enough to expose? (read txn queue depth, write queue depth, event-loop utilization, throttle counter…)
- How does the app dev's
backpressure hook compose with Harper's internal throttling — does it pre-empt or coexist?
Decomposition into 2-4 concrete sub-issues is the right next step; this umbrella stays open as the design context.
Related
- bari notes profiling improvements have already landed in 4.7.
- The
throttle higher-order function referenced is in server/throttle.ts.
Tracked in Jira: CORE-3021
Status: Not Ready — needs decomposition
🤖 Filed by Claude on behalf of Kris.
Investigative / design umbrella. Customer (PXG team) wants better self-service tooling around Harper's throttling and event-loop backpressure so app developers can react to it earlier and customize behavior, rather than only seeing it as warnings or errors in logs.
Heart of the request (from Dawson)
Mechanisms to consider
backpressurehook/function the app dev can call to make app-side decisions (stop accepting new work, shut down a background loop, etc.).server/throttle.ts— the throttle higher-order function) so app devs can use the same heuristics Harper itself uses, without re-implementing them incorrectly.Why this is Not Ready
The ask is broad enough that it needs decomposition before scheduling:
utilsglobal? Plugin-API additions onscope? Operations-API hooks?backpressurehook compose with Harper's internal throttling — does it pre-empt or coexist?Decomposition into 2-4 concrete sub-issues is the right next step; this umbrella stays open as the design context.
Related
throttlehigher-order function referenced is inserver/throttle.ts.Tracked in Jira: CORE-3021
Status: Not Ready — needs decomposition
🤖 Filed by Claude on behalf of Kris.