Skip to content

fix: record the positions a terminated run leaves open - #181

Merged
mohammadp1001 merged 1 commit into
feat/report-run-costfrom
fix/record-positions-left-open-on-shutdown
Aug 13, 2026
Merged

fix: record the positions a terminated run leaves open#181
mohammadp1001 merged 1 commit into
feat/report-run-costfrom
fix/record-positions-left-open-on-shutdown

Conversation

@mohammadp1001

Copy link
Copy Markdown
Owner

Stacked on #180. Branched off feat/report-run-cost so the two changes to
run()'s finally block do not conflict. Merge #180 first; the diff shown here
against main includes #180's commit.

The problem

run-4 was killed at 17:21 UTC on 2026-08-13 holding 16 shares of XLE whose
exit window had already expired at 17:22:38
. Nothing closed it, nothing warned,
and nothing recorded it. The fact existed only at the broker until someone thought
to go and look.

run-3 died exactly the same way and only escaped because it happened to be flat
at the time. Two for two.

The cause is simple: there was no signal handling at all. SIGTERM killed the
process outright, so run()'s finally block - which already contains
_drain_open_positions - never executed.

The fix

Termination now raises RunTerminatedError, which lets the shutdown path run at
all. What that path does was a deliberate choice between two options:

  • It does not trade on the way out. Closing positions on a signal would
    realise them at whatever price the reaper happened to pick.
  • It does not drain either. Draining waits candle by candle, and a process
    being killed does not have that time.

So it exits fast and says exactly what it is leaving behind - on stderr and as
POSITIONS_ABANDONED telemetry:

WARNING: 1 position(s) left OPEN and UNMONITORED:
  XLE 16.0 @ 60.93  stop 59.71  window closes 2026-08-13 17:22:00+00:00
These have no stop-loss until a later run reconciles them. Close them at the
broker, or start the next run with --reconcile.

Ticker, size, entry, stop, and window deadline - enough to act on by hand, which
is precisely what was missing today.

Previous handlers are restored on the way out, so embedding or importing the
scheduler does not permanently redirect the host process's signals.

What this does not fix

A hard kill. SIGKILL and Windows TerminateProcess are not deliverable to
any handler, here or anywhere else, so if that is what reaped the last two runs
this changes nothing for them. Startup reconciliation (#177) remains the only net
for that case.

What it does cover is every orderly stop: Ctrl-C, a supervisor's SIGTERM, a
container shutdown. The root cause of the reaping itself is still unknown and is
recorded in the handoff as an open question.

Verification

  • ruff check alphoryn/ tests/ clean
  • 745 tests, 100% coverage
  • The "signal missing on this platform" branch is tested by patching the signal
    name list rather than relying on SIGBREAK's absence, so coverage holds on both
    Windows and Linux
  • Reproduced the CI environment: GOOGLE_APPLICATION_CREDENTIALS=/nonexistent/adc.json python -m pytest -q

The 2026-08-13 run was killed at 17:21 UTC holding 16 shares of XLE whose exit
window had already expired. Nothing closed it, nothing warned, and nothing
recorded it - the fact existed only at the broker until someone thought to look.
The run before it died the same way and only escaped because it happened to be
flat.

There was no signal handling at all, so SIGTERM killed the process outright and
run()'s finally block never executed. Termination now raises RunTerminatedError,
which lets the shutdown path run.

A terminated run does not trade on its way out - closing positions on a signal
would realise them at whatever price the reaper happened to pick. It also does
not drain, because draining waits candle by candle and a process being killed
does not have that time. Instead it exits fast and says exactly what it is
leaving behind, on stderr and as POSITIONS_ABANDONED telemetry: ticker, size,
entry, stop, and window deadline - enough to act on by hand.

Previous handlers are restored on the way out so embedding the scheduler does
not permanently redirect the host process's signals.

This cannot help against a hard kill. SIGKILL and TerminateProcess are not
deliverable to any handler, and startup reconciliation (#177) remains the only
net for that case. What it covers is every orderly stop: Ctrl-C, a supervisor's
SIGTERM, a container shutdown.
@mohammadp1001
mohammadp1001 force-pushed the fix/record-positions-left-open-on-shutdown branch from a19b3d8 to eb79c86 Compare August 13, 2026 22:20
@mohammadp1001
mohammadp1001 merged commit f4bb458 into feat/report-run-cost Aug 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant