DicomReceiver: root-cause — late dcmrecv events after finalize (ASSOCIATION_FINALIZED may not be truly terminal)

[MichaelLeeHobbs]

Summary

PR #28 patched a symptom — a stray ASSOCIATION_RECEIVED that dcmrecv's stdout can deliver after a worker has finalized its association, which (pre-fix) re-registered a phantom association on the consumer side. The guard there is safe because ASSOCIATION_RECEIVED carries no payload.

This issue tracks the underlying root cause, which is still open and is the more important bug: why do dcmrecv lifecycle events (ASSOCIATION_RECEIVED, and potentially STORED_FILE/FILE_RECEIVED) ever arrive after the pool has begun finalizing an association?

Why it matters

ASSOCIATION_FINALIZED is contractually the terminal event for an association — every FILE_RECEIVED must precede it. That guarantee is the whole reason FINALIZED exists (we can't rely on Association Release being last, since file move/parse work may still be in flight; see #25).

If a FILE_RECEIVED can arrive after finalization, then FINALIZED fired too early — and that is a potential lost image:

• FILE_RECEIVED is deliberately not suppressed (correctly — it represents an object already written to disk; dropping it would lose data).
• But an un-suppressed late FILE_RECEIVED arriving after FINALIZED means the consumer already closed that association's books, so the file is mis-/under-counted or stranded.

So the same timing flaw that produces the (harmless) stray ASSOCIATION_RECEIVED could, in principle, produce a (harmful) late FILE_RECEIVED. We need to understand and close the timing flaw, not just filter its payload-free symptom.

Hypotheses to investigate

1. stdout/stderr cross-stream reordering. DcmtkProcess.handleData maintains separate buffers for stdout and stderr and emits line events from each; the parser is fed from both. Node provides no ordering guarantee between the two pipes. If dcmrecv writes some lifecycle lines to stdout and others to stderr, the parser can observe them out of dcmrecv's actual write order (e.g. Association Release parsed before a trailing Stored received object…). First thing to confirm: which stream(s) does dcmrecv emit Stored received object… vs Association Release/Association Received on?
2. Insufficient finalize deferral. wireAssociationComplete defers the drain by a single setImmediate (DicomReceiver: FILE_RECEIVED fires after worker reset to idle — ASSOCIATION_FINALIZED ordering broken #25). Under heavy load a still-buffered stdout chunk may be parsed after finalizeAssociation has run (_finalized = true), i.e. after the one-tick window.
3. Worker reuse / shared child stream. The long-lived dcmrecv child's continuous stdout straddles association boundaries; the pool's per-association state machine is decoupled from it.

(Note: Node's EventEmitter is synchronous and ordered, so this is not intra-emitter reordering — it's cross-pipeline/cross-stream timing.)

Proposed investigation

Instrument every late event (one that arrives while worker.finalized is set, in both wireAssociationReceived and wireFileReceived) and record:

• the file path (for FILE_RECEIVED) and whether it's already in worker.files (duplicate vs new path),
• the source stream (stdout vs stderr) of the originating line,
• timing relative to ASSOCIATION_COMPLETE / ASSOCIATION_FINALIZED.

Run under the extreme-concurrency soak (50-parallel single-file, the harness that surfaced the ~1/1500).

Key question: is any late FILE_RECEIVED ever a new path? If yes → confirmed lost-image risk → fix finalize timing/ordering so FINALIZED is genuinely terminal.

Candidate fixes (depending on findings)

• If cross-stream reordering: merge stdout+stderr into a single ordered line stream before parsing, or anchor association boundaries to a single stream.
• If deferral insufficiency: gate FINALIZED on a stronger drain condition rather than a fixed one-tick setImmediate.

Acceptance criteria

• Root mechanism identified and documented.
• Demonstrated that no FILE_RECEIVED can be observed after ASSOCIATION_FINALIZED for an association (the terminal-event contract holds), or that any such event is provably a harmless duplicate.
• No silent image loss under the soak.

References

• PR fix(DicomReceiver): suppress late dcmrecv events after finalize #28 (symptom fix: suppress late ASSOCIATION_RECEIVED)
• DicomReceiver: FILE_RECEIVED fires after worker reset to idle — ASSOCIATION_FINALIZED ordering broken #25 (introduced the setImmediate finalize deferral + FINALIZED terminal event)
• Code: src/servers/DicomReceiver.ts (finalizeAssociation, wireAssociationComplete, wireFileReceived, wireAssociationReceived), src/DcmtkProcess.ts (handleData/processLines — separate stdout/stderr buffers)