Skip to content

Feature/probinso/fast completion detection - #185

Open
robinson96 wants to merge 21 commits into
mainfrom
feature/probinso/fast_completion_detection
Open

Feature/probinso/fast completion detection#185
robinson96 wants to merge 21 commits into
mainfrom
feature/probinso/fast_completion_detection

Conversation

@robinson96

Copy link
Copy Markdown
Collaborator

SUMMARY

Refactor completion handling behind a CompletionDetector interface so
machine launch/cleanup code can select between detector strategies instead
of hard-coding one polling path.

Adds 2 queue-based completion detection modes alongside the legacy polling behavior. completion_queue (or reap) uses a reaper thread to notice when subprocesses end and mark the corresponding test for later processing. (Lower overhead, but not technically safe, but reasonably safe in practice). completion_queue_simple launches 1 thread per test to notice when that test is finished, marking it for later processing. Higher thread count and memory overhead, but safer threading model. Does not scale past a few hundred concurrent tests.

Add detector configuration plumbing, completion stats/spans hooks, updated
scheduler-extension documentation, and focused detector tests.

Record and summarize warnings when the queue-mode reaper observes child
exits that do not belong to registered ATS tests, so the known
waitpid(-1) race is visible in final reporting.

Restore Flux processor-capacity reporting through the normal ATS summary
path and keep unsupported queue-mode paths explicitly rejected where needed.

@robinson96
robinson96 requested review from ryanq4 and stephdempsey July 9, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant