Bound watch_for_import so stuck transfers can't stall downloads (fixes #30)#31
Bound watch_for_import so stuck transfers can't stall downloads (fixes #30)#31bugrax wants to merge 4 commits into
Conversation
…wouterdebie#30) watch_for_import polled is_imported() in an unbounded loop that only exits once every video file is imported. A transfer containing a file the *arr never imports (typically a sample outside a skip_directories folder) makes is_imported() permanently false, so the task loops forever. These accumulate over time until the process stops picking up new transfers entirely (downloads stall; a restart clears it). Give up watching after MAX_IMPORT_WAIT (2h) with a warning, so a stuck transfer ends its task instead of polling the *arr/put.io APIs forever. Genuine imports complete within a poll or two, so the bound doesn't affect normal operation. A better long-term fix is improved import mapping (the skip_directories TODO), noted in the issue.
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a bounded wait to prevent watch_for_import from polling indefinitely when is_imported() can never become true (e.g., transfers containing files the *arr won’t import).
Changes:
- Introduce
MAX_IMPORT_WAIT(2h) as an upper bound for import polling. - Track start time with
Instantand stop watching after the timeout with a warning.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The unconditional 'removed' log fired on the give-up path too, even though the warning there says nothing was cleaned up. Move it into the imported branch so the log matches what happened.
The give-up timeout was a hard-coded 2h. The right value depends on library size, IO speed and *arr load, so expose it as import_timeout_secs (default 7200). Setting it to 0 disables the bound (watch indefinitely).
Use #[serde(default = ...)] returning 7200 instead of #[serde(default)], which would default to 0 (disabling the bound) if the field is missing from a config source without the Figment default layer.
|
These were addressed in follow-up commits already on the branch:
On giving up leaving local/remote copies behind: that's an intentional trade-off — leaking one stuck transfer is far better than the unbounded loop stalling every download — and it's re-evaluated on the next restart. The accompanying log makes it greppable. Open to surfacing it as an explicit failure event if you'd like. |
Fixes #30.
Problem
watch_for_importpollsis_imported()in an unbounded loop and only exits once every video file of a transfer is imported:If a transfer contains a video file the *arr will never import — most commonly a sample that isn't inside a
skip_directoriesfolder —is_imported()is permanently false and the task loops forever. Each such transfer leaves a forever-running task polling the *arr/put.io APIs everypolling_interval. They accumulate, and after ~10h with a large library the process stops picking up newly-completed transfers entirely (downloads stall with manyCOMPLETEDtransfers left un-pulled on put.io). A restart only clears it temporarily.The tell-tale sign in the logs is the main file of the same transfer being reported
found imported by radarrover and over every ~10s — the movie imported fine, but a sibling sample keeps the transfer from ever completing.Fix
Give up watching after
MAX_IMPORT_WAIT(2h) with a warning, so a stuck transfer ends its task instead of polling forever. Genuine imports are detected within a poll or two of the *arr importing, so this generous bound doesn't affect normal operation; it just caps how long a never-importing transfer can sit in a polling loop.This is a targeted fix for the stall. The deeper fix is better import-mapping (the existing
skip_directoriesTODO — e.g. ignoring sample files, or treating a transfer as imported once it has left the *arr download queue), which would let these transfers complete and be cleaned up rather than be abandoned.Note
A given-up transfer's local and put.io copies are no longer auto-cleaned (it stops being tracked). That's a deliberate trade-off — leaking one stuck transfer is far better than stalling every download — and it is re-evaluated on the next restart.