Skip to content

Harden artifact loading for downloads and cache validation#10

Closed
mbsantiago wants to merge 1 commit into
mainfrom
fix/harden-artifact-loading
Closed

Harden artifact loading for downloads and cache validation#10
mbsantiago wants to merge 1 commit into
mainfrom
fix/harden-artifact-loading

Conversation

@mbsantiago

Copy link
Copy Markdown
Owner

Summary

  • add retry, timeout, and exponential backoff support to load_artifact downloads to make network fetches more resilient
  • add optional artifact integrity validation (expected_sha256, expected_size) for both newly downloaded files and cached files
  • tighten URL detection to HTTP(S) URLs only and clean up partial or invalid files from cache automatically

Testing

  • ruff check src/audioclass/utils.py tests/test_utils.py --select E,F,B,I
  • pytest tests/test_utils.py (fails in this environment: missing optional dependency soundfile imported by tests/conftest.py)

Copilot AI review requested due to automatic review settings April 23, 2026 22:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens audioclass.utils.load_artifact by improving URL detection, adding resilient HTTP download behavior (timeout + retry/backoff + streaming), and introducing optional integrity checks for downloaded and cached artifacts.

Changes:

  • Restrict URL detection to valid HTTP(S) URLs and add a regression test for Windows-like paths.
  • Add streaming downloads with configurable timeout, retries, and exponential backoff.
  • Add optional cached/downloaded artifact validation via expected SHA256 and/or expected file size, and clean up invalid/partial cache files.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/audioclass/utils.py Adds HTTP(S)-only URL detection, download retry/backoff + streaming, and artifact integrity validation with cache cleanup.
tests/test_utils.py Adds coverage for URL detection, timeout/streaming usage, retry behavior, validation failures, and partial-file cleanup.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/audioclass/utils.py

if attempt == retries:
raise

Copilot AI Apr 23, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_download_with_retry writes into the same fileobj across retry attempts, but on a requests.RequestException it does not rewind/truncate the file before retrying. If a network error happens mid-stream (e.g., ChunkedEncodingError from iter_content), the next attempt will append to the partial content, producing a corrupted artifact even if the later attempt succeeds. Reset/truncate the file between attempts (or download into a temp file per attempt and only move into place after a successful full download).

Suggested change
fileobj.seek(0)
fileobj.truncate()

Copilot uses AI. Check for mistakes.
Comment thread src/audioclass/utils.py
Comment on lines 164 to 166
basename = Path(urlparse(path).path).name
new_path = directory / basename

Copilot AI Apr 23, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

basename = Path(urlparse(path).path).name can be empty for valid HTTP(S) URLs that end with / (or otherwise have no filename). In that case new_path becomes the cache directory itself and new_path.open('wb') will fail with IsADirectoryError. Consider explicitly handling the empty-basename case (e.g., derive a filename from the URL hash or raise a clear ValueError).

Copilot uses AI. Check for mistakes.
@mbsantiago mbsantiago closed this Apr 23, 2026
@codecov

codecov Bot commented Apr 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.03226% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.60%. Comparing base (39abc8e) to head (60adefa).

Files with missing lines Patch % Lines
src/audioclass/utils.py 79.03% 4 Missing and 9 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #10      +/-   ##
==========================================
- Coverage   90.81%   89.60%   -1.22%     
==========================================
  Files          17       17              
  Lines         599      654      +55     
  Branches       61       74      +13     
==========================================
+ Hits          544      586      +42     
- Misses         29       33       +4     
- Partials       26       35       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants