Make test_download.py hermetic; remove dead wget path; harden zip/gz decompress (closes #42)

[iskandr]

Closes #42.

The flaky test

test_fetch_decompress hit a live Ensembl FTP server and looped:

for use_wget_if_available in [True, False]:
    for timeout in [None, 10**6]:
        path1 = fetch_file(URL, decompress=True, use_wget_if_available=..., timeout=timeout)
    assert ...

The file is only downloaded on the first iteration; every later call finds it
cached and skips the download. So whether a broken code path is exercised
depends on iteration order (as #42 notes, [True, False] vs [False, True]
flip the outcome), and the test needs the cached files manually deleted between
runs. Plus the live FTP dependency made CI noisy/slow.

Changes

• Hermetic, deterministic tests. Serve a local .gz/.zip over a file://
  URL and redirect the cache to a tmp dir (same approach as
  test_streaming_download.py). No network, no cross-run cached state. Adds
  explicit cache-reuse + force coverage in place of the order-dependent loop,
  plus a .zip case (only .gz was covered before).
• Remove the dead wget path. subprocess.call(...)'s non-zero exit was
  ignored, so a failed wget silently left a partial/empty file; and the 1.6.0
  modernization's streaming Python downloader (_stream_to_file, _download buffers entire response in memory; no streaming/progress hook #49) already
  handles both http(s) and ftp (urllib uses passive FTP by default, matching
  the old --passive-ftp). Drops the subprocess/errno imports and the
  use_wget_if_available threading through the private helpers.
• Non-breaking deprecation. use_wget_if_available stays on the public
  fetch_file / Cache.fetch as an ignored no-op that emits a
  DeprecationWarning when explicitly passed, so existing callers don't break.
• Hardened decompression (footgun fix surfaced while testing the zip path).
  Zip members are now streamed to the destination via z.open() +
  copyfileobj instead of ZipFile.extract(), which wrote into the current
  working directory and recreated the member's stored path — a cwd-pollution
  and ../ path-traversal risk. Gunzip is likewise streamed to disk rather than
  read fully into memory (consistent with _download buffers entire response in memory; no streaming/progress hook #49). Member selection (named match,
  else biggest) is unchanged.

Behavior change to note (no CHANGELOG in repo)

Cache.fetch's use_wget_if_available default flipped True → None; downloads
that previously preferred wget now always use the streaming Python downloader.
FTP parity is preserved (passive mode by default), and the removed path had the
silent-partial-download bug, so this is a net improvement — but it is a
behavior change for FTP-via-Cache worth a release note.

Test plan

• pytest tests/test_download.py tests/test_streaming_download.py — all green.
• Full suite: 24 passed. ./lint.sh clean. download.py coverage 69% → 80%.

Open question for the maintainer

I kept use_wget_if_available as a deprecated no-op (non-breaking). If you'd
rather a clean break, I can drop it from the public API entirely — there are no
in-repo callers; the only exposure is downstream openvax packages.

[coveralls]

Coverage Report for CI Build 27786313738

Coverage increased (+5.6%) to 79.043%

Details

• Coverage increased (+5.6%) from the base build.
• Patch coverage: 23 of 23 lines across 2 files are fully covered (100%).
• 1 coverage regression across 1 file.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

1 previously-covered line in 1 file lost coverage.

File	Lines Losing Coverage	Coverage
datacache/download.py	1	82.05%

---

Coverage Stats

Relevant Lines:	439
Covered Lines:	347
Line Coverage:	79.04%
Coverage Strength:	2.37 hits per line

---

💛 - Coveralls