Skip to content

test-coverage always fails: a truncated covr trace file aborts merge_coverage and skips the codecov upload #1207

Description

@mattfidler

The test-coverage workflow has been failing on main and on essentially every open PR branch. It is not any one branch's fault, and the test suite itself passes -- the run ends [ FAIL 0 | WARN 31 | SKIP 58 | PASS 76505 ] and then dies during covr's merge:

Error in readRDS(f) : error reading from connection
Calls: <Anonymous> ... merge_coverage.character -> lapply -> FUN -> as.list
Execution halted

Because the step exits non-zero, codecov/codecov-action is skipped, so no report is uploaded at all. Each failed run burns 2--5 hours first.

Cause

covr:::add_hooks() appends this to the installed package's load script:

reg.finalizer(ns, function(...) { covr:::save_trace(Sys.getenv("COVERAGE_DIR", "<lib>")) }, onexit = TRUE)

So every R process that loads the instrumented rxode2 writes a covr_trace_* file into the merge directory when it exits -- including the helper R subprocesses the test suite spawns (test-omp-num-threads.R, test-pkg-exported-funs.R, ...). package_coverage() then does:

trace_files <- list.files(path = install_path, pattern = "^covr_trace_[^/]+$", full.names = TRUE)
coverage <- merge_coverage(trace_files)

and merge_coverage.character() is just lapply(x, function(f) as.list(readRDS(f))) -- no error handling. One partial file takes down the entire run.

Evidence

Downloading the coverage-test-failures artifact from a PR run (31139091448) and a main run (31040944854) gives the same picture both times: 25 trace files, exactly 8 unreadable. The bad ones have a valid gzip header and real compressed data that simply stops (gzip: unexpected end of file), at a size that is an exact multiple of 4096 -- a write that was cut off, not a corrupt file.

Fingerprinting the readable traces shows the truncated ones cost almost nothing:

trace covered expressions
1 file (the testthat run, 26 MB) 15972 -- intact
6 files 0
4 files 41 (.onAttach only)
6 files ~525 (rxForget)
8 files truncated

All the real coverage is in the one intact trace. The 24 others are near-empty helper processes.

Fix

Confirmed against the real artifact files: unpatched, merge_coverage() errors; with unreadable traces skipped it returns a normal coverage object with 16027 of 20218 expressions covered.

PR #1208 makes the workflow drop unreadable traces and report which ones, so a poisoned helper trace costs a little coverage instead of the whole report.

That is deliberately a tolerance fix, not a root-cause fix -- it does not explain why those 8 subprocesses die mid-write. Worth a separate look, but the job should not be red in the meantime.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions