CI Health Check: Tests Failing
Summary
The CI is currently broken on the main branch. Automated health check PR #14 revealed multiple failures across different Julia versions.
Failed Jobs
- Julia 1.0 on ubuntu-latest: Threading configuration error
- Julia 1.6 on ubuntu-latest: Visual regression test failures (3 errored tests)
Detailed Analysis
Julia 1.0 Failure
Root Cause: Invalid threading configuration in CI workflow
Error Message:
ERROR: julia: -t,--threads=<n>[,auto|<m>]; n must be an integer >= 1
Package FiniteVolumeMethod1D errored during testing
Issue: The CI workflow file (.github/workflows/CI.yml) sets JULIA_NUM_THREADS: ${{ matrix.julia-threads }} on line 18, but matrix.julia-threads is never defined in the matrix configuration (lines 22-28). This passes an empty value to Julia's -t flag, causing the error.
Suggested Fix: Either:
- Remove the
JULIA_NUM_THREADS environment variable if threading is not needed
- Add
julia-threads to the matrix configuration with appropriate values
Julia 1.6 Failure
Root Cause: Visual regression test failures due to reference image mismatches
Test Results: 67 passed, 0 failed, 3 errored, 0 broken
Failed Tests:
Porous-Medium - Reference image mismatch at docs/src/figures/porous_surface.png
Robin Diffusion - Reference image mismatch at docs/src/figures/robin_diffusion_surface.png
Reaction-Diffusion - Reference image mismatch at docs/src/figures/dirichlet_source_surface.png
Error Message:
LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")',
or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE`
to "true" and re-run the tests via Pkg.
Possible Causes:
- Upstream dependency changes affecting plot rendering
- Julia version differences in plotting libraries
- Platform-specific rendering differences
Suggested Fix Approach:
- Run tests locally on Julia 1.6 to reproduce the issue
- Review the actual vs reference images to determine if differences are acceptable
- If differences are minor/acceptable: Update reference images by setting
JULIA_REFERENCETESTS_UPDATE=true
- If differences indicate a real regression: Investigate changes in dependencies or plotting code
Links
Priority
High - CI is broken on main branch, blocking development and future PRs.
cc @ChrisRackauckas
CI Health Check: Tests Failing
Summary
The CI is currently broken on the main branch. Automated health check PR #14 revealed multiple failures across different Julia versions.
Failed Jobs
Detailed Analysis
Julia 1.0 Failure
Root Cause: Invalid threading configuration in CI workflow
Error Message:
Issue: The CI workflow file (
.github/workflows/CI.yml) setsJULIA_NUM_THREADS: ${{ matrix.julia-threads }}on line 18, butmatrix.julia-threadsis never defined in the matrix configuration (lines 22-28). This passes an empty value to Julia's-tflag, causing the error.Suggested Fix: Either:
JULIA_NUM_THREADSenvironment variable if threading is not neededjulia-threadsto the matrix configuration with appropriate valuesJulia 1.6 Failure
Root Cause: Visual regression test failures due to reference image mismatches
Test Results: 67 passed, 0 failed, 3 errored, 0 broken
Failed Tests:
Porous-Medium- Reference image mismatch atdocs/src/figures/porous_surface.pngRobin Diffusion- Reference image mismatch atdocs/src/figures/robin_diffusion_surface.pngReaction-Diffusion- Reference image mismatch atdocs/src/figures/dirichlet_source_surface.pngError Message:
Possible Causes:
Suggested Fix Approach:
JULIA_REFERENCETESTS_UPDATE=trueLinks
Priority
High - CI is broken on main branch, blocking development and future PRs.
cc @ChrisRackauckas