Skip to content

CI has never been green: integration fixtures never generated on runners #86

Description

@duremovich

Every CI run since the workflow was added (2026-05-20) has failed. The DXC pin (5ca4a61) fixed the build leg; the test leg then revealed the underlying gap: ~18 integration tests per matrix leg fail because fixture media is never generated on the runner.

Findings (2026-07-03):

  • tests/fixtures/audio/tone_1khz.mov is generated locally by scripts/perf/setup-audio-fixtures.ps1 (ffmpeg) — CI has ffmpeg (cached install step) but never runs the script.
  • test_media/Basic loop.mov has no generator at all — it only exists on the dev machine. Tests referencing it (park_then_import_visible, others) cannot pass anywhere else.
  • Several failures look like slow-runner timing flakiness on top (e.g. section_loop_normal fails one AssertClipMediaFrame with import succeeding — 2-core WARP runner timing). The WaitSeconds margins may need re-sizing for CI.
  • Failing sets differ slightly between Tracy ON and OFF legs — consistent with timing flakiness, not deterministic breakage.

Suggested shape:

  1. CI step: run setup-audio-fixtures.ps1 (+ a new setup-test-media.ps1 that generates the test_media/ set — gradient_seq, blend_bg/fg, and an ffmpeg-generated replacement for Basic loop.mov, updating the scripts that reference it).
  2. Then chase the residual timing flakes on the runner (bigger WaitSeconds margins per the repo rule, or quarantine label).

Until this lands, the green gate for development remains local ctest -j8 (as it has been all along).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtier:coreLives in core (GPLv3, in-repo); always free, fully featured

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions