Skip to content

chore: add Infection mutation testing - #22

Merged
erikfrerejean merged 1 commit into
6from
chore/add-infection-mutation-testing
Jul 28, 2026
Merged

chore: add Infection mutation testing#22
erikfrerejean merged 1 commit into
6from
chore/add-infection-mutation-testing

Conversation

@erikfrerejean

Copy link
Copy Markdown
Member

Adds Infection mutation testing to the module, mirroring the silverstripe-grid setup: task mutate runs infection/infection over src/ against the PHPUnit suite, gated on minMsi in .docker/app/infection.json5. Local-only — deliberately not wired into CI, same as grid.

Measured result

99% covered MSI, 100% mutation code coverage, stable across three consecutive runs. minMsi / minCoveredMsi pinned at 98 — a whole number below the achieved score, so the single documented equivalent survivor cannot red-line the build.

143 mutations generated: 128 killed, 14 ignored, 1 escaped
Mutation Code Coverage: 100%   Covered Code MSI: 99%

PHPUnit config had to be modernised first

Infection rewrites phpunit.xml.dist and validates it against the PHPUnit 11 XSD, which rejects cacheResultFile and <coverage><include>. Both still ran fine under PHPUnit 11 but blocked the tool outright, so they move to cacheDirectory and <source><include> — the forms silverstripe-grid already uses.

Coverage gaps this surfaced

Calibration turned up real holes rather than just noise:

  • Permission delegation was never asserted. No test passed an explicit $member to canCreate / canView / canEdit / canDelete, so the very thing those overrides exist for went unverified on both models. Now covered in both directions: an explicit privileged member grants access to an unprivileged session, and an explicit unprivileged member is refused while an admin holds the session.
  • getLink() had no coverage for an unrecognised LinkType (a legacy or empty column value) falling through to null instead of an unhandled match error.
  • LinkingMode() had no coverage for a non-page item carrying a stale LinkedPageID — it must report link, not current, or an item switched from page to url lights up as active on the old page.

Suppression policy

Every suppression carries an inline proof, and the config documents the triage order: delete dead code → assert the behaviour → only then suppress at the narrowest scope.

The $member ?? 0 IncrementInteger mutants are suppressed with reasoning: the 0 is a type-narrowing device, not behaviour (Permission::checkMember() maps any falsy first argument to the current user, but its declared @param is int|Member, which PHPStan max requires). The mutated ?? 1 resolves member ID 1 instead, and killing it would mean asserting on whichever member owns that auto-increment ID — InnoDB does not roll the counter back with SapphireTest's per-test transaction, so it drifts with execution order. These were observed flipping killed/escaped between identical runs. Their Coalesce twins are deliberately left exposed and are killed by the new delegation tests.

One deliberate survivor is documented rather than suppressed: Menu::IsProtected()'s LogicalAnd, which is only distinguishable by putting '' into protected_menus.

Also

Test metadata moved from @dataProvider docblocks to #[DataProvider] attributes, clearing all PHPUnit 11 deprecation notices (the docblock form is removed in PHPUnit 12).

No production code changed.

Verification

  • task mutate → exit 0, 99% MSI with the gate active
  • task test → 61 tests, 86 assertions, no deprecations
  • task analyse → no errors

Mirrors the silverstripe-grid setup: `task mutate` runs infection/infection
over src/ against the PHPUnit suite, gated on minMsi in
.docker/app/infection.json5. Local-only, not wired into CI.

The PHPUnit config needed modernising first — Infection rewrites
phpunit.xml.dist and validates it against the PHPUnit 11 XSD, which rejects
`cacheResultFile` and `<coverage><include>`. Both still ran under PHPUnit 11
but blocked the tool outright.

Calibration surfaced genuine coverage gaps rather than just noise:

- No test ever passed an explicit $member to canCreate/canView/canEdit/
  canDelete, so the permission delegation those overrides exist for was
  unasserted on both models. Now covered in both directions.
- getLink() had no coverage for an unrecognised LinkType (legacy or empty
  column value) falling through to null instead of an unhandled match.
- LinkingMode() had no coverage for a non-page item carrying a stale
  LinkedPageID, which must report 'link' rather than 'current'.

Suppressions each carry an inline proof. The `$member ?? 0` IncrementInteger
mutants are suppressed because killing them would require asserting on
whichever member owns auto-increment ID 1, and InnoDB does not roll that
counter back with SapphireTest's per-test transaction — they were observed
flipping killed/escaped between identical runs. Their Coalesce twins are
left exposed and are killed by the new delegation tests.

Measured 99% MSI over three consecutive runs; minMsi pinned at 98 so the
one documented equivalent survivor cannot red-line the build.

Test metadata also moved from @dataProvider docblocks to #[DataProvider]
attributes, clearing all PHPUnit 11 deprecation notices.
@erikfrerejean
erikfrerejean merged commit a02094c into 6 Jul 28, 2026
4 checks passed
@erikfrerejean
erikfrerejean deleted the chore/add-infection-mutation-testing branch July 28, 2026 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant