Fix skipped tests in Image Placeholders and Modern Image Formats by b1ink0 · Pull Request #2523 · WordPress/performance

b1ink0 · 2026-06-06T18:38:32Z

Summary

Relevant technical choices

This PR updates the test command to use the wordpress Docker image instead of the cli Docker image.

So I did some digging and found that the cli image will not support ImageMagick at all due to Alpine's limitations and the way the image is configured. There is a comment in the Docker image itself that explains why it is not expected to work:

https://github.com/docker-library/wordpress/blob/862322ddbba563563ae63e1dcee31d477626eb8e/cli/php8.2/alpine/Dockerfile#L50-L55
# WARNING: imagick is likely not supported on Alpine: https://github.com/Imagick/imagick/issues/328
# https://pecl.php.net/package/imagick
	pecl install imagick-3.8.1; \
	docker-php-ext-enable imagick; \
	rm -r /tmp/pear; \
	\
Given this, I think we should just switch to the wordpress image for image-related plugins and test suites.

Originally posted by @b1ink0 in #2521

Use of AI Tools

Used AI tools to help identify the root cause of the issue and verify the fix.

github-actions · 2026-06-06T18:44:23Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: b1ink0 <b1ink0@git.wordpress.org>
Co-authored-by: westonruter <westonruter@git.wordpress.org>
Co-authored-by: nickchomey <nickchomey@git.wordpress.org>
Co-authored-by: thelovekesh <thelovekesh@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

b1ink0 · 2026-06-06T22:24:58Z

+	 *
+	 * @return bool True if AVIF encoding is supported, false otherwise.
+	 */
+	public function check_avif_encoding_support(): bool {


This is still required due to following:

The tests are now passing for PHP 8.1+ and PHP 7.2+ versions due to the upstream fix, but PHP 8.1 and PHP 7.2 will not be fixed because support for this Docker image has been dropped (see docker-library/wordpress#992.

We have the option to skip these failing tests for this specific version, or we can remove PHP 8.1 from testing.

For now, I will skip these failing tests on these version.

#2245 (comment)

b1ink0 · 2026-06-06T22:29:08Z

Note sure why Codecov is failing to upload

I think we need to update the pined Codecov action version.

https://github.com/WordPress/performance/actions/runs/27075344831/job/79911662004?pr=2523

nickchomey · 2026-06-07T00:28:53Z

FWIW, I tried setting all of them to wordpress and it didnt make a difference. So, seems that you caught the two where this currently matters

westonruter · 2026-06-11T14:03:21Z

 		"test-php:performance-lab": "wp-env --config=.wp-env.test.json run cli --env-cwd=/var/www/html/wp-content/plugins/performance composer test:performance-lab",
 		"test-php:auto-sizes": "wp-env --config=.wp-env.test.json run cli --env-cwd=/var/www/html/wp-content/plugins/performance composer test:auto-sizes",
-		"test-php:dominant-color-images": "wp-env --config=.wp-env.test.json run cli --env-cwd=/var/www/html/wp-content/plugins/performance composer test:dominant-color-images",
+		"test-php:dominant-color-images": "wp-env --config=.wp-env.test.json run wordpress --env-cwd=/var/www/html/wp-content/plugins/performance composer test:dominant-color-images",


Any reason to not use the wordpress instead of cli for all of the plugins?

I had this question/thought as well, and even confirmed that the tests all pass after changing them all to wordpress.

I assume the thought was that WordPress is perhaps heavier?

@westonruter, after some investigation, I found that the tests-cli was introduced in PR #544 before it was phpunit. However, I could not find any discussion explaining the decision to use tests-cli instead of tests-wordpress.

Also, @thelovekesh commented on the issue:

wp-env generates the Dockerfile, builds it, and runs it, so it's wp-env specific. Also, tests should run in the tests WordPress image and not the CLI image.

Reference: #2521 (comment)

I think we should move everything to the wordpress image now, as that would simplify a lot of the setup and maintenance.

updated in b509cb1.

westonruter · 2026-06-11T14:04:22Z

 		"test-e2e:auto-sizes": "wp-scripts test-playwright --config tools/e2e/playwright.config.ts --project=auto-sizes",
 		"lint-php": "composer lint:all",
-		"test-php": "wp-env --config=.wp-env.test.json run cli --env-cwd=/var/www/html/wp-content/plugins/performance composer test:plugins",
+		"test-php": "npm-run-all -s --continue-on-error test-php:performance-lab test-php:auto-sizes test-php:dominant-color-images test-php:embed-optimizer test-php:image-prioritizer test-php:optimization-detective test-php:speculation-rules test-php:view-transitions test-php:web-worker-offloading test-php:webp-uploads",


Can you elaborate on this change?

Ah, right, it's because we have the same test command in both package.json and composer.json, but only here do we control where it is run.

Alternatively, this could be just the following, if we move all execution from cli:

Suggested change

"test-php": "npm-run-all -s --continue-on-error test-php:performance-lab test-php:auto-sizes test-php:dominant-color-images test-php:embed-optimizer test-php:image-prioritizer test-php:optimization-detective test-php:speculation-rules test-php:view-transitions test-php:web-worker-offloading test-php:webp-uploads",

"test-php": "wp-env --config=.wp-env.test.json run wordpress --env-cwd=/var/www/html/wp-content/plugins/performance composer test:plugins",

Done in b509cb1.

nickchomey · 2026-06-11T15:34:51Z

As per weston's review, I think using wordpress for all plugins would be cleaner/easier for people to follow than a mix of cli and wordpress. Unless there's a strong reason to minimize use of wordpress?

Co-authored-by: Weston Ruter <weston@ruter.net>

The v6.0.1 action imported the GPG key from a Keybase account that Codecov deleted during a migration, causing "No public key" signature verification failures. v6.0.2 (a copy of v7.0.0) switches to the codecovsecops account with the original key. See codecov/codecov-action#1955. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nickchomey · 2026-06-11T17:51:29Z

Should any of the other mentions/usages of cli be changed to wordpress?

https://github.com/b1ink0/performance/blob/2ea8db54ac343ae581ce5e570bf303c39e7567a3/bin/test-php-watch.sh#L32
https://github.com/b1ink0/performance/blob/2ea8db54ac343ae581ce5e570bf303c39e7567a3/.github/workflows/plugin-check.yml#L210-L228
https://github.com/b1ink0/performance/blob/2ea8db54ac343ae581ce5e570bf303c39e7567a3/.github/workflows/php-test-plugins.yml#L93

b1ink0 · 2026-06-11T17:54:31Z

Should any of the other mentions/usages of cli be changed to wordpress?

https://github.com/b1ink0/performance/blob/2ea8db54ac343ae581ce5e570bf303c39e7567a3/bin/test-php-watch.sh#L32 https://github.com/b1ink0/performance/blob/2ea8db54ac343ae581ce5e570bf303c39e7567a3/.github/workflows/plugin-check.yml#L210-L228 https://github.com/b1ink0/performance/blob/2ea8db54ac343ae581ce5e570bf303c39e7567a3/.github/workflows/php-test-plugins.yml#L93

Ohh missed that yeah we need to update those as well thanks.

working on it

codecov · 2026-06-11T17:56:12Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.14%. Comparing base (b45a9d5) to head (43a1f05).
⚠️ Report is 10 commits behind head on trunk.

Additional details and impacted files

@@            Coverage Diff             @@
##            trunk    #2523      +/-   ##
==========================================
- Coverage   69.29%   69.14%   -0.16%     
==========================================
  Files          90       90              
  Lines        7723     7723              
==========================================
- Hits         5352     5340      -12     
- Misses       2371     2383      +12

Flag	Coverage Δ
multisite	`69.14% <ø> (-0.16%)`	⬇️
single	`35.50% <ø> (-0.26%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Co-authored-by: nickchomey <nickchomey@users.noreply.github.com>

westonruter · 2026-06-11T18:05:29Z

@b1ink0 I was looking into this as well and got Claude's opinion:

.github/workflows/plugin-check.yml only runs wp plugin check / wp plugin install / wp plugin list — no unit tests, no image processing, so imagick is irrelevant. The cli service is actually the conventional wp-env container for running wp-cli commands. The only argument for changing it is the team's broader "move everything to the wordpress image for consistency/maintenance" goal — and notably, wp-env start provisions both images regardless, so there's no perf cost either way. It's purely a consistency choice, not a correctness fix.

So seems fine.

Oh, but Plugin Check is now failing. It seems this needs to be reverted.

Okay I will remove the change for plugin-check.yml as it actually requires cli for wp cli commands.

Done in 43a1f05.

Use wordpress docker container for image related plugins

c284697

Fix previously skipped tests

68a36a3

nickchomey mentioned this pull request Jun 6, 2026

Bug: Many tests being skipped for dominant-color-images plugin #2521

Closed

Skip AVIF encoding-related tests on unsupported environments

a0d719d

b1ink0 requested a review from adamsilverstein as a code owner June 6, 2026 22:14

b1ink0 commented Jun 6, 2026

View reviewed changes

b1ink0 requested a review from westonruter June 6, 2026 22:32

westonruter added the [Type] Enhancement A suggestion for improvement of an existing feature label Jun 7, 2026

westonruter requested a review from thelovekesh June 7, 2026 05:25

nickchomey mentioned this pull request Jun 8, 2026

Fix/tidy and gd tests #2527

Open

westonruter reviewed Jun 11, 2026

View reviewed changes

Replace all use of cli image with wordpress image

b509cb1

Co-authored-by: Weston Ruter <weston@ruter.net>

b1ink0 requested a review from mukeshpanchal27 as a code owner June 11, 2026 17:12

Merge branch 'trunk' into fix/skipped-tests

14171aa

b1ink0 requested a review from westonruter June 11, 2026 17:26

westonruter approved these changes Jun 11, 2026

View reviewed changes

Update missed replacements of cli with wordpress

1f22821

Co-authored-by: nickchomey <nickchomey@users.noreply.github.com>

westonruter reviewed Jun 11, 2026

View reviewed changes

Revert changes in plugin check workflow file

43a1f05

westonruter approved these changes Jun 11, 2026

View reviewed changes

westonruter merged commit b1608b2 into WordPress:trunk Jun 11, 2026
31 of 32 checks passed

	"test-php": "npm-run-all -s --continue-on-error test-php:performance-lab test-php:auto-sizes test-php:dominant-color-images test-php:embed-optimizer test-php:image-prioritizer test-php:optimization-detective test-php:speculation-rules test-php:view-transitions test-php:web-worker-offloading test-php:webp-uploads",
	"test-php": "wp-env --config=.wp-env.test.json run wordpress --env-cwd=/var/www/html/wp-content/plugins/performance composer test:plugins",

Conversation

b1ink0 commented Jun 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Relevant technical choices

Use of AI Tools

Uh oh!

github-actions Bot commented Jun 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

b1ink0 commented Jun 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nickchomey commented Jun 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

b1ink0 Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nickchomey commented Jun 11, 2026

Uh oh!

nickchomey commented Jun 11, 2026

Uh oh!

b1ink0 commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

b1ink0 commented Jun 6, 2026 •

edited

Loading

github-actions Bot commented Jun 6, 2026 •

edited

Loading

b1ink0 commented Jun 6, 2026 •

edited

Loading

b1ink0 Jun 11, 2026 •

edited

Loading

b1ink0 commented Jun 11, 2026 •

edited

Loading

codecov Bot commented Jun 11, 2026 •

edited

Loading