Client Side Media iteration for WordPress 7.0

[adamsilverstein]

Overall Goal

Graduate client side media from experiments to a core feature. Client side media will provide a progressive enhancement to the current server based approach to media processing and provide more robust and consistent media processing across WordPress installs, regardless of their underlying hosting stack. If the user's browser is not capable of processing media, the editor will silently fall back to existing Server-Side processing without error (with only a developer facing console message).

Testing the feature

A testing pull request is now available to test the beta Client Side Media feature. Developers can help test this feature by following testing instructions on the PR.

Goal for WordPress 7.0

Graduate client side media to a core feature, potentially reducing server image processing CPU usage by >80% (as long as the user’s device is capable of performing the process) and providing modern media processing capabilities for all WordPresses.

• With client side media, users will upload media in the WordPress block editor just as they do today.
• When an image is uploaded, the creation of sub-sized images will happen inside the browser (client side) instead of today’s server based approach.
• The original image and each processed image are sent to WordPress for storage with the end result being the same set of sub-sized images and image meta data stored as if the current server side approach was used.

By shifting processing to the client side, numerous advanced capabilities are unlocked. Client side processing can use the latest technologies and codecs available (and WordPress can update media libraries with each new version of WordPress) - something we can’t expect and don’t see on the server side.

The existing Media Experiments plugin from @swissspidy contains a very solid foundation that already solves many of the complex technical challenges involved in adding this feature.

Background & Previous Work

Details

• The current Client side media experiment was merged in Add experiment for client-side media processing #64650 - it includes the initial infrastructure to add the new feature.
• @swissspidy has developed this feature as an experiment in Gutenberg and continued work to refine the features in the Media Experiments plugin: https://github.com/swissspidy/media-experiments
• In May 2024 Pascal posted a tracking ticket including a roadmap: Tracking: Client-side media processing #61447
• In Aug, 2024 Pascal opened a PR to bring full client side media to Gutenberg: [WIP] Add client-side media processing #64278, Although the PR was never merged, it includes a detailed discussion with lead contributors on the preferred architectural approach for Gutenberg.
• In Aug 2025 @adamsilverstein started work to bring over a minimal set of features: Add/client side media uploads #70623

This ticket is an effort to revive these previous work streams to ultimately graduate media experiments to a core feature.

In scope for WordPress 7.0

Details

The plan for 7.0 is to open a series of sub-issues to bring the required pieces for client side media handling into Gutenberg. The issues and the PRs associated with them will remain small enough to be reviewed and merged. These issues when completed will provide the same set of features provided in the very best available server processing environment:

Core Client-Side Processing Features:

• Client-side image compression for all new uploads using WebAssembly (WASM) and modern libraries
• Honors all core settings for image sizes, quality, max-size
• Support for MozJPEG encoding, letting JPEGs approach WebP compression levels while remaining fully compatible with existing decoders
• Client-side generation of all image thumbnails/sub-sizes during upload
• Modern image format support (AVIF, WebP) with support for existing core output mapping filter
• Handling for existing core auto-rotation and large-image-size logic

Architecture Changes:

• New @wordpress/upload-media package to handle client-side processing
• Upload queue and compression logic built into the block-editor package
• Integration of wasm-vips for high-performance image processing
• Cross-origin isolation support for SharedArrayBuffer (required for WASM)

User Experience:

• Graceful fallback to existing server upload mechanism for browsers without complete support
• Asynchronous uploads as thumbnails are generated in parallel on device
• Fewer issues trying to upload exotic image formats - eg. JXL or UHDR etc.
• Progress bar/spinner for client-side processing

7.X

Once the core capabilities have landed, a series of enhancements is unlocked for releases beyond 7.0.

**Extended Media Type Support:**

• Animated GIF to video conversion (MP4/WebM)
• Video transcoding and compression (e.g., MOV to MP4)
• Audio transcoding (e.g., OGG to MP3)
• Automatic creation of poster images for uploaded videos
• HEIC/HEIF image format conversion (iPhone images) to JPEG - libheif's LGPL v3 license remains a blocker ... see Leverage server-side HEIC conversion swissspidy/media-experiments#578
• Start with a server side fallback when available
• Continue evaluating decoding options
• PDF poster image generation

Advanced Features:

• Enable optimization of all existing images in a post
• Import external images with automatic compression
• Image editing capabilities (crop, rotate, scale)
• BlurHash generation and dominant color extraction for placeholder images
• Upload Media UI - to show all in-progress items and allow cancellation of individual uploads.
• Webcam support - record a video for your site
• Process persistence - complete uploads after a disconnect or reload

Media Library Integration:

• Extend client-side processing beyond block editor to media library as part of a media library rewrite
• UI for client side regenerating/recompressing existing media library images
• Bulk optimization tools for entire media library

Performance & Quality:

• Additional codec options - eg. JPEGXL
• Support for Ultra HDR (best quality across surfaces)
• Per-size format optimization (smallest file size selection)
• Advanced compression settings and quality controls, eg. UI for selecting image compression level (slider before/after)
• Saliency Detection - detect the most relevant part of an image for improved thumbnail cropping

Progressive enhancement

Details

Cross-Origin Isolation Strategy & Fallbacks
To mitigate the risks of enabling SharedArrayBuffer (required for multi-threaded WASM performance) without breaking the existing plugin/embed ecosystem, we will adopt a Progressive Enhancement strategy:

1. Environment Detection: The editor will detect the environment capability at runtime using window.crossOriginIsolated.
2. Tiered Execution Model:

• Tier A (Ideal): If the environment is isolated (headers are present), use Multi-threaded WASM for maximum performance.
• Tier B (Fallback): If the browser is incompatible or performance is too slow, silently fall back to the standard Server-Side upload mechanism.

3. "Credentialless" Investigation:

• We will not enforce Cross-Origin-Embedder-Policy: require-corp globally as it breaks non-compliant third-party resources (embeds/scripts).
• We will experiment with Cross-Origin-Embedder-Policy: credentialless as a progressive enhancement for Chrome/Firefox and continue to monitor for WebKit support.
• Credentialless allows third-party embeds to load without CORP headers, resolving the "broken embed" issue while still enabling SharedArrayBuffer.

4. Iframe encapsulation (Long-term):

• Evaluate moving the media processing worker into a dedicated credentialless iframe (if supported) to isolate the processing environment without requiring the entire parent window (the Editor) to be strictly isolated.

Known risks / challenges

Details

• @shopify/web-worker is deprecated (though stable) - we may need to maintain it since popular alternatives do not use a compatible license.
• wasm-vips must be inlined to prevent CORS issues. The media-experiments plugin already does this.
• WASM binary bundles are large and can cause bloat to the repository size which stores each updated copy.
• From Adam Zielinski: Storing them in the repo as regular files works, but WebAssembly binaries can take a couple of megabytes, and then every time you update them, at least in git the repository size grows by another, say, 20 megs (it may be less of a problem in SVN). Perhaps it's time to consider switching to Git LFS in the Gutenberg/wordpress-develop git repositories.
• One risk is about compatibility with other parts of the editor (plugins, embed previews) a because of the cross-origin isolation requirement for SharedArrayBuffer.
• Lack of <iframe credentialless> support is not ideal as it degrades editor UX in some cases
• libheif's LGPL v3 license may be a blocker for HEIC support.
• Metadata Preservation (EXIF/IPTC) - the feature must include the same EXIF extraction capabilities as the server side approach, and all meta data must be passed back to the server for storage.
• To use SharedArrayBuffer (essential for wasm-vips performance), Safari (WebKit) requires the site to serve Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp. Chrome and Firefox users can use credentialless to avoid this issue.
• The Consequence:
• Embeds Break if they don’t serve the compatible CORP headers ().
• External Scripts Break: Admin plugins that load scripts from CDNs (e.g., analytics, support chat widgets) will fail unless those CDNs support these headers.
• Plugin compatibility - numerous WordPress plugins rely on server-side hooks like wp_generate_attachment_metadata, image_resize_dimensions, and wp_handle_upload. Not all of these hooks will continue to fire. We may want to simulate firing them or add similar JS side hooks to match functionality. Any changes to the fired hooks will need to be thoroughly documented and carefully communicated to developers.

FAQ

Key architectural decisions and their rationale.

Details

• Why inline WASM?

  • Headers can't be controlled in all WP environments, making serving proper CORS headers consistently impossible. Inlining solves this issue.

• Why @shopify/web-worker despite deprecation?

  • License compatibility - no other popular library provides a compatible license

• Why new sideload endpoint vs. extending existing endpoints?

  • Existing endpoints are designed to process images, we need to upload images that have already been processed.

Glossary

Details

Terms for contributors who may be new to the feature:

• SharedArrayBuffer: A JavaScript object that allows sharing raw binary memory between the main thread and Web Workers. It enables multi-threaded WASM processing but requires Cross-Origin Isolation for security reasons.
• Cross-Origin Isolation (COOP/COEP): A security configuration achieved by setting two HTTP headers: Cross-Origin-Opener-Policy (COOP) and Cross-Origin-Embedder-Policy (COEP). These headers isolate the page from cross-origin resources.
• wasm-vips: A WebAssembly port of libvips, a fast image processing library. It provides client-side image operations (resize, convert, compress) at near-native speed and is the core engine powering this feature's image processing.
• Sideload (in WordPress context): Uploading an image from a URL or external source rather than through the standard upload form. In this project, it refers to uploading pre-processed image sub-sizes from the browser directly to the server, bypassing server-side image generation.
• Sub-sizes: The various thumbnail and intermediate image sizes that WordPress generates from an uploaded image (e.g., thumbnail, medium, large, full). These are defined in Settings → Media and by themes/plugins via add_image_size(), and are stored as separate files on the server. These image sub-sizes are then used to serve the best sized images for users display resolution via the image tag's srcset attribute.