Media: Add HEIC canvas fallback for client-side processing

[adamsilverstein]

Description

HEIC (HEIF with HEVC codec) images are the default photo format on iPhones but are not universally supported by web servers or browsers. Currently, if the server lacks HEIC support (no ImageMagick HEVC module), uploads may fail entirely because wp_prevent_unsupported_mime_type_uploads blocks them.

Additionally, Safari users are excluded from client-side media processing because Safari lacks Document-Isolation-Policy support (required for SharedArrayBuffer/VIPS). However, Safari can natively decode HEIC via createImageBitmap() using macOS platform codecs.

Solution

Implement a server-first, canvas-fallback strategy with two processing tiers:

Tier 1: HEIC infrastructure (all browsers)

HEIC MIME type registration, the custom REST controller (with sideload endpoint and HEIC bypass), and REST field/index registrations load whenever the media processing feature filter is enabled — independent of VIPS/SharedArrayBuffer availability.

Tier 2: Full VIPS processing (Chromium only)

Cross-origin isolation (DIP headers), WASM MIME types, and the __clientSideMediaProcessing flag only load when the browser supports SharedArrayBuffer.

Upload flows

Server has HEIC support: Upload → server processes everything → done.

Browser can decode, VIPS available (Chromium): Upload HEIC → canvas convert → JPEG → sideload as scaled → generate sub-sizes client-side via VIPS → sideload each → finalize.

Browser can decode, no VIPS (Safari): Upload HEIC → canvas convert → JPEG → sideload as scaled with generate_sub_sizes: true → server generates all sub-sizes from the JPEG → finalize.

Neither supports HEIC: Upload → all client-side decode strategies fail → error shown to user.

Client-side decoding strategies

The canvas conversion tries three strategies in order:

1. createImageBitmap() + OffscreenCanvas — Works natively in Safari. The simplest path.
2. WebCodecs ImageDecoder API — Uses platform codecs via WebCodecs. May work in future Chrome if HEIC is added.
3. HEIC container parsing + WebCodecs VideoDecoder — Chrome 107+ on macOS. Parses ISOBMFF container to extract HEVC bitstream, decodes via hardware-accelerated VideoDecoder (macOS VideoToolbox). Handles grid/tiled HEIC (common iPhone format).

No HEIC decoder or patented codec is shipped — only standard container parsing is done client-side, and decoding uses the browser's already-licensed platform decoders.

Browser support matrix

Browser	Platform	Strategy	HEIC decode	Sub-size generation
Safari	macOS/iOS	createImageBitmap	Works	Server-side
Chrome 107+	macOS	VideoDecoder (HEVC via VideoToolbox)	Works	Client-side (VIPS)
Chrome	Windows (with HEVC codec)	VideoDecoder	Should work	Client-side (VIPS)
Chrome	Linux	—	No platform HEVC decoder	—
Firefox	All	—	No HEIC/HEVC support	—

Implementation

PR: #76731

Technical diagram

flowchart TD
    A["User selects HEIC file"] --> B{"Server has<br/>HEIC support?"}

    B -->|Yes| C["Upload HEIC"]
    C --> D["Server generates<br/>all sub-sizes"]
    D --> E["Done ✓"]

    B -->|No| F{"Browser can<br/>decode HEIC?"}

    F -->|No| G["Error: cannot<br/>process HEIC ✗"]

    F -->|Yes| H["Upload original HEIC<br/>(generate_sub_sizes: false)"]
    H --> I["Canvas convert<br/>HEIC → JPEG"]

    I --> J{"VIPS available?<br/>(SharedArrayBuffer)"}

    J -->|"Yes (Chromium)"| K["Sideload JPEG"]
    K --> L["Generate each sub-size<br/>client-side via VIPS"]
    L --> M["Sideload each<br/>sub-size to server"]
    M --> N["Finalize ✓"]

    J -->|"No (Safari)"| O["Sideload JPEG<br/>(generate_sub_sizes: true)"]
    O --> P["Server generates<br/>all sub-sizes from JPEG"]
    P --> Q["Finalize ✓"]

    subgraph decode ["Canvas decoding strategies (tried in order)"]
        direction TB
        S1["1. createImageBitmap()"] -->|fails| S2["2. ImageDecoder API"]
        S2 -->|fails| S3["3. HEIC parser +<br/>VideoDecoder"]
        S3 -->|fails| S4["Error: cannot\nprocess HEIC ✗"]
    end

    I -.->|"uses"| decode

    style E fill:#2d6,stroke:#1a4,color:#fff
    style N fill:#2d6,stroke:#1a4,color:#fff
    style Q fill:#2d6,stroke:#1a4,color:#fff
    style G fill:#d33,stroke:#a22,color:#fff
    style S4 fill:#d33,stroke:#a22,color:#fff

Loading