Skip to content

Add SkSL image filter support (ToImageFilter / SKRuntimeImageFilterBuilder)#3778

Open
mattleibow wants to merge 35 commits into
mainfrom
mattleibow/dev-runtime-shader-image-filters
Open

Add SkSL image filter support (ToImageFilter / SKRuntimeImageFilterBuilder)#3778
mattleibow wants to merge 35 commits into
mainfrom
mattleibow/dev-runtime-shader-image-filters

Conversation

@mattleibow

@mattleibow mattleibow commented Apr 28, 2026

Copy link
Copy Markdown
Collaborator

Add support for custom SkSL-based image filters via SKRuntimeEffect.ToImageFilter and SKRuntimeImageFilterBuilder. This wraps SkImageFilters::RuntimeShader (Skia m98+, maxSampleRadius m116+), enabling custom GPU image processing effects that compose with built-in filters.

Fixes #3776

API

SKRuntimeImageFilterBuilder (primary API)

using var builder = SKRuntimeEffect.BuildImageFilter(sksl);
builder.Uniforms["strength"] = 4.0f;

// Auto-detect single child, use source image
var filter = builder.Build();

// With maxSampleRadius for neighbor-sampling shaders
var filter = builder.Build(maxSampleRadius: 1.0f);

// Multi-child (e.g. unsharp mask)
builder.Inputs["content"] = null;   // source image
builder.Inputs["blurred"] = blur;   // blur filter
var filter = builder.Build();

SKRuntimeEffect.ToImageFilter (low-level)

effect.ToImageFilter()
effect.ToImageFilter(childShaderName, input)
effect.ToImageFilter(childShaderName, input, maxSampleRadius)
effect.ToImageFilter(uniforms, children)
effect.ToImageFilter(uniforms, children, childShaderName, input)
effect.ToImageFilter(uniforms, children, childShaderName, input, maxSampleRadius)
effect.ToImageFilter(uniforms, children, inputs)
effect.ToImageFilter(uniforms, children, inputs, maxSampleRadius)

New types

  • SKRuntimeImageFilterBuilder — builder with Inputs property and Build() overloads
  • SKRuntimeEffectImageFilterInputs — dictionary-backed collection mapping child shader names to SKImageFilter? inputs. Names/Count/Contains reflect set entries. Validates names against the effect.

C API (sk_runtimeeffect.h)

  • sk_runtimeeffect_make_image_filter — single-child with maxSampleRadius
  • sk_runtimeeffect_make_image_filter_with_children — multi-child, uses STArray for stack-backed allocations

Companion skia PR: mono/skia#202

Tests

18 tests in SKRuntimeEffectTest.ImageFilters:

  • Auto-detect, named child, null name, with input filter, maxSampleRadius
  • Multi-child creation and rendering
  • Inputs: Names/Count/Contains, Reset, invalid name throws, null vs unset
  • ToImageFilter directly on effect with/without explicit uniforms
  • All objects properly disposed (GarbageCleanupFixture clean)

Gallery Samples

  • Custom SkSL Image Filter — 5 interactive effects (invert, grayscale, sepia, vignette, edge detect). Cached builders.
  • Unsharp Mask — Multi-child pattern. Cached builder, adjustable blur radius and strength.

Expose SkImageFilters::RuntimeShader (available since Skia m98, with
maxSampleRadius since m116) through SkiaSharp's full API stack.

C API (externals/skia):
- sk_imagefilter_new_runtime_shader: single-child variant
- sk_imagefilter_new_runtime_shader_with_children: multi-child with
  maxSampleRadius for boundary handling

C# wrapper (SKImageFilter):
- CreateRuntimeShader(builder, childShaderName)
- CreateRuntimeShader(builder, childShaderName, input)
- CreateRuntimeShader(builder, maxSampleRadius, childShaderName)
- CreateRuntimeShader(builder, maxSampleRadius, childShaderName, input)

Tests: 12 tests covering creation, rendering, uniform passing,
chaining with other filters, and maxSampleRadius.

Gallery sample: RuntimeShaderImageFilterSample with 5 interactive
SkSL effects (invert, grayscale, sepia, vignette, edge detect).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

📦 Try the packages from this PR

Warning

Do not run these scripts without first reviewing the code in this PR.

Step 1 — Download the packages

bash / macOS / Linux:

curl -fsSL https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.sh | bash -s -- 3778

PowerShell / Windows:

iex "& { $(irm https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.ps1) } 3778"

Step 2 — Add the local NuGet source

dotnet nuget add source ~/.skiasharp/hives/pr-3778/packages --name skiasharp-pr-3778
More options
Option Description
--successful-only / -SuccessfulOnly Only use successful builds
--force / -Force Overwrite previously downloaded packages
--list / -List List available artifacts without downloading
--build-id ID / -BuildId ID Download from a specific build

Or download manually from Azure Pipelines — look for the nuget artifact on the build for this PR.

Remove the source when you're done:

dotnet nuget remove source skiasharp-pr-3778

…e-shader-image-filters

# Conflicts:
#	externals/skia
@mattleibow mattleibow force-pushed the mattleibow/dev-runtime-shader-image-filters branch from 115fb71 to 664f4dc Compare April 28, 2026 03:15
mattleibow and others added 6 commits May 5, 2026 21:39
Move the runtime shader image filter API from static factories on
SKImageFilter to instance methods that match the existing pattern:

- SKRuntimeEffect.ToImageFilter() — matches ToShader/ToColorFilter/ToBlender
- SKRuntimeShaderBuilder.BuildImageFilter() — matches Build() → SKShader

Remove SKImageFilter.CreateRuntimeShader static methods.
Update tests and gallery sample to use the new API.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Reorder parameters: childShaderName first (required), input second
  (common), maxSampleRadius last (rare) — matches upstream C++ usage
- Add simpler ToImageFilter overloads on SKRuntimeEffect that don't
  require explicit uniforms/children (matches ToShader()/ToBlender())
- Remove redundant maxSampleRadius-first overloads in favor of
  (childShaderName, input, maxSampleRadius) order
- Move runtime shader image filter tests from SKImageFilterTest to
  SKRuntimeEffectTest.ImageFilters nested class alongside existing
  Shaders/ColorFilters test classes
- Add ToImageFilter test that exercises the effect directly with
  explicit uniforms and children

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Redesign the RuntimeShader image filter API based on thorough analysis
of the upstream C++ implementation (SkRuntimeImageFilter.cpp):

C API (sk_runtimeeffect.h):
- Rename from sk_imagefilter_new_runtime_shader* to
  sk_runtimeeffect_make_image_filter* matching existing pattern
- Single-child and multi-child variants

SKRuntimeEffect.ToImageFilter (10 overloads):
- () — auto-detect single child, implicit source
- (string?, SKImageFilter?) — nullable name + input
- (string?, SKImageFilter?, float) — with maxSampleRadius
- (string[], SKImageFilter?[]) — multi-child parallel arrays
- (string[], SKImageFilter?[], float) — multi-child with radius
- Same 5 with (uniforms, children, ...) prepended

SKRuntimeShaderBuilder.BuildImageFilter (5 overloads):
- Same single/multi-child patterns, delegates to Effect.ToImageFilter

Tests (14 in SKRuntimeEffectTest.ImageFilters):
- Zero-arg auto-detect, fails with multiple children
- Named child, null name auto-detect, with input, with radius
- Rendering tests with pixel verification
- Multi-child creation and rendering
- ToImageFilter directly on effect with/without explicit uniforms

Samples:
- Updated RuntimeShaderImageFilterSample
- New UnsharpMaskSample: multi-child pattern from upstream gm
  (content + blurred children, adjustable blur and strength)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add SKRuntimeImageFilterBuilder with Build() overloads matching
  the existing SKRuntimeShaderBuilder/ColorFilter/Blender pattern
- Add SKRuntimeEffect.BuildImageFilter() factory method
- Remove BuildImageFilter from SKRuntimeShaderBuilder
- Configure libSkiaSharp.json to marshal childShaderName as LPStr
  and childShaderNames as LPArray/LPStr, eliminating manual string
  marshalling (GCHandle, GetEncodedText, pinning)
- Fix em-dash characters to ASCII dashes in comments
- Update tests and samples to use new builder type
- Suppress pre-existing CS0618 in FillPathSample.cs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e-shader-image-filters

# Conflicts:
#	externals/skia
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mattleibow mattleibow changed the title Add SKImageFilter.CreateRuntimeShader Add SkSL image filter support (ToImageFilter / BuildImageFilter) May 5, 2026
mattleibow and others added 3 commits May 6, 2026 00:42
Force all gallery samples to explicitly declare these properties.
Fix DateOnly? to non-nullable DateOnly across all samples and
SampleManager. Add DateAdded to new image filter samples.
Fix picker cast in RuntimeShaderImageFilterSample.
Fix sample category references (SampleCategories -> SampleManager).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New class mirrors SKRuntimeEffectUniforms/SKRuntimeEffectChildren
pattern. Builder.Inputs["child"] = blur sets image filter inputs
by child shader name. Build() reads from Inputs automatically.

Builder now has just Build() and Build(float maxSampleRadius).
No more parallel string[]/SKImageFilter?[] arrays in user code.

Usage:
  builder.Inputs["content"] = null;   // source image
  builder.Inputs["blurred"] = blur;   // blur filter
  var filter = builder.Build();

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

📖 Documentation Preview

The documentation for this PR has been deployed and is available at:

🔗 View Staging Site
🔗 View Staging Docs
🔗 View Staging Gallery (Blazor)
🔗 View Staging Gallery (Uno Platform)
🔗 View Staging SkiaFiddle

This preview will be updated automatically when you push new commits to this PR.


This comment is automatically updated by the documentation staging workflow.

mattleibow and others added 8 commits May 6, 2026 01:55
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Only children the user actually set via Inputs[name] = filter are
passed to the C API. Unset children are not included, matching the
C++ behavior where only listed childShaderNames get overwritten.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Names, Count, Contains, and enumeration now map to the dictionary
of explicitly set inputs. Validation on Add still checks against
the effect's child names via a HashSet.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
WriteTo writes into caller-provided arrays instead of allocating.
Caller uses Utils.RentArray for pooled arrays. Internal ToImageFilterMulti
takes explicit count for rented arrays that may be oversized.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
C++ ref-counts uniform data and children without copying, so these
internal accessors could skip the defensive copy. However, since C#
writes directly into the SKData buffer, sharing is unsafe if uniforms
are mutated after creating a filter. Keep using ToData/ToArray for
now; #3859 tracks the proper fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mattleibow mattleibow force-pushed the mattleibow/dev-runtime-shader-image-filters branch from bb4f3bd to 4b6d6e0 Compare May 6, 2026 00:47
mattleibow and others added 4 commits May 6, 2026 02:53
…vs unset

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…stances

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Consistent with children handles pattern. Both use pooled IntPtr
arrays from ArrayPool via Utils.RentHandlesArray.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Private ToImageFilter methods now take SKRuntimeEffectUniforms and
SKRuntimeEffectChildren directly, using AsArray() to access the
backing SKObject[] without copying. Uniforms still use ToData()
since the SKData buffer is mutable and must be snapshotted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Comment thread binding/SkiaSharp/SKRuntimeEffect.cs Outdated
Comment thread binding/SkiaSharp/SKRuntimeEffect.cs Outdated
Comment thread tests/Tests/SkiaSharp/SKRuntimeEffectTest.cs Outdated
Comment thread samples/Gallery/Shared/Samples/RuntimeShaderImageFilterSample.cs Outdated
Comment thread binding/SkiaSharp/SKRuntimeEffect.cs
Comment thread binding/SkiaSharp/SKRuntimeEffect.cs
…t coverage

- ArgumentOutOfRangeException uses nameof(name) not the value
- Names returns dictionary Keys directly (no ToList allocation)
- ToData() held in using local to prevent GC before P/Invoke
- Null-vs-unset test renders and compares actual pixels
- cachedBuilders sized from ShaderSources.Length

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comment thread binding/SkiaSharp/SKRuntimeEffect.cs
Comment thread samples/Gallery/Shared/Samples/RuntimeShaderImageFilterSample.cs
mattleibow and others added 2 commits May 6, 2026 18:03
- SKRuntimeEffectImageFilterInputs now queries child type metadata
  and only accepts uniform shader children, rejecting colorFilter
  and blender children with a clear error message
- Replace per-frame float[] allocation with stackalloc span for
  vignette resolution uniform

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
stackalloc span cannot be passed to ref struct SKRuntimeEffectUniform
due to scope escape rules. Revert to float[] - acceptable for a sample.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e-shader-image-filters

# Conflicts:
#	externals/skia

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

samples/Gallery/Shared/Samples/RuntimeShaderImageFilterSample.cs:113

  • OnDestroy() does not call base.OnDestroy(). The base implementation handles cleanup for the animation refresh loop (CTS cancellation/disposal) and is called by other samples; please call base.OnDestroy() to keep lifecycle behavior consistent.
	protected override void OnDestroy()
	{
		sourceBitmap?.Dispose();
		sourceBitmap = null;
		for (var i = 0; i < cachedBuilders.Length; i++)
		{
			cachedBuilders[i]?.Dispose();
			cachedBuilders[i] = null;
		}
	}

samples/Gallery/Shared/Samples/UnsharpMaskSample.cs:54

  • OnDestroy() does not call base.OnDestroy(). The base method cancels/disposes the animation refresh CTS; calling it keeps cleanup consistent with other samples and avoids leaks if animation is added later.
	protected override void OnDestroy()
	{
		sourceBitmap?.Dispose();
		sourceBitmap = null;
		builder?.Dispose();
		builder = null;
	}

Comment on lines +95 to +102
protected override Task OnInit()
{
using var stream = new SKManagedStream(SampleMedia.Images.Baboon);
sourceBitmap = SKBitmap.Decode(stream);
for (var i = 0; i < ShaderSources.Length; i++)
cachedBuilders[i] = SKRuntimeEffect.BuildImageFilter(ShaderSources[i]);
return Task.CompletedTask;
}
Comment on lines +40 to +46
protected override Task OnInit()
{
using var stream = new SKManagedStream(SampleMedia.Images.Baboon);
sourceBitmap = SKBitmap.Decode(stream);
builder = SKRuntimeEffect.BuildImageFilter(UnsharpShader);
return Task.CompletedTask;
}
mattleibow and others added 2 commits May 17, 2026 18:42
The gallery redesign (main) made ApiTags abstract on SampleBase.
Add tags for SKRuntimeEffect, SKRuntimeImageFilterBuilder, and
related APIs to both RuntimeShaderImageFilterSample and
UnsharpMaskSample.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ramezgerges

Copy link
Copy Markdown
Contributor

Review

C# side

🔴 Missing GC.KeepAlive across the new P/Invoke calls

The two new private methods omit the GC.KeepAlive calls that every sibling method in this file already performs. Compare:

// SKRuntimeEffect.cs — existing ToShader
fixed (IntPtr* ch = childrenHandles) {
    var shader = SkiaApi.sk_runtimeeffect_make_shader (...);
    GC.KeepAlive (uniforms);
    GC.KeepAlive (this);   // <-- present
    return shader;
}
// new ToImageFilterSingle — no KeepAlive
fixed (IntPtr* ch = childrenHandles) {
    return SKImageFilter.GetObject (SkiaApi.sk_runtimeeffect_make_image_filter (
        Handle, uniformsHandle, ch, (IntPtr)childrenHandles.Length,
        maxSampleRadius, childShaderName ?? string.Empty, input?.Handle ?? IntPtr.Zero));
}

Handle reads this, and after the last managed use the JIT may treat this (and the input filter in the single-child path) as collectible during the native call — the finalizer can then unref the underlying native object mid-call. This is exactly the lifetime failure class the recent hardening work targets ("Keep 'this' alive across instance P/Invoke calls", "Hold native-wrapper arguments alive across P/Invoke calls"), so the new entry points should match that pattern rather than regress it.

Requested change — in both ToImageFilterSingle and ToImageFilterMulti, inside the fixed block before returning, add:

  • GC.KeepAlive(this)
  • GC.KeepAlive(input) (single-child path)
  • GC.KeepAlive(uniformData) — the using already covers it, but keep it explicit for parity with the sibling methods

C side (sk_runtimeeffect.cpp / sk_imagefilter.cpp)

🟡 Unused include in sk_imagefilter.cpp

The diff adds #include "include/effects/SkRuntimeEffect.h" to sk_imagefilter.cpp, but nothing in that file references it — the new runtime image-filter functions live entirely in sk_runtimeeffect.cpp. Please drop the include.

🟡 make_image_filter_with_children doesn't guard against null child-shader names

SkImageFilters::RuntimeShader's multi-child overload is documented to return nullptr if any name is null. But the shim constructs the string_view before the call:

names.push_back(std::string_view(childShaderNames[i]));   // strlen(nullptr) on a null entry = UB

std::string_view(const char*) on nullptr is UB (a crash), not a graceful null return. The single-child function guards this defensively (childShaderName ? std::string_view(childShaderName) : std::string_view()); the multi-child one does not. The C# layer never passes null names (they're validated Dictionary keys), so this is safe today and consistent with "the C API trusts the caller" — flagging it as an optional parity/robustness guard, not a blocker.

🟢 Factor the shared preamble into a helper (keep both entry points)

The two functions are near-identical in their preamble — same builder construction and child-binding loop:

SkRuntimeShaderBuilder builder(sk_ref_sp(AsRuntimeEffect(effect)), sk_ref_sp(AsData(uniforms)));
auto effectChildren = AsRuntimeEffect(effect)->children();
for (size_t i = 0; i < childCount && i < effectChildren.size(); i++) {
    builder.child(effectChildren[i].name) = sk_ref_sp(AsFlattenable(children[i]));
}

Extract that into a static helper and have both entry points call it:

static void sk_runtimeeffect_bind_children(
        SkRuntimeShaderBuilder& builder, sk_runtimeeffect_t* effect,
        sk_flattenable_t** children, size_t childCount) {
    auto effectChildren = AsRuntimeEffect(effect)->children();
    for (size_t i = 0; i < childCount && i < effectChildren.size(); i++) {
        builder.child(effectChildren[i].name) = sk_ref_sp(AsFlattenable(children[i]));
    }
}

Each function then becomes:

SkRuntimeShaderBuilder builder(sk_ref_sp(AsRuntimeEffect(effect)), sk_ref_sp(AsData(uniforms)));
sk_runtimeeffect_bind_children(builder, effect, children, childCount);
// ... single- or multi-child RuntimeShader call

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

[api] Add SkSL image filter support (SKRuntimeEffect.ToImageFilter)

3 participants