Add native WGSL (genwgsl) shader generator with dedicated data library by ashwinbhat · Pull Request #2996 · AcademySoftwareFoundation/MaterialX

ashwinbhat · 2026-07-03T01:25:22Z

The current glsl to wgsl pathway offered in MaterialX is not suitable for runtime systems.
There is an elaborate pipeline from MaterialX> GLSL -> SPIR-V -> WGSL -> [Browser]. The browser internally might convert WGSL -> SPIRV before creating byte code.

The transplied WGSL is verbose and difficult to read. The GLSL data structures are flattened sometimes due to the SPIR-V path.

This Introduces a C++ MaterialXGenWgsl backend with genwgsl shader target that has a a dedicated WGSL data library.

The WGSL data library is pre-transpiled using a helper script (glsl_to_wgsl.py) that uses Naga to create the shader nodes in WGSL data library

The MaterialX three.js viewer backend now uses WgslShaderGenerator. Unlike GLSLES, three.js does not support direct raw shaders. therefore we have an adapter to create Three.js TSL.

Note about AI use:
The script glsl_to_wgsl.py was authored using AI tools that uses naga and then performs cleanup of the shader code using simple lookup.
Screenshots:

WebGPU viewer is host here: https://ashwinbhat.github.io/MaterialX/index-webgpu.html

Introduce a native WebGPU/WGSL shader generator as a new "genwgsl" target, replacing the old GLSL-rewriting WgslShaderGenerator in MaterialXGenGlsl. - MaterialXGenWgsl is HwShaderGenerator based generator that emits complete standalone WGSL so that there is no transpiling required. - MATERIALX_BUILD_GEN_WGSL option to enable this project - Add js and python binding. The main addition is a helper python script that uses naga to convert glsl data library to wgsl data library. The script is not a general purpose converter. It run in few phases 1. wraps a glsl fragment into a shader and uses naga to generate wgsl. 2. the generated wgsl is cleaned up to keep doc comments, and maintian similar code style as glsl 3. Add a banner to alert future authors to not edit the wgsl directly. Note: only stright forward simple conversion is done. there are cases with glsl overloads in math and lib that have hand written wgsl counterparts. see source\MaterialXGenWgsl\tools\glsl_to_wgsl.py and associated readme.md

Added genwgsl library what was transpiled offline. Add an opt-in `MATERIALX_GENERATE_WGSL_LIBRARY` build option that regenerates and validates the library at build time. This needs Cargo the official build tool and package manager for Rust. MATERIALX_CARGO_PATH can be set to define path to Cargo. See https://doc.rust-lang.org/cargo/getting-started/installation.html

Render MaterialX materials over WebGPU by using the native WgslShaderGenerator The main change here to support Three.js wgslFn: WGSL function node, a TSL function. https://threejs.org/docs/#FunctionNode - mxtsladapter.js: converts the complete WGSL module to TSL-portable format - wgslmanifest.js: reconstruct the reflection manifest in JS from the generated WGSL + Shader uniform ports - viewer.js/index.js: dual WebGL (ESSL) + WebGPU (WGSL) backends selected per bundle; WebGPU path uses WebGPURenderer + the TSL bridge.

We might look into naga-cli

jstone-lucasfilm · 2026-07-07T03:33:00Z

Thanks for this substantial contribution, @ashwinbhat, and indeed this seems more promising than your previous #2983.

Before moving forward with detailed review, I think it's worthwhile to consider a simplification of this idea:

What if the GLSL-to-WGSL transformation step occurred only in GitHub CI, rather than on the developer's machine? The machine-generated WGSL files would then become a derived artifact rather than a maintained set of code in the repository. The source GLSL files would remain the single source of truth for hardware shading, CI would run the transpiler on each PR and package the results into our build artifacts, and developers not working with WGSL could rely upon CI to validate that they haven't broken the WGSL target with their changes. Since the generated files would no longer be committed, drift between the two libraries becomes impossible by construction, rather than managed through community conventions.

On the build side, we might then default MATERIALX_BUILD_GEN_WGSL to OFF, with GitHub CI providing Naga and setting it to ON, so that ordinary source builds require no Python or Rust toolchain. The hand-written portion of the library (the lib/ helpers, the image and light nodes, and the adapted BSDFs) would remain in the repository as maintained code, analogous to the small libraries of hand-written code in MSL and Slang today.

This direction would preserve what I see as the core contributions of this PR -- the native HwShaderGenerator-derived backend, the resource binding model, and the WebGPU viewer integration -- while shifting the transpiled library from maintained code to build infrastructure. Would you be open to exploring this path?

These will be generated as part of the CI. Note: there are still a few wgsl files that are hand written since their glsl counterparts use overloads, defines and storage qualifiers

- Add generate_wgsl rule that installs naga-cli and updates data library with wgsl TODO: check if cargo is availble on CI runners

ashwinbhat · 2026-07-07T21:42:16Z

Thanks for this substantial contribution, @ashwinbhat, and indeed this seems more promising than your previous #2983.

Before moving forward with detailed review, I think it's worthwhile to consider a simplification of this idea:

What if the GLSL-to-WGSL transformation step occurred only in GitHub CI, rather than on the developer's machine? The machine-generated WGSL files would then become a derived artifact rather than a maintained set of code in the repository. The source GLSL files would remain the single source of truth for hardware shading, CI would run the transpiler on each PR and package the results into our build artifacts, and developers not working with WGSL could rely upon CI to validate that they haven't broken the WGSL target with their changes. Since the generated files would no longer be committed, drift between the two libraries becomes impossible by construction, rather than managed through community conventions.

On the build side, we might then default MATERIALX_BUILD_GEN_WGSL to OFF, with GitHub CI providing Naga and setting it to ON, so that ordinary source builds require no Python or Rust toolchain. The hand-written portion of the library (the lib/ helpers, the image and light nodes, and the adapted BSDFs) would remain in the repository as maintained code, analogous to the small libraries of hand-written code in MSL and Slang today.

This direction would preserve what I see as the core contributions of this PR -- the native HwShaderGenerator-derived backend, the resource binding model, and the WebGPU viewer integration -- while shifting the transpiled library from maintained code to build infrastructure. Would you be open to exploring this path?

Thanks @jstone-lucasfilm for the valuable feedback. I've removed the generated wgsl files and moved the generation to GitHub CI.

The MaterialX WebGPU viewer is hosted here: https://ashwinbhat.github.io/MaterialX/index-webgpu.html

kwokcb

I don't follow all the remapping from GLSL to WGSL but assume it's good.

For building:

It would still be useful to have a script or set of instructions on how to build locally if you want to change and test anything. e.g. if I modify or add a GLSL file how do you rebuild it to test.

For release a few questions

There is a comment about switching between web pages
- Curious why are there 2 web pages -- seems like it's the simplest way to webpack ?
Are there any issues with instantiating both a WebGL or WebGPU generators at the same time if desired on web ?
Are all the generated WGSL files installed (visible to look at) in the release ?

kwokcb · 2026-07-09T17:38:57Z

+ * Configure genContext for WebGPU material generation.
+ * @returns {boolean} Whether the surface is transparent.
+ */
+function configureWebGPUGenContext(mx, gen, genContext, elem)


This file is getting large and hard to read. Would it make sense to have a separate file for WebGPU specific functions.

I can move it into a separate file as a future refactoring step

kwokcb · 2026-07-09T17:39:26Z

+// Rendering backend, selected at build time per bundle (see webpack.config.js). The WebGL
+// bundle uses classic THREE.WebGLRenderer + RawShaderMaterial (ESSL); the WebGPU bundle
+// aliases `three` to three/webgpu and uses WebGPURenderer + NodeMaterial (WGSL via the
+// upstream WgslShaderGenerator). A toggle switches between the two HTML pages.


Curious why 2 web pages ? Does it make sense to instead do a run-time switch, or allow for 2 different canvases with different associated renderers ?

I couldn't figure out how to get the UX and other elements to work seamlessly. We can improve this in future if the core generator is approved.

kwokcb · 2026-07-09T17:43:23Z

        python python/Scripts/generateshader.py resources/Materials/Examples --target glsl --validator C:/vcpkg/installed/x64-windows-release/tools/glslang/glslangValidator.exe
        python python/Scripts/generateshader.py resources/Materials/Examples/StandardSurface --target essl --validator C:/vcpkg/installed/x64-windows-release/tools/glslang/glslangValidator.exe
        python python/Scripts/generateshader.py resources/Materials/Examples/StandardSurface --target vulkan --validator C:/vcpkg/installed/x64-windows-release/tools/glslang/glslangValidator.exe
-        python python/Scripts/generateshader.py resources/Materials/Examples/StandardSurface --target wgsl --validator "C:/vcpkg/installed/x64-windows-release/tools/glslang/glslangValidator.exe --target-env vulkan1.3 --quiet"


Why is this removed ? If wgsl is available should it not be run still.

The current validator uses glsllang that does not support wgsl syntax.
Note that the current generator was generating "glsl" code but the new one will generate "wgsl" code

kwokcb · 2026-07-09T17:44:30Z

      - name: Create Archive Name
        run: echo "MATERIALX_ARCHIVE=MaterialX-${RELEASE_TAG//v}" >> $GITHUB_ENV

+      - name: Install Python


Curious if it's possible to call into a shared setup from main and release ?

We have a config that is no Python based, so I kept it separate.

ashwinbhat · 2026-07-09T20:57:39Z

I don't follow all the remapping from GLSL to WGSL but assume it's good.

For building:

It would still be useful to have a script or set of instructions on how to build locally if you want to change and test anything. e.g. if I modify or add a GLSL file how do you rebuild it to test.

I'll a document in the developer docs that adds steps/instructions and details about the remapping.

Are there any issues with instantiating both a WebGL or WebGPU generators at the same time if desired on web ?

Curious why are there 2 web pages -- seems like it's the simplest way to webpack ?
WebGL and WebGPU are separate shader generators so it should work. The reason for separate pages is because the workflows are different

WebGL : THREE.WebGLRenderer + RawShaderMaterial (ESSL)
WebGPU :THREE.WebGPURenderer + TSL/NodeMaterial (WGSL)

Are all the generated WGSL files installed (visible to look at) in the release ?

Yes they should be part of zip, there should be a way to verify this.

ashwinbhat · 2026-07-09T21:59:09Z

@kwokcb I added a developer doc which will be helpful to answer some of your question about local developer workflow

jstone-lucasfilm · 2026-07-11T22:23:08Z

Thanks for turning this around so quickly, @ashwinbhat. Moving the node fragments to CI generation is exactly the direction I was hoping for, and the WebGPU viewer looks great.

One remaining piece I'd like to resolve before diving into detailed review: the BSDF microfacet library still lands in the repository as a committed, hand-written WGSL copy of genglsl/lib/mx_microfacet*.glsl. This hardware shading library is arguably one of the most critical parts of the MaterialX project, and I'd rather not have two references that readers must navigate and keep in agreement.

The goal I'd propose is that the GLSL microfacet library remains the single source of truth, and the WGSL form is a CI-generated artifact that is never committed -- the same treatment the WGSL node fragments now get. From reading glsl_to_wgsl.py, this looks within reach: the main obstacle (WGSL's lack of overloading) is already solved for call sites via CALL_MAP, and the genuinely WGSL-specific residue in the lib is small.

Here's a proposed set of next steps to consider:

Extend the transpiler to also generate the lib/ helpers, so nothing under genwgsl/lib/ is committed.
Upstream any genuine divergences into the GLSL source, e.g. mx_orthonormal_basis has already drifted (the sign branch moved to fix a tangent-frame artifact), and that fix currently lives only in the WGSL copy.

It's worth noting that generation from GLSL makes drift like the mx_orthonormal_basis case impossible by construction. In the longer term I'd see render-comparison coverage for the WGSL target (building on our nightly MSL/OSL comparisons) as a natural validation gate, though that's future context rather than a request for this PR.

Does this seem like a reasonable path to you?

ashwinbhat added 3 commits July 2, 2026 17:38

ashwinbhat requested review from jstone-lucasfilm and kwokcb July 3, 2026 01:25

ashwinbhat and others added 3 commits July 2, 2026 19:16

Add Python binding for WgslShaderGenerator

dc547d0

fixup: disable wgsl SPIRV validation

402e599

We might look into naga-cli

Merge branch 'main' into bhata/wgsl_generator_with_library

ac17415

ashwinbhat added 3 commits July 7, 2026 09:36

Delete auto-generated wgsl shaders fragments

fae361f

These will be generated as part of the CI. Note: there are still a few wgsl files that are hand written since their glsl counterparts use overloads, defines and storage qualifiers

Enable wgsl data library generation during CI

bbd7fb2

- Add generate_wgsl rule that installs naga-cli and updates data library with wgsl TODO: check if cargo is availble on CI runners

fix (build): Enable WGSL on python enabled runners

a2703ab

This was referenced Jul 7, 2026

WebGPU Viewer #2983

Closed

Update MaterialX WebViewer to use WebGPU #2558

Open

Improve WgslShaderGenerator to generate actual wgsl source #2751

Open

Merge branch 'main' into bhata/wgsl_generator_with_library

0e96799

kwokcb reviewed Jul 9, 2026

View reviewed changes

Add developer documentation for WGSL Shader Generation

8b0b063

Merge branch 'main' into bhata/wgsl_generator_with_library

7d89a79

Uh oh!

Conversation

ashwinbhat commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jstone-lucasfilm commented Jul 7, 2026

Uh oh!

ashwinbhat commented Jul 7, 2026

Uh oh!

kwokcb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashwinbhat commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ashwinbhat commented Jul 9, 2026

Uh oh!

jstone-lucasfilm commented Jul 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ashwinbhat commented Jul 3, 2026 •

edited

Loading

ashwinbhat commented Jul 9, 2026 •

edited

Loading