Skip to content

feat: add WebGPU expert workers - #552

Closed
gauravsaini wants to merge 2 commits into
JustVugg:devfrom
gauravsaini:webgpu-upstream
Closed

feat: add WebGPU expert workers#552
gauravsaini wants to merge 2 commits into
JustVugg:devfrom
gauravsaini:webgpu-upstream

Conversation

@gauravsaini

Copy link
Copy Markdown

Summary

Adds an optional browser WebGPU expert-worker path while preserving compatibility with the existing native expert-worker wire contract.

  • Adds a dependency-free Python WebSocket/data proxy and worker registry.
  • Adds a browser WebGPU FFN worker and f32 safetensors exporter.
  • Adds C coordinator dispatch through the shared COLIEX01 framing.
  • Adds explicit compatibility gates: C byte-level framing test, Python browser dispatch test, and a native TCP client through the WebGPU proxy.
  • Keeps the first format little-endian f32 so native and browser endpoints have an auditable contract.

Validation

  • make colibri
  • make test-c
  • uv run python -m unittest discover -s tests -p 'test_webgpu.py' (3 tests)
  • uv run python -m py_compile webgpu.py tools/export_webgpu_expert.py
  • node --check web/public/webgpu-worker.js
  • git diff --check

This is intentionally separate from dense sharding (#550) and distributed native expert workers (#551). Existing PR #380 remains open.

@JustVugg

Copy link
Copy Markdown
Owner

Thank you β€” and this is the cleanest of your three split PRs (nearly all the surface is in an isolated webgpu.h + Python, only +23 lines in colibri.c). But I'm going to be honest about direction: the project's focus right now is depth on the models we already run on a single consumer box (GLM-5.2, Inkling, DeepSeek), not multi-machine/browser breadth. A WebGPU-worker offload path, however clean, parks a bespoke protocol as dormant code in every build and doesn't advance that goal today. So I'm deferring this β€” not rejecting: it's env-gated OFF (default path provably safe) and well-isolated, so it stays open and mergeable for when breadth becomes the priority. It also conflicts now with #550/#551 (all three touch the same moe() insertion) β€” only one can merge clean anyway. No work lost.

@JustVugg JustVugg added the enhancement New feature or request label Jul 28, 2026
@JustVugg

JustVugg commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Closing this one, ten days after deferring it, and you are owed a straight reason rather than silence.

Nothing about the engineering changed my mind β€” what I said in July still stands, and it was complimentary: this was the cleanest of your three split PRs, nearly all of it isolated in webgpu.h plus Python, only +23 lines touching colibri.c. That is how a contributor should stage an experimental backend.

The direction did not change either, which is the actual problem. The project's work since then has gone entirely into depth on the four model families that run on one machine β€” a Vulkan backend, Kimi K3's GPU tier, Windows portability, packaging, the shared routing telemetry. A WebGPU worker path is orthogonal to all of it, and leaving it open implied it was queued when it was not.

Carrying it open costs more than closing it. A dormant bespoke protocol in every build is surface that has to keep compiling as four engines and three GPU backends move underneath it, and nobody would have noticed if it stopped working β€” we had exactly that happen with two engines that quietly stopped compiling on Windows for months (#720/#736). An unowned, unexercised code path is not free.

So: closed as not-now, not as wrong. The branch stays yours, and if the project ever moves toward distributed or browser execution this is the natural starting point and I would rather reopen this than have someone start over.

The same applies to #550 and #551, and I would rather say it plainly than leave all three drifting. #550 was the strongest of the three and shipped a real token-exact baseline-vs-delegated gate β€” that discipline is the reason I keep saying yours were well-made rather than unwelcome. #551 has the one substantive objection: cluster_worker_run reimplements the expert FFN as a parallel copy of moe()'s inner compute with nothing proving the worker's output matches local, and a re-implemented core kernel needs that gate before it can land regardless of priority.

Thank you for splitting all three out when asked. That made them reviewable, and it is why I can tell you exactly which one has a technical blocker and which two are purely about where attention is going.

@JustVugg JustVugg closed this Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants