Problem
At tensor4all/cubecl@1c88bb6, a CUDA binding's cursor is recorded when its managed allocation is bound, but scheduler-managed kernel writes do not publish a newer cursor/version for their output bindings. Raw external CUDA writes have the same gap.
Relevant flow:
ComputeClient::empty applies the layout policy and submits ComputeServer::initialize_memory;
- allocation
Command::bind records the current stream cursor on the managed-memory slice;
launch_checked resolves every binding and materializes its resource, but Command::resource / MemoryManagement::get_resource only fetch storage and do not update the binding cursor;
CudaServer::raw_stream resolves the current stream with no bindings;
MultiStream decides whether a target stream needs an event by comparing handle_cursor(binding) with last_synced[origin].
This permits the following race even for an ordinary fresh kernel output:
- thread A allocates output on origin stream at binding cursor
k;
- before A submits the initialization/output kernel, thread B uses an unrelated origin-stream binding and records
B.last_synced[A] >= k;
- A submits the kernel at a later stream cursor, but the output binding cursor remains
k;
- after A returns the output, B first uses it, sees cursor
k already synchronized, and skips an event covering A's kernel.
The output does not need to escape before its producer returns; the unrelated origin binding is sufficient to establish the stale target-side synchronization. Separate submit calls can interleave on the shared server channel.
An external cudaMemsetAsync performed after get_resource / raw_stream has the same defect, plus it is not represented by a CubeCL launch at all. Keeping the allocation handle alive provides lifetime safety but not dependency publication. Holding with_server only after ordinary allocation also leaves the old binding cursor.
This was found while Tenferro replaced a host-zero H2D upload with device initialization:
Requested core contract
Every scheduler-managed operation that writes a binding must publish a binding version/cursor covering that write. Allocation plus immediate initialization must also be atomic with respect to target-stream alignment, or expose a single scheduler command that binds, initializes, and publishes the result.
The smallest acceptable design should:
- distinguish read and write bindings at scheduler admission;
- use the client's normal layout/allocation policy;
- enqueue work and publish the written binding cursor/version under the same server ordering boundary;
- make later cross-stream use wait for an event that includes the latest write, even if the target previously synchronized to the origin through an unrelated binding;
- preserve asynchronous execution, error propagation, resource lifetime, and existing same-stream behavior;
- avoid a global runtime lock at the public backend layer.
After the ordinary write-publication contract is fixed, provide a scheduler-managed way to enqueue an external CUDA write against explicit output bindings. A narrower scheduler-managed zero/memset operation is enough for the current consumer.
Acceptance
- a deterministic two-thread/multi-stream test forces target synchronization between fresh allocation and producer launch, then proves target use waits for the produced value;
- repeat the interleaving test for an admitted external write or scheduler-managed memset;
- same-stream and cross-stream zero initialization produce exact zero;
- no host allocation, H2D/D2H transfer, or global public-backend lock;
- existing layout policy and error/stream-health behavior remain intact.
This is not a request for a raw allocator or another untracked pointer escape hatch. The missing invariant is publication of every binding write in CubeCL's dependency graph.
Problem
At
tensor4all/cubecl@1c88bb6, a CUDA binding's cursor is recorded when its managed allocation is bound, but scheduler-managed kernel writes do not publish a newer cursor/version for their output bindings. Raw external CUDA writes have the same gap.Relevant flow:
ComputeClient::emptyapplies the layout policy and submitsComputeServer::initialize_memory;Command::bindrecords the current stream cursor on the managed-memory slice;launch_checkedresolves every binding and materializes its resource, butCommand::resource/MemoryManagement::get_resourceonly fetch storage and do not update the binding cursor;CudaServer::raw_streamresolves the current stream with no bindings;MultiStreamdecides whether a target stream needs an event by comparinghandle_cursor(binding)withlast_synced[origin].This permits the following race even for an ordinary fresh kernel output:
k;B.last_synced[A] >= k;k;kalready synchronized, and skips an event covering A's kernel.The output does not need to escape before its producer returns; the unrelated origin binding is sufficient to establish the stale target-side synchronization. Separate
submitcalls can interleave on the shared server channel.An external
cudaMemsetAsyncperformed afterget_resource/raw_streamhas the same defect, plus it is not represented by a CubeCL launch at all. Keeping the allocation handle alive provides lifetime safety but not dependency publication. Holdingwith_serveronly after ordinary allocation also leaves the old binding cursor.This was found while Tenferro replaced a host-zero H2D upload with device initialization:
Requested core contract
Every scheduler-managed operation that writes a binding must publish a binding version/cursor covering that write. Allocation plus immediate initialization must also be atomic with respect to target-stream alignment, or expose a single scheduler command that binds, initializes, and publishes the result.
The smallest acceptable design should:
After the ordinary write-publication contract is fixed, provide a scheduler-managed way to enqueue an external CUDA write against explicit output bindings. A narrower scheduler-managed zero/memset operation is enough for the current consumer.
Acceptance
This is not a request for a raw allocator or another untracked pointer escape hatch. The missing invariant is publication of every binding write in CubeCL's dependency graph.