Skip to content

Publish CUDA binding write cursors for kernel and external writes #16

Description

@Ryo-wtnb11

Problem

At tensor4all/cubecl@1c88bb6, a CUDA binding's cursor is recorded when its managed allocation is bound, but scheduler-managed kernel writes do not publish a newer cursor/version for their output bindings. Raw external CUDA writes have the same gap.

Relevant flow:

  • ComputeClient::empty applies the layout policy and submits ComputeServer::initialize_memory;
  • allocation Command::bind records the current stream cursor on the managed-memory slice;
  • launch_checked resolves every binding and materializes its resource, but Command::resource / MemoryManagement::get_resource only fetch storage and do not update the binding cursor;
  • CudaServer::raw_stream resolves the current stream with no bindings;
  • MultiStream decides whether a target stream needs an event by comparing handle_cursor(binding) with last_synced[origin].

This permits the following race even for an ordinary fresh kernel output:

  1. thread A allocates output on origin stream at binding cursor k;
  2. before A submits the initialization/output kernel, thread B uses an unrelated origin-stream binding and records B.last_synced[A] >= k;
  3. A submits the kernel at a later stream cursor, but the output binding cursor remains k;
  4. after A returns the output, B first uses it, sees cursor k already synchronized, and skips an event covering A's kernel.

The output does not need to escape before its producer returns; the unrelated origin binding is sufficient to establish the stale target-side synchronization. Separate submit calls can interleave on the shared server channel.

An external cudaMemsetAsync performed after get_resource / raw_stream has the same defect, plus it is not represented by a CubeCL launch at all. Keeping the allocation handle alive provides lifetime safety but not dependency publication. Holding with_server only after ordinary allocation also leaves the old binding cursor.

This was found while Tenferro replaced a host-zero H2D upload with device initialization:

Requested core contract

Every scheduler-managed operation that writes a binding must publish a binding version/cursor covering that write. Allocation plus immediate initialization must also be atomic with respect to target-stream alignment, or expose a single scheduler command that binds, initializes, and publishes the result.

The smallest acceptable design should:

  • distinguish read and write bindings at scheduler admission;
  • use the client's normal layout/allocation policy;
  • enqueue work and publish the written binding cursor/version under the same server ordering boundary;
  • make later cross-stream use wait for an event that includes the latest write, even if the target previously synchronized to the origin through an unrelated binding;
  • preserve asynchronous execution, error propagation, resource lifetime, and existing same-stream behavior;
  • avoid a global runtime lock at the public backend layer.

After the ordinary write-publication contract is fixed, provide a scheduler-managed way to enqueue an external CUDA write against explicit output bindings. A narrower scheduler-managed zero/memset operation is enough for the current consumer.

Acceptance

  • a deterministic two-thread/multi-stream test forces target synchronization between fresh allocation and producer launch, then proves target use waits for the produced value;
  • repeat the interleaving test for an admitted external write or scheduler-managed memset;
  • same-stream and cross-stream zero initialization produce exact zero;
  • no host allocation, H2D/D2H transfer, or global public-backend lock;
  • existing layout policy and error/stream-health behavior remain intact.

This is not a request for a raw allocator or another untracked pointer escape hatch. The missing invariant is publication of every binding write in CubeCL's dependency graph.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions