unable to run example

[gadhvirushiraj]

I am attempting to run the ibm/sphere_ibm example on a Mac (Apple Silicon M1) within a Python 3.12 environment. While standard CFD examples like lid_driven_cavity_2d run correctly on the CPU/WARP backend, the sphere example fails during the immersed boundary preparation phase.

Error:
RuntimeError: "cuda" device requested but this build of Warp does not support CUDA

I would appreaciate any ways to fix this, or anything that I am missing.

[mehdiataei]

Hi @gadhvirushiraj you must manually choose the CPU device for the Warp kernels.

I think that error happens on:
wp.launch(kernel=voronoi_area_kernel, dim=num_faces, inputs=[faces_wp, vertices_wp, face_areas_wp, vertex_areas_wp], device="cuda")

just overwrite this in your local library to use cpu.

[gadhvirushiraj]

Hi @mehdiataei,
Thank you for the quick response. I applied your suggested fix.

But I still encounter the following error;

❯ python sphere.py
Warp DeprecationWarning: The symbol `warp.utils.ScopedTimer` will soon be removed from the public API. Use `warp.ScopedTimer` instead.
Warp DeprecationWarning: The symbol `warp.mat` will soon be removed from the public API. Use `warp.types.matrix` instead.
Warp DeprecationWarning: The symbol `warp.vec` will soon be removed from the public API. Use `warp.types.vector` instead.
Warp 1.11.1 initialized:
   CUDA not enabled in this build
   Devices:
     "cpu"      : "arm"
   Kernel cache:
     /Users/rushirajgadhvi/Library/Caches/warp/1.11.1
Sphere IBM Simulation Configuration:
  Grid size: (425, 150, 150)
  Sphere radius: 25.0
  Sphere center: [141.66666666666666, 75.0, 75.0]
  Omega: 1.9607843137254901
  Inlet velocity: 0.02
  Reynolds number: 300
  Max steps: 20000
  IBM max iterations: 4
  IBM tolerance: 1e-05
  IBM relaxation: 0.5
Module xlb.helper.ibm_helper 72b73e8 load on device 'cpu' took 710.99 ms  (compiled)
Voronoi areas calculated successfully
Sum of Voronoi areas: 7844.59619140625
Mesh area: 7844.596175060069
Frontal area (theoretical): 1963.50
registered bc RegularizedBC_283694349 with id 1
registered bc ExtrapolationOutflowBC_283666034 with id 2
registered bc FullwayBounceBackBC_296092661 with id 3
Module xlb.operator.stepper.ibm_stepper aca98f5 load on device 'cpu' took 503.15 ms  (compiled)
Module xlb.operator.equilibrium.quadratic_equilibrium dbb5841 load on device 'cpu' took 665.38 ms  (compiled)
Module xlb.operator.boundary_masker.indices_boundary_masker ab94f53 load on device 'cpu' took 595.23 ms  (compiled)
Module xlb.operator.boundary_condition.boundary_condition 345233d load on device 'cpu' took 631.56 ms  (compiled)
Module xlb.operator.stepper.nse_stepper d061d2d load on device 'cpu' took 2105.76 ms  (compiled)
Module xlb.operator.stepper.ibm_stepper 48b7afb load on device 'cpu' took 651.73 ms  (compiled)
Module xlb.operator.stepper.ibm_stepper c5e8785 load on device 'cpu' took 759.62 ms  (compiled)
Module xlb.operator.stepper.ibm_stepper 70c839d load on device 'cpu' took 902.12 ms  (compiled)
Traceback (most recent call last):
  File "/Users/rushirajgadhvi/Desktop/connectfour/sphere.py", line 257, in <module>
    f_0, f_1, lag_forces = stepper(
                           ^^^^^^^^
  File "/Users/rushirajgadhvi/miniforge3/envs/connectfour/lib/python3.12/site-packages/xlb/operator/operator.py", line 84, in __call__
    raise Exception(f"Error captured for backend with key {key} for operator {self.__class__.__name__}: {error}\n {traceback_str}")
Exception: Error captured for backend with key ('IBMStepper', <ComputeBackend.WARP: 2>, '(self, f_0, f_1, s_lagr_vertices_wp, lagr_solid_vertex_areas_wp, lagr_solid_velocities_wp, bc_mask, missing_mask, omega, timestep)') for operator IBMStepper: Device cpu is not a CUDA device
 Traceback (most recent call last):
  File "/Users/rushirajgadhvi/miniforge3/envs/connectfour/lib/python3.12/site-packages/xlb/operator/operator.py", line 74, in __call__
    result = backend_method(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rushirajgadhvi/miniforge3/envs/connectfour/lib/python3.12/site-packages/xlb/operator/stepper/ibm_stepper.py", line 415, in warp_implementation
    wp.synchronize_stream()
  File "/Users/rushirajgadhvi/miniforge3/envs/connectfour/lib/python3.12/site-packages/warp/_src/context.py", line 7103, in synchronize_stream
    stream = runtime.get_device(stream_or_device).stream
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rushirajgadhvi/miniforge3/envs/connectfour/lib/python3.12/site-packages/warp/_src/context.py", line 3581, in stream
    raise RuntimeError(f"Device {self} is not a CUDA device")
RuntimeError: Device cpu is not a CUDA device

[mehdiataei]

Can you comment out wp.synchronize_stream()? since stream is None for you. I will ask our friends at NVIDIA to fix this in the future at Warp-level.

[mehdiataei]

Hi @gadhvirushiraj The fix in Warp is now merged. Can you check if you still see the issue without commenting that line? Thanks.

[hsalehipour]

Is this issue still relevant @gadhvirushiraj ?

[gadhvirushiraj]

It is solved! Thanks @mehdiataei