@tomwhite and I work on Cubed, a framework for running distributed multidimensional array computations at scale atop serverless architectures. We use it for genetics and geoscience workloads.
Cubed can run on all sorts of DAG frameworks, including Dask, Ray, Apache-Beam and Modal, but really only requires an extremely simple parallel map primitive. We use lithops for that now, but really even lithops is more complex than what we need, and Burla's remote_parallel_map is essentially the minimum primitive required.
It could be fun to try to make a Cubed executor for Burla and run some scientific workloads...
@tomwhite and I work on Cubed, a framework for running distributed multidimensional array computations at scale atop serverless architectures. We use it for genetics and geoscience workloads.
Cubed can run on all sorts of DAG frameworks, including Dask, Ray, Apache-Beam and Modal, but really only requires an extremely simple parallel map primitive. We use lithops for that now, but really even lithops is more complex than what we need, and Burla's
remote_parallel_mapis essentially the minimum primitive required.It could be fun to try to make a Cubed executor for Burla and run some scientific workloads...