I just realised that the current hook that we use to insert graphchain into dask is delayed_optimize, which only actually applies for "raw" Delayed instances, but doesn't actually apply to e.g. the DataFrame API, which is a very popular part of dask. As described here, these types have their own optimisation hooks that we would need to implement. I will look in to writing one for the DataFrame API. I think we should be able to integrate it fairly easily into the current codebase.
I just realised that the current hook that we use to insert
graphchainintodaskisdelayed_optimize, which only actually applies for "raw"Delayedinstances, but doesn't actually apply to e.g. theDataFrameAPI, which is a very popular part ofdask. As described here, these types have their own optimisation hooks that we would need to implement. I will look in to writing one for theDataFrameAPI. I think we should be able to integrate it fairly easily into the current codebase.array_optimizedataframe_optimizebag_optimize