Is your feature request related to a problem? Please describe.
My group is working on a PDE-based modeling framework where the models typically include forcing from external data, for example wind fields or emissions. The data typically includes three spatial dimensions and one time dimension. In order to ingest the forcing data into the model, we use interpolation. In different situations, we want to be able to either 1) interpolate in all four dimensions, or 2) use data that is pre-gridded in the 3 spatial dimensions and interpolate in the time dimension.
The problem is that it is typically a very large amount of data, too much to fit into memory all at once.
Describe the solution you’d like
Because these are physical simulations, we are always moving monotonically forward (or occasionally backward) in time. Therefore, we can use a "streaming interpolator" where only ~2 time points (for all of the spatial points) are held in memory at any given time, and there is some sort of mechanism to update which time points are held in memory based on the state of the integrator.
Describe alternatives you’ve considered
We have developed a version of this system, with some of the core code found here. However, our version doesn't currently work with GPUs and as it is based on Interpolations.jl, it does not allow for different types of interpolation to be used for different dimensions. (It looks like #31 and #32 will add that functionality for this package). I met with @ChrisRackauckas and he suggested that something like this could be good to add here.
Additional context
Our models are developed using ModelingToolkit, so good integration with MTK would be a bonus.
Is your feature request related to a problem? Please describe.
My group is working on a PDE-based modeling framework where the models typically include forcing from external data, for example wind fields or emissions. The data typically includes three spatial dimensions and one time dimension. In order to ingest the forcing data into the model, we use interpolation. In different situations, we want to be able to either 1) interpolate in all four dimensions, or 2) use data that is pre-gridded in the 3 spatial dimensions and interpolate in the time dimension.
The problem is that it is typically a very large amount of data, too much to fit into memory all at once.
Describe the solution you’d like
Because these are physical simulations, we are always moving monotonically forward (or occasionally backward) in time. Therefore, we can use a "streaming interpolator" where only ~2 time points (for all of the spatial points) are held in memory at any given time, and there is some sort of mechanism to update which time points are held in memory based on the state of the integrator.
Describe alternatives you’ve considered
We have developed a version of this system, with some of the core code found here. However, our version doesn't currently work with GPUs and as it is based on Interpolations.jl, it does not allow for different types of interpolation to be used for different dimensions. (It looks like #31 and #32 will add that functionality for this package). I met with @ChrisRackauckas and he suggested that something like this could be good to add here.
Additional context
Our models are developed using ModelingToolkit, so good integration with MTK would be a bonus.