Cool project gad. I'm wondering how from-the-grpund implementations like what you have would compare against existing libraries which, as you say, abstract away all that you've implemented.
Looking at your cuda kernels, and how you use them, I feel like a lot of pythons overhead should not matter when calling your NN objects, and so seems to me like the speed might be as good as NNs implemented in JAX or TF or whatever you want.
Would be interested to hear your thoughts
Cool project gad. I'm wondering how from-the-grpund implementations like what you have would compare against existing libraries which, as you say, abstract away all that you've implemented.
Looking at your cuda kernels, and how you use them, I feel like a lot of pythons overhead should not matter when calling your NN objects, and so seems to me like the speed might be as good as NNs implemented in JAX or TF or whatever you want.
Would be interested to hear your thoughts