Hi @ppwwyyxx,
I have been trying to reproduce your method of multi-gpu RAM sharing within PyTorch Lightning.
I can't get it to share between GPU workers though and thought I might reach out and ask whether you have any clue.
For this reason I uploaded my attempt here: https://github.com/luzuku/ram-share-lightning
I replaced the dataset with random numbers and I am only looking at total_pss and it sadly doubles when doubling the number of GPUs. Maybe I overlooked something?
I would really appreciate it if you could find some time. You could also include a working Lightning implementation in your repository for anybody else interested.
Hi @ppwwyyxx,
I have been trying to reproduce your method of multi-gpu RAM sharing within PyTorch Lightning.
I can't get it to share between GPU workers though and thought I might reach out and ask whether you have any clue.
For this reason I uploaded my attempt here: https://github.com/luzuku/ram-share-lightning
I replaced the dataset with random numbers and I am only looking at
total_pssand it sadly doubles when doubling the number of GPUs. Maybe I overlooked something?I would really appreciate it if you could find some time. You could also include a working Lightning implementation in your repository for anybody else interested.