I will need to spend time creating a reproducible example if required to help, but I am getting an error "Error: callr subprocess failed: could not start R, exited with non-zero status". Not sure if this is a bug with future.callr, or maybe more likely, with me or my machine, so I'm appealing for some help here... thanks in advance.
I am using an AMD Ryzen 9 5950 16-core, 32-thread machine, 32Gb RAM, and I started a process using future.callr using plan(callr, workers = 31). I would expect the whole job to run for about 9 hours, but after 1.5 hours I got the error above. I tried running a similar script on another less powerful computer, with a 2-year old Intel i5 chip with 4 cores/8 threads, that has 12Gb RAM, but using doParallel with makeCluster(7) and it ran successfully. Could my error be a RAM issue? (31 jobs divided into 32 GB RAM is a little more than 1GB RAM per callr R session, whereas 7 clusters into 12GB is 1.7 GB per cluster?)
Not sure if this is enough information to help, let me know what other info would help if not.
FYI, I tried running the same R script using parallel processing, but using doParallel, with makeCluster(31) on the more powerful machine above, and got an error "unserialize(socklist[[n]]):error reading from connection", so not sure if these two errors are related, and/or have to do with this particular machine (AMD Ryzen 9 5950), or available RAM, or what.
Any help would be greatly appreciated! I know this is a future.callr Github repo, but if anybody has any advice that would help fix my doParallel error I would welcome that too, either way.
I will need to spend time creating a reproducible example if required to help, but I am getting an error "Error: callr subprocess failed: could not start R, exited with non-zero status". Not sure if this is a bug with future.callr, or maybe more likely, with me or my machine, so I'm appealing for some help here... thanks in advance.
I am using an AMD Ryzen 9 5950 16-core, 32-thread machine, 32Gb RAM, and I started a process using future.callr using plan(callr, workers = 31). I would expect the whole job to run for about 9 hours, but after 1.5 hours I got the error above. I tried running a similar script on another less powerful computer, with a 2-year old Intel i5 chip with 4 cores/8 threads, that has 12Gb RAM, but using doParallel with makeCluster(7) and it ran successfully. Could my error be a RAM issue? (31 jobs divided into 32 GB RAM is a little more than 1GB RAM per callr R session, whereas 7 clusters into 12GB is 1.7 GB per cluster?)
Not sure if this is enough information to help, let me know what other info would help if not.
FYI, I tried running the same R script using parallel processing, but using doParallel, with makeCluster(31) on the more powerful machine above, and got an error "unserialize(socklist[[n]]):error reading from connection", so not sure if these two errors are related, and/or have to do with this particular machine (AMD Ryzen 9 5950), or available RAM, or what.
Any help would be greatly appreciated! I know this is a future.callr Github repo, but if anybody has any advice that would help fix my doParallel error I would welcome that too, either way.