Our algorithm approximately wants 5 minutes to be executed, due to time consuming calculations and heavy load of data I use multiple futures to execute the code in parallel.
The plan strategy I use is plan(multisession). We have a plumber API endpoint and when we initiate concurrent calls to the API and hence to the algorithm I receive the following error :
FutureError: Cannot resolve MultisessionFuture (), because the connection to the worker is corrupt: Connection (connection: index=4, description="<-localhost:11803", class="sockconn", mode="a+b", text="binary", opened="opened", can read="yes", can write="yes", id=3711, raw_id="<pointer: 0xe7f>") is no longer valid.
After a quick search I found out that R assigns the same workers to the new API call, how can we avoid the worker corruption
Our algorithm approximately wants 5 minutes to be executed, due to time consuming calculations and heavy load of data I use multiple futures to execute the code in parallel.
The plan strategy I use is plan(multisession). We have a plumber API endpoint and when we initiate concurrent calls to the API and hence to the algorithm I receive the following error :
FutureError: Cannot resolve MultisessionFuture (), because the connection to the worker is corrupt: Connection (connection: index=4, description="<-localhost:11803", class="sockconn", mode="a+b", text="binary", opened="opened", can read="yes", can write="yes", id=3711, raw_id="<pointer: 0xe7f>") is no longer valid.
After a quick search I found out that R assigns the same workers to the new API call, how can we avoid the worker corruption