Implement the "auto" mode for the langrunner to automatically choose the right infrastructure setup.
Hint for the selection could be based on,
- Hardware available for the access
- Latency vs cost optimization
- Whether to keep the infra up for continuous experimentation or tear down immediately after the current run
- Prioritize local cluster first before bursting into cloud for more resources.
Implement the "auto" mode for the langrunner to automatically choose the right infrastructure setup.
Hint for the selection could be based on,