With the current timer/periodical bundle building behaviour of the control plane, we are currently running a single replica in our deployment (Kubernetes) to avoid duplicate builds from multiple replicas.
We have two questions:
- Are there other problems beyond duplicate builds if we run multiple replicas?
In the worst case we could tolerate duplicate builds. ie. bundles would be identical and just overwrite each other in object storage (or rather from what i observed won't be uploaded if they are the same). Before we go down that path, we wanted to check if there are other failure modes we're not seeing.
We're also aware of #104 which would indirectly address this by making builds explicit rather than periodic. This would allow scaling the control plane and letting an external cronjob trigger builds, with the work naturally distributed across pods.
- Is there a way to run the control plane with bundle building disabled to serve the control plane API?
The main pain point with a single replica is availability of the control plane APIs during restarts and deployments. Ideally we'd like a way to run the HTTP server without the bundle build loop active, so we could scale API replicas independently from the single worker responsible for builds.
With the current timer/periodical bundle building behaviour of the control plane, we are currently running a single replica in our deployment (Kubernetes) to avoid duplicate builds from multiple replicas.
We have two questions:
In the worst case we could tolerate duplicate builds. ie. bundles would be identical and just overwrite each other in object storage (or rather from what i observed won't be uploaded if they are the same). Before we go down that path, we wanted to check if there are other failure modes we're not seeing.
We're also aware of #104 which would indirectly address this by making builds explicit rather than periodic. This would allow scaling the control plane and letting an external cronjob trigger builds, with the work naturally distributed across pods.
The main pain point with a single replica is availability of the control plane APIs during restarts and deployments. Ideally we'd like a way to run the HTTP server without the bundle build loop active, so we could scale API replicas independently from the single worker responsible for builds.