[Proof of concept] DB-based configuration management

[chaoran-chen]

Currently, an instance of Loculus is configured almost entirely through a single values.yaml file. It holds both technical settings (image versions, resource limits, database URLs) and domain configuration (list of organisms, their metadata fields, reference genomes, branding, link-outs, data-use terms, and so on). This works, but it places day-to-day configuration in the hands of whoever can edit the deployment repository and run a release. Adding an organism or correcting a field label means editing YAML, opening a pull request, and waiting for a redeploy.

The problems this POC is responding to

• Changing configuration is harder than it should be for administrators. It currently requires familiarity with Helm, Kubernetes, Git, and the deployment pipeline. It is difficult for a domain expert without much technical expertise.

• There is very little validation, and mistakes are costly. values.yaml is large and only loosely checked. A typo, a misindented block, or a field that references something that does not exist can break templating or, worse, produce an instance that deploys but behaves incorrectly. Errors tend to surface late — at deploy or run time — and a single edit can take an instance down. There is no draft, no review of the content of a change, and no straightforward way to see who changed what, or to roll back.

• The approach is approaching hard technical limits. A Helm release is capped at roughly 1 MB, and instances with many or large organisms are not far from that ceiling (Alternatives to Helm #5952). We have already had to work around it: feat(deployment,website): Split website organism configs into separate ConfigMaps #6447 split the website organism configuration into one Kubernetes object per organism specifically to stay below the size limit. We also needed to introduce a custom approach with [[URL:...]] to address the challenge of configuring reference genomes.

The proposed direction

This proof-of-concept separates the two kinds of configuration that are mixed today. Technical configs stay in Helm: deployments, DB credentials, etc. Domain configuration moves into a database and is edited through an administrator panel

Detailed architecture documentation is under https://github.com/loculus-project/loculus/tree/demo/config-experiments/config-architecture.

The username and password for the admin are: loculus_administrator

🚀 Preview: Add preview label to enable

[claude]

This PR may be related to: #5952 (Alternatives to Helm — this PR moves config from Helm values into the database), #3603 (lineage system definitions should go under organism — this PR introduces per-organism config management in the admin UI)