feat(deployment,website): Split website organism configs into separate ConfigMaps

[theosanderson-agent]

Summary

Splits the website organism configuration out of the monolithic loculus-website-config ConfigMap into one ConfigMap per organism. The website now reads website_config.json plus optional organisms/*.json files and merges them at startup.

This keeps generated website organism config data out of a single Kubernetes object so instances with many or large organisms stay comfortably below the ConfigMap/Secret object size limit.

Implementation

• Render loculus-website-config with global website config only and one loculus-web-org-config-* ConfigMap per organism.
• Mount the website config through a projected volume so organisms/*.json lands under the same CONFIG_DIR.
• Update deploy.py config to generate split local test config files.
• Document the split config layout and ignore generated split config fixtures.

Size check

Using current Pathoplexus values with the Bundibugyo organism branch:

• old monolithic loculus-website-config: 976,493 bytes of website_config.json
• new global loculus-website-config: 2,568 bytes of website_config.json in the main render
• largest split organism ConfigMap: 81,638 bytes (loculus-web-org-config-mpox)
• largest ConfigMap/Secret data block across demo/main/production/staging: ~123.2 KB (loculus-backend-config), about 11.75% of 1 MiB

The reconstructed split website config matches the old monolithic website config for the Pathoplexus main render.

🚀 Preview: Add preview label to enable

[claude]

Claude finished @theosanderson's task in 3m 35s —— View job

---

Code Review

• Gather context and understand the PR
• Review changed files
• Post inline comments
• Summarize findings

---

Overall this is a clean, well-motivated change. The size numbers in the PR description make the need obvious, the Helm splitting logic is straightforward, and the website-side merge is simple and well-tested. A few small things to tighten up:

Inline comments posted:

1. config.ts:77 — fs.existsSync doesn't check that organisms is a directory; if it's accidentally a file, readdirSync will throw with a confusing error. Suggest a statSync guard or an ENOENT-catching try/catch.

2. config.ts:85 — the split-config files silently override any same-key organism in the base website_config.json. This is correct given the Helm template always strips organisms from the base config, but a test case or comment for the override behaviour would make it explicit.

3. deploy.py:520 — config["organisms"] = {} is redundant (the rendered website_config.json already has organisms: {}). A comment explaining this coupling to the Helm template would help future readers.

4. loculus-website.yaml:57 — {{ range }} / {{ end }} without - whitespace trimming injects blank lines between projected configMap entries. Harmless, but inconsistent with the rest of the chart.

Nothing blocking; the core approach is sound and the test for split-config merging in config.spec.ts is a good addition.

[theosanderson]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 962bffe521

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[theosanderson]

@corneliusroemer @maverbiest @tombch would one of you have a chance to review? I think this is quite important as I think it could currently be blocking adding any more organisms

[maverbiest]

@theosanderson I might have some time over the weekend/early next week so will do my best to review then

[tombch]

Feel free to skip as this is not an issue but I guess these could be moved into the websites .gitignore?

[tombch]

Is there a reason why the truncation limits to 63 characters, rather than the limit for config map names (253 characters)?

[tombch]

Ah, I have had a look around and organism names are used in a bunch of labels which limit to 63 characters, so I guess this is already a thing that needs restricting to lower than 253 chars (as mentioned in the resolved comment, this is already an issue)

[tombch]

Tested it locally as well, looks good to me!