From b8ab4e23949822461d5d9dbb6bec4899b7eff26a Mon Sep 17 00:00:00 2001 From: Minh Ha-Duong Date: Tue, 23 Jun 2026 08:33:19 +0200 Subject: [PATCH] docs(deploy): document secret/API-key rotation procedure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A key rotation on 2026-06-22 took an hour because `docker restart` was used to pick up a swapped OPENAI_API_KEY. `docker restart` re-reads nothing — it bounces the process with the environment baked in at container creation, so the old (revoked) key kept being served. Only `docker compose up` (via `up.sh --remote`) re-reads env_file. Document the correct procedure in HOWTO-deploy.md, plus two traps that bit during the same incident: a key pasted as a bare value with no `VAR=` prefix, and a repeated query passing on a cached embedding while the new key was actually broken. Co-Authored-By: Claude Opus 4.8 --- deploy/HOWTO-deploy.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/deploy/HOWTO-deploy.md b/deploy/HOWTO-deploy.md index 63e1ff5..15034c8 100644 --- a/deploy/HOWTO-deploy.md +++ b/deploy/HOWTO-deploy.md @@ -49,6 +49,28 @@ this will do a remote git pull, restart and validate. ./validate.sh --remote ``` +## Rotating secrets and API keys + +Edit the relevant file in `secrets/env/` (e.g. `r2r-full.env`), push it, then +**recreate** the affected container: + +```bash +./push_secrets.sh +./up.sh --remote # recreates containers, re-reading env_file +``` + +- Write each secret as a real assignment: `OPENAI_API_KEY=sk-proj-...`. A bare + value with no `VAR=` prefix leaves the variable undefined. +- **`docker restart` does NOT pick up a changed key.** It bounces the process + with the environment baked in at container *creation*. Only `up.sh --remote` + (i.e. `docker compose up -d --force-recreate `) re-reads `env_file`. +- Verify the swap with a **novel** query, not a repeated one: R2R caches query + embeddings, so a repeated query can return 200 from cache and hide a bad key. + Confirm the loaded key without printing it: + ```bash + docker exec cidir2r-r2r-1 sh -c 'printf %s "$OPENAI_API_KEY" | tail -c 4' + ``` + ## Backuping and updating the corpus Data 1. Create a snapshot locally: