Skip to content

security: rate-limit the auth and engine-facing endpoints (#63) - #101

Merged
richardmhope merged 1 commit into
mainfrom
m4-rate-limiting
Jul 31, 2026
Merged

security: rate-limit the auth and engine-facing endpoints (#63)#101
richardmhope merged 1 commit into
mainfrom
m4-rate-limiting

Conversation

@richardmhope

Copy link
Copy Markdown
Contributor

Ceilings on what an unauthenticated caller can reach. These are not the last line of defence —
authentication is — so they are sized to stop floods rather than to meter usage.

Bucket Keyed on Default Why
/auth/login, /auth/callback client address 20/min Unlimited, these make the API hammer the IdP on someone else's behalf; the callback is the cheapest way to make it verify a JWT against a JWKS.
Rejected engine tokens client address 30/min Bounds credential stuffing and puts it in the metrics.
Accepted engine requests engine id 600/min Per engine, not per address.

The three decisions worth reviewing

Each alternative looks reasonable until you consider the failure mode:

  1. It fails open. An unreachable counter store allows the request and logs a warning. A limiter
    that locks every operator out of the console when Redis hiccups has done more damage than the
    traffic it was defending against. There is a test for this specifically.
  2. Engines are charged per engine, not per address. A fleet behind one NAT would otherwise share
    a budget and throttle itself — --scale engine=N would stop meaning anything. Tested by
    exhausting one engine's budget and showing another is unaffected from the same address.
  3. Only rejected token presentations are charged to an address. That is what lets the
    credential-stuffing bucket be small without any risk to a healthy fleet, which never touches it.

X-Forwarded-For

Ignored unless ICEBERG_TRUSTED_PROXY_HOPS says how many proxies sit in front. The header is
caller-controlled: trusting it by default would let an attacker take a fresh identity per request,
which is the same as having no limit. Behind a load balancer this must be set, or every request
is charged to the balancer — .env.example and docs/security.md both say so in as many words.

Mechanics

Fixed-window counters in Redis (INCR + EXPIRE), so a limit means the same thing across replicas;
in-memory is the fallback and is a real limit, just a per-replica one. Refusals answer 429 with
Retry-After; allowed requests carry advisory RateLimit-Limit/-Remaining/-Reset, attached in
middleware so no rate-limited route can forget them.

21 tests: window behaviour, key isolation, fail-open, Redis key expiry, all five
X-Forwarded-For cases, enforce semantics, and end-to-end refusal through the app on login,
callback, bad engine tokens, and an over-budget engine.

make lint, make type and the full suite pass.

Note: touches the same regions of config.py, .env.example and docker-compose.yml as
#100, so whichever merges second will need a trivial rebase. Happy to do that.

Closes #63

🤖 Generated with Claude Code

Ceilings on what an unauthenticated caller can reach. Not the last line of
defence — authentication is — so these are sized to stop floods, not to meter
usage.

| Bucket | Keyed on | Default |
|---|---|---|
| `/auth/login`, `/auth/callback` | client address | 20/min |
| Rejected engine tokens | client address | 30/min |
| Accepted engine requests | **engine id** | 600/min |

Three decisions are the substance here, because the alternatives all look
reasonable until you think about the failure mode:

* **It fails open.** An unreachable counter store allows the request and logs a
  warning. A limiter that locks every operator out of the console when Redis
  hiccups has done more damage than the traffic it was defending against.
* **Engines are charged per engine, not per address.** A fleet behind one NAT
  would otherwise share a budget and throttle itself, and `--scale engine=N`
  would stop meaning anything.
* **Only *rejected* token presentations are charged to an address.** That is what
  lets the credential-stuffing bucket be small without any risk to a healthy
  fleet — a working engine never touches it.

`X-Forwarded-For` is ignored unless `ICEBERG_TRUSTED_PROXY_HOPS` says how many
proxies are in front. The header is caller-controlled, so trusting it by default
would let an attacker take a fresh identity per request — the same as having no
limit. Behind a load balancer this must be set, which the docs say in as many
words.

Counters are fixed-window in Redis, so a limit means the same thing across
replicas; in-memory is the fallback and is a real limit, just a per-replica one.
Refusals answer 429 with `Retry-After`; allowed requests carry advisory
`RateLimit-*` headers, attached in middleware so no route can forget them.

Closes #63

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@richardmhope
richardmhope merged commit 1afcaf1 into main Jul 31, 2026
3 checks passed
@richardmhope
richardmhope deleted the m4-rate-limiting branch July 31, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rate limiting on auth & engine endpoints

1 participant