diff --git a/CHANGELOG.md b/CHANGELOG.md index 197797d..ed1e700 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,14 @@ Gatekeeper is pre-1.0. The configuration schema and credential source interface ## v0.17.2 — 2026-07-15 +### Added + +- **Eight new deployment and how-to guides** — the docs audit found the guide catalog stopped at reference material for several features with real setup friction. New task-oriented walkthroughs, each verified against code (all YAML parse-checked through `gatekeeper.LoadConfig`, Go snippets compile-checked, and the container guide exercised live against the published image): deploying behind a TCP load balancer with PROXY protocol (GCP `proxyHeader: PROXY_V1` backend config, gatekeeper-first rollout ordering, client-IP-forgery caveats); running the `ghcr.io/majorcontext/gatekeeper` container image (config/CA mounting, client wiring, and why in-container Docker healthchecks can't work on the distroless image — it has no shell); the Postgres data plane with Neon end-to-end (account- vs project-scoped API keys, the libpq `host=`/`hostaddr=` split for local testing without DNS); host command credentials with worked 1Password and `pass` examples; GCP service account tokens (including the precise key-rotation semantics: only the Google token endpoint rejecting a JWT drops the cached key, and only in Secret Manager mode — a downstream 401 rides out the TTL); MCP relay setup (honest about MCP having no `gatekeeper.yaml` surface — it's Go-library-only via `RunContextData`); the full credential lifecycle (startup fetch, 75%-of-TTL refresh whose backoff jitter applies after the 60s cap so waits can reach ~75s, token-exchange's 1-minute cache, 401/403 eviction); and the three Keep policy scopes (`http`, `mcp-`, `llm-gateway`), including that `mcp-*` denials set no `X-Moat-Blocked` header unlike the other two scopes + +### Changed + +- **Docs corpus now complies with its own style guide** — a compliance sweep nobody had run found `concepts/` and `guides/` used Title Case headings throughout while `getting-started/` and `reference/` were sentence case per `docs/STYLE-GUIDE.md`; 106 headings across 18 files converted, plus scattered terminology and abbreviation-order fixes. Systemic gaps flagged for a policy call rather than fixed: first-use abbreviation expansion is ignored corpus-wide, guide files use `--` where the rest of the corpus uses em dashes (new guides written to the em-dash convention), and frontmatter title casing is uncodified + ### Fixed - **Gatekeeper's own OTel diagnostics no longer feed back into the failing OTel log-export pipeline** — `configureLogging` fans every slog record out to `otelslog.NewHandler("gatekeeper")` unconditionally: the bridge's `Enabled` defers entirely to the OTel logger, not the console handler's configured level, so nothing before v0.17.0's DEBUG demotion (and nothing since) kept the bridge from seeing a record just because it was noisy. That meant `logOTelError` ([#47](https://github.com/majorcontext/gatekeeper/pull/47))'s own DEBUG record on a failed export was itself enqueued right back into the OTel log-export pipeline that had just failed: failed export → DEBUG diagnostic → diagnostic fanned out to the bridge → re-enqueued for the next export → that export fails too, now carrying the diagnostic → another diagnostic logged → re-enqueued again, and so on for as long as the collector stays unreachable. Bounded — one record per export attempt, and the batch queue drops on overflow rather than growing without limit — but a genuine feedback loop, not merely repeated independent failures. `gatekeeper.OTelDiagnosticKey` is a new exported marker attribute; `logOTelError` tags its DEBUG record with it, and `configureLogging`'s otelslog bridge handler is now wrapped in a filter that drops any record carrying the marker before forwarding — the console/file handler is unaffected and still logs every record, marked or not, at its configured level. Diagnostics about the OTel pipeline itself are now visible locally but never re-enter the pipeline they're reporting on ([#48](https://github.com/majorcontext/gatekeeper/issues/48)) diff --git a/docs/README.md b/docs/README.md index 44826da..54e6172 100644 --- a/docs/README.md +++ b/docs/README.md @@ -32,6 +32,14 @@ - [OpenTelemetry](./content/guides/08-opentelemetry.md) — Distributed tracing, metrics, and logs - [Go Library](./content/guides/09-go-library.md) — Embed the proxy engine in a custom Go application - [WebSocket Support](./content/guides/10-websockets.md) — WebSocket upgrades through TLS interception +- [Deploying Behind a TCP Load Balancer](./content/guides/11-load-balancer-proxy-protocol.md) — recover real client IPs with PROXY protocol behind a TCP-terminating load balancer +- [Running the Container Image](./content/guides/12-docker-deployment.md) — pull and run the published container image with config and CA material mounted +- [Postgres Data Plane with Neon](./content/guides/13-postgres-neon.md) — run the Postgres listener against a Neon project with run-token authentication +- [Host Command Credentials](./content/guides/14-process-credentials.md) — inject credentials from any secret manager that has a CLI via the process source +- [GCP Service Account Tokens](./content/guides/15-gcp-service-account.md) — mint and refresh GCP OAuth2 access tokens from a service account key +- [MCP Relay Setup](./content/guides/16-mcp-relay.md) — configure the MCP relay end-to-end using the Go library +- [Credential Caching, Refresh, and Invalidation](./content/guides/17-credential-lifecycle.md) — the full credential lifecycle from startup fetch through 401/403-triggered eviction +- [Keep Policy Scopes](./content/guides/18-keep-policy-scopes.md) — the http, mcp-, and llm-gateway policy scopes and their denial logs ### Reference diff --git a/docs/content/concepts/01-tls-interception.md b/docs/content/concepts/01-tls-interception.md index c912c3d..6ab9462 100644 --- a/docs/content/concepts/01-tls-interception.md +++ b/docs/content/concepts/01-tls-interception.md @@ -4,17 +4,17 @@ description: "How Gatekeeper terminates TLS connections, generates per-host cert keywords: ["gatekeeper", "TLS interception", "MITM proxy", "certificate generation"] --- -# TLS Interception +# TLS interception Gatekeeper is a TLS-intercepting proxy. It terminates the client's TLS connection, reads the plaintext HTTP request, injects credentials, and forwards the request to the real server over a separate TLS connection. This is a man-in-the-middle architecture — the client must trust gatekeeper's CA certificate. This page explains why TLS interception is necessary and how the certificate chain works. -## Why MITM Is Necessary +## Why MITM is necessary HTTP proxies see CONNECT tunnels as opaque byte streams. Without interception, the proxy knows the destination host but cannot read or modify the encrypted HTTP request inside the tunnel. Credential injection requires access to the plaintext request headers — so gatekeeper must terminate the client's TLS, read the request, inject headers, and re-encrypt for the upstream server. -## The CONNECT Flow +## The CONNECT flow When a client sends an HTTPS request through gatekeeper, the flow has five stages: @@ -37,7 +37,7 @@ Client Gatekeeper api.github.com |<-- 200 OK ---------------| | ``` -## Two Separate TLS Connections +## Two separate TLS connections The proxy maintains two independent TLS sessions per intercepted request: @@ -48,11 +48,11 @@ The proxy maintains two independent TLS sessions per intercepted request: These connections use independent keys and cipher suites. The client never sees the origin server's certificate — it only sees gatekeeper's generated certificate. -## Per-Host Certificate Generation +## Per-host certificate generation When gatekeeper intercepts a CONNECT tunnel for a host, `CA.GenerateCert` creates a certificate on the fly: -- The certificate's `CommonName` and SAN (Subject Alternative Name) match the target host. +- The certificate's `CommonName` and Subject Alternative Name (SAN) match the target host. - IP addresses are added as IP SANs; hostnames as DNS SANs. - Each certificate is signed by gatekeeper's CA private key. - Generated certificates are cached in memory by hostname to avoid repeated key generation. @@ -60,19 +60,19 @@ When gatekeeper intercepts a CONNECT tunnel for a host, `CA.GenerateCert` create The CA supports RSA, EC, and Ed25519 private keys via PKCS1, PKCS8, and SEC 1 formats. -## ALPN Negotiation +## ALPN negotiation The client-facing TLS handshake advertises `h2` before `http/1.1` in its ALPN `NextProtos` list, so gatekeeper prefers HTTP/2 with clients that support it and falls back to HTTP/1.1 otherwise. The negotiated protocol determines which transport gatekeeper uses to reach the upstream server: when the client negotiated `h2`, gatekeeper forwards over an `http2.Transport`, since an h2 request cannot be round-tripped through an HTTP/1.1 transport without framing errors; otherwise it forwards over a standard `http.Transport`. This matters for gRPC clients, which require h2 end-to-end — gatekeeper's `http2.Transport` never falls back to HTTP/1.1, so if the upstream only speaks HTTP/1.1, a connection from an h2 client to that upstream fails. WebSocket upgrades follow a related but separate path through TLS interception — see [WebSockets](../guides/10-websockets.md). -## Why the Client Must Trust the CA +## Why the client must trust the CA The dynamically generated certificates are not signed by a public CA. Clients reject them unless they explicitly trust gatekeeper's CA certificate. In container environments, the CA certificate is mounted into the container's trust store (e.g., `/etc/ssl/certs/`). Without this, every HTTPS request through the proxy fails with a certificate verification error. > **Note:** Applications with certificate pinning will fail even with the CA trusted. This is expected — interception requires replacing the origin certificate. -## Non-CONNECT Relay Path +## Non-CONNECT relay path Plain HTTP requests (no TLS) bypass the interception flow entirely. Gatekeeper reads the request directly, injects credentials, and forwards it using a standard `http.Transport`. No certificate generation occurs. diff --git a/docs/content/concepts/02-credential-injection.md b/docs/content/concepts/02-credential-injection.md index f38d063..e0c746f 100644 --- a/docs/content/concepts/02-credential-injection.md +++ b/docs/content/concepts/02-credential-injection.md @@ -4,11 +4,11 @@ description: "How Gatekeeper matches hostnames, injects authentication headers, keywords: ["gatekeeper", "credential injection", "host matching", "authorization headers"] --- -# Credential Injection +# Credential injection Gatekeeper injects authentication headers into proxied HTTP requests based on hostname matching. Clients never handle raw credentials — they send requests through the proxy, which resolves the correct credential and sets the appropriate header before forwarding to the upstream server. -## Host Matching +## Host matching Each credential is configured with a `host` pattern. When gatekeeper intercepts a request, it looks up credentials for the target hostname, stripping any port from the request host unconditionally before comparing — there is no notion of a default or matched port in credential lookup. @@ -23,7 +23,7 @@ A `host` pattern cannot contain a port — `credentials[].host` is validated and Host comparison is case-insensitive. `API.GitHub.com` matches `api.github.com`. -## Header Injection +## Header injection The default injection header is `Authorization`. Override it with the `header` field: @@ -42,7 +42,7 @@ Gatekeeper injects credentials in two modes: 2. **Auto-injection.** If the client sends no matching header, gatekeeper injects the credential unconditionally. When multiple credentials share the same header name for a host, the `claude` grant is deprioritized — it is only injected when the client explicitly sends a placeholder. -## Grant Names +## Grant names The `grant` field is an optional label that identifies a credential for logging and MCP relay matching. Grant names appear in canonical log lines and OpenTelemetry span attributes. @@ -57,7 +57,7 @@ credentials: Built-in grant names (`github`, `anthropic`, `openai`, `aws`, and others) map to predefined host patterns. These mappings are used by network policy to auto-allow hosts for configured grants. -## Prefix and Format +## Prefix and format For `Authorization` headers, gatekeeper ensures the value includes an auth scheme prefix. The behavior depends on configuration: @@ -77,7 +77,7 @@ credentials: var: GITHUB_TOKEN ``` -## Multiple Credentials Per Host +## Multiple credentials per host A host can have multiple credential entries with different header names. All matching credentials are injected: @@ -98,6 +98,6 @@ credentials: When multiple credentials share the same header name, placeholder replacement takes priority. If no placeholder matched, auto-injection picks the non-`claude` grant to avoid overriding explicit OAuth flows. -## Credential Stripping +## Credential stripping Gatekeeper removes `Proxy-Authorization` and `Proxy-Connection` headers from all forwarded requests. These are hop-by-hop headers used between the client and the proxy — they must never reach the upstream server. Injected credential headers (like `Authorization`) are also redacted in log output to prevent credential leakage. diff --git a/docs/content/concepts/03-credential-sources.md b/docs/content/concepts/03-credential-sources.md index ddebbca..3843613 100644 --- a/docs/content/concepts/03-credential-sources.md +++ b/docs/content/concepts/03-credential-sources.md @@ -4,11 +4,11 @@ description: "How Gatekeeper resolves credentials from pluggable backends includ keywords: ["gatekeeper", "credential sources", "background refresh", "credential resolver"] --- -# Credential Sources +# Credential sources Gatekeeper resolves credentials from pluggable backends called **credential sources**. Each source implements a single method — `Fetch` — that returns a credential value. Sources range from simple (read an environment variable) to complex (exchange tokens with an external STS). -## The Source Interface +## The source interface All credential sources implement `CredentialSource`: @@ -21,7 +21,7 @@ type CredentialSource interface { `Fetch` retrieves the current credential value. It accepts a context for cancellation and timeout — gatekeeper enforces a 10-second timeout on all startup fetches. `Type` returns a string identifier for logging (e.g., `"env"`, `"aws-secretsmanager"`). -## Static vs Dynamic Sources +## Static vs dynamic sources **Static sources** return the same value on every call. They are fetched once at startup and cached: @@ -40,7 +40,7 @@ type CredentialSource interface { | `gcp-service-account` | key location, `scopes` | Mints GCP OAuth2 access tokens from a service account key | | `github-app` | `app_id`, `installation_id`, private key | Generates GitHub App installation tokens | -## RefreshingSource and Background Refresh +## RefreshingSource and background refresh Sources whose credentials expire implement `RefreshingSource`: @@ -59,7 +59,7 @@ type RefreshingSource interface { The `github-app` and `gcp-service-account` sources are `RefreshingSource`s. Both produce tokens that expire after one hour, so gatekeeper refreshes them every 45 minutes. The `process` source is also a `RefreshingSource`: when its command output is AWS `credential_process`-format JSON, TTL is the time until the embedded `Expiration`; otherwise it reports a fixed interval (`ttl` in the source config, default 5 minutes). -## Source Deduplication +## Source deduplication When multiple credential entries share the same `SourceConfig` (identical `type`, `var`, `secret`, etc.), gatekeeper fetches the credential once and applies it to all matching hosts. A single background refresh goroutine updates every host that shares the source. @@ -85,7 +85,7 @@ credentials: Both entries share the same `github-app` source. Gatekeeper makes one API call to GitHub, generates one installation token, and applies it to both `api.github.com` (as `Bearer`) and `github.com` (as `Basic x-access-token:token`). -## CredentialResolver for Dynamic Resolution +## CredentialResolver for dynamic resolution Some credential flows require per-request context — for example, RFC 8693 token exchange, where the proxy exchanges a caller's identity token for a scoped access token. These flows use `CredentialResolver` instead of `CredentialSource`: @@ -97,7 +97,7 @@ Unlike static sources (fetched once at startup), resolvers are called on every r The `token-exchange` source type creates a `CredentialResolver`. All other source types create a `CredentialSource`. -## Error Handling +## Error handling Credential source errors at startup are fatal — gatekeeper refuses to start if any `Fetch` call fails. This fail-fast behavior prevents the proxy from running without required credentials. diff --git a/docs/content/concepts/04-network-policy.md b/docs/content/concepts/04-network-policy.md index 080e81f..2d00357 100644 --- a/docs/content/concepts/04-network-policy.md +++ b/docs/content/concepts/04-network-policy.md @@ -4,11 +4,11 @@ description: "How Gatekeeper enforces network access control with permissive and keywords: ["gatekeeper", "network policy", "allow list", "strict mode"] --- -# Network Policy +# Network policy Gatekeeper enforces network policy to control which hosts a client can reach through the proxy. Policy evaluation happens before credential injection — blocked requests never receive credentials. -## Permissive vs Strict +## Permissive vs strict The `policy` field controls the default behavior: @@ -27,7 +27,7 @@ network: In permissive mode, the allow list is ignored. All traffic passes through. -## Allow List Mechanics +## Allow list mechanics The allow list is a set of host patterns. Each pattern follows the same matching rules as credential host patterns: @@ -37,7 +37,7 @@ The allow list is a set of host patterns. Each pattern follows the same matching When a client sends a CONNECT request, gatekeeper extracts the host and port and checks them against the allow list. If no pattern matches, the tunnel is refused. -## Grant Hosts Are Auto-Allowed +## Grant hosts are auto-allowed When callers pass grant names to `SetNetworkPolicy`, gatekeeper expands each grant to its known host patterns and adds them to the allow list automatically. This is used by moat's daemon layer, which passes per-run grants when registering runs. @@ -50,7 +50,7 @@ For example, the `github` grant expands to: In standalone mode (`gatekeeper.yaml`), grant expansion does not apply — only the explicit `allow` list is used. Add credential hosts to the allow list manually when using strict mode. -## Interaction with Credential Injection +## Interaction with credential injection Network policy and credential injection are independent checks that run in sequence: @@ -59,17 +59,17 @@ Network policy and credential injection are independent checks that run in seque This ordering has a security property: credentials are never sent to unauthorized hosts. Even if a credential pattern matches a host that is blocked by network policy, the credential is never injected because the request never reaches the injection step. -## Per-Path Rules +## Per-path rules When gatekeeper has path-level rules (configured via `RequestChecker`), it evaluates them on the inner HTTP request after TLS interception — not on the CONNECT tunnel. The CONNECT request only carries the host, not the path. Gatekeeper intercepts the tunnel, reads the plaintext request, and then checks `method` and `path` against the rules. > **Note:** Per-path rules require TLS interception (a CA must be configured). Without interception, only host-level allow/deny applies. Gatekeeper logs a warning if path rules are configured but the CA is missing. -## Host Gateway Policy +## Host gateway policy When a request targets a host gateway address (synthetic hostname or loopback), gatekeeper applies a separate check: the destination port must be in the run's `AllowedHostPorts` list. This prevents containers from reaching arbitrary services on the host machine. See [Host Gateway](./07-host-gateway.md) for details. -## HTTP-Scope Keep Policy +## HTTP-scope Keep policy On intercepted requests, a second, distinct policy layer runs after network policy allows the request: if the run's `RunContextData.KeepEngines` has an entry keyed `"http"`, gatekeeper evaluates that Keep engine against the request before forwarding it. Unlike network policy, this check can inspect the request body — the engine's rules are evaluated against a parsed HTTP call (method, host, headers, and body), not just the host. @@ -77,7 +77,7 @@ This layer is engine-driven: the `"http"` engine comes from an embedder's `RunCo Body inspection fails closed — if the body can't be parsed or evaluation errors, the request is denied rather than passed through. A denial returns `403 Forbidden` with `X-Moat-Blocked: keep-policy` (distinct from network policy's `407`/`request-rule` denials) and a plaintext body describing the host and, for body-inspection failures, that the body couldn't be inspected. -## Blocked Response Format +## Blocked response format Blocked requests receive one of two denial styles depending on which layer denied them: diff --git a/docs/content/concepts/05-mcp-relay.md b/docs/content/concepts/05-mcp-relay.md index bad9f46..bc90673 100644 --- a/docs/content/concepts/05-mcp-relay.md +++ b/docs/content/concepts/05-mcp-relay.md @@ -4,18 +4,18 @@ description: "How Gatekeeper relays Model Context Protocol requests to remote MC keywords: ["gatekeeper", "MCP relay", "model context protocol", "SSE streaming"] --- -# MCP Relay +# MCP relay Gatekeeper relays Model Context Protocol (MCP) requests to remote MCP servers with credential injection. MCP clients that cannot route traffic through an HTTP proxy connect to gatekeeper's relay endpoint directly, and gatekeeper forwards requests to the real MCP server with authentication headers attached. -## What MCP Relay Does +## What MCP relay does MCP servers often require authentication — an API key, OAuth token, or other credential. The MCP relay solves two problems: 1. **Credential injection for MCP.** The client sends requests to gatekeeper without credentials. Gatekeeper looks up the configured grant for the target MCP server and injects the real credential before forwarding. 2. **Proxy bypass.** Some MCP clients do not respect `HTTP_PROXY` settings. The relay endpoint (`/mcp/{server-name}`) accepts direct HTTP connections, eliminating the need for proxy-aware clients. -## Request Flow +## Request flow A relay request follows this path: @@ -36,17 +36,17 @@ A relay request follows this path: A request to `/mcp/context7/v1/endpoint` forwards to `https://mcp.context7.com/mcp/v1/endpoint` with the `Authorization` header set to the resolved credential. -## Daemon-Mode Token-Embedded Path +## Daemon-mode token-embedded path The relay path above (`/mcp/{server-name}`) relies on `Proxy-Authorization` to resolve run context, which requires the request to go through the proxy mechanism. When gatekeeper runs with a `ContextResolver` (daemon mode — moat's per-run registration, not standalone `gatekeeper.yaml` mode) and a request arrives directly rather than proxied, gatekeeper also serves `/mcp/{token}/{server-name}[/path]`: the run's proxy auth token is embedded in the URL itself, since a direct request carries no `Proxy-Authorization` header. Gatekeeper extracts the token, resolves it to run context, strips the token from the path, and dispatches to the same relay handling described above. This form only exists in daemon mode — standalone gatekeeper has no `ContextResolver` and does not serve it. -## SSE Streaming +## SSE streaming MCP uses Server-Sent Events (SSE) for streaming responses. Gatekeeper supports this with a per-chunk flush loop (`streamResponseBody`), not `io.Copy`'s buffered copy: it reads the upstream body in 4096-byte chunks and, after writing each chunk to the client, calls `Flush()` on the `http.ResponseWriter` if it implements `http.Flusher` — so events reach the client as they arrive rather than waiting for a larger buffer to fill. The relay HTTP client has no client-level timeout — MCP SSE streams are long-lived connections that may remain open indefinitely. -## Credential Injection Modes +## Credential injection modes MCP credential injection works in two modes: @@ -56,7 +56,7 @@ MCP credential injection works in two modes: For OAuth grants (grant names starting with `oauth:`), the credential value is automatically prefixed with `Bearer `. -## Keep Policy Evaluation +## Keep policy evaluation When a Keep policy engine is configured for an MCP server (keyed as `mcp-{server-name}`), gatekeeper evaluates `tools/call` requests before forwarding: @@ -65,7 +65,7 @@ When a Keep policy engine is configured for an MCP server (keyed as `mcp-{server - If the engine returns `Redact`, tool arguments are mutated according to the policy's mutation rules, and the modified request is forwarded. - Non-JSON request bodies are denied (fail-closed) when a policy is configured. -## Error Handling +## Error handling - Unknown server name: `404` with a message listing available server count. - Credential resolution failure: `500` with the grant name and a suggested `moat grant` command. diff --git a/docs/content/concepts/06-observability.md b/docs/content/concepts/06-observability.md index 92a1063..691a8d1 100644 --- a/docs/content/concepts/06-observability.md +++ b/docs/content/concepts/06-observability.md @@ -8,7 +8,7 @@ keywords: ["gatekeeper", "observability", "OpenTelemetry", "metrics", "logging"] Gatekeeper produces structured logs, distributed traces, and request metrics via OpenTelemetry. The proxy core has no direct OTel dependency — instrumentation is layered on externally through callbacks and HTTP middleware. -## Callback-Based Architecture +## Callback-based architecture The `proxy` package defines two callback types for instrumentation: @@ -17,7 +17,7 @@ The `proxy` package defines two callback types for instrumentation: The `gatekeeper` package (standalone server wiring) sets these callbacks at startup. The callbacks write canonical log lines, enrich OTel spans, and record metrics. This design keeps `proxy/proxy.go` free of OTel imports. -## OTelHandler Middleware +## OTelHandler middleware `proxy.OTelHandler` wraps the proxy's `http.Handler` with OpenTelemetry tracing and metrics: @@ -39,7 +39,7 @@ The `statusRecorder` implements `http.Hijacker` by delegating to the underlying `OTelHandler` wraps only the HTTP proxy listener. The Postgres data-plane listener (see [Postgres Data Plane](./08-postgres-data-plane.md)) is a separate `net.Listener` that this middleware never sees, so it produces no `proxy.request`-family spans or `proxy.request.duration`/`proxy.request.count` metrics — `postgres` connections are observable only through the canonical log line below. -## Canonical Log Lines +## Canonical log lines Gatekeeper emits one wide structured log entry per request at completion. Each log line contains all request context in a single record: @@ -67,7 +67,7 @@ Gatekeeper emits one wide structured log entry per request at completion. Each l Log level is determined by outcome: `ERROR` for server errors or transport failures, `WARN` for policy denials or client errors, `INFO` for successful requests. -## Client IP Attribution Behind a Load Balancer +## Client IP attribution behind a load balancer By default, `client_ip` comes from the raw TCP peer address of the proxy listener's accepted connection. That's accurate for a directly-exposed listener, but not for one that sits behind a TCP-terminating load balancer (e.g. GCP's global TCP Proxy load balancer) — every connection appears to originate from the load balancer's own front-end range (`35.191.0.0/16` for GCP), not the real client. @@ -75,7 +75,7 @@ Setting `network.proxy_protocol: true` wraps the proxy listener with PROXY proto Because the header is honored from any peer that can reach the listener, only enable `proxy_protocol` when the port is reachable solely through the load balancer, and never use `client_ip` for security decisions — it's a logging convenience, not an authenticated identity. -## Request ID Tracking +## Request ID tracking Every request receives a unique identifier. Gatekeeper checks for an `X-Request-Id` header from the caller. If present, it is reused. Otherwise, gatekeeper generates a TypeID with a `req` prefix (e.g., `req_01h455vb4pex5vsknk084sn02q`). @@ -86,7 +86,7 @@ The request ID is: - Stored in the request context for extraction by loggers and span enrichment. - Included in canonical log lines and OTel span events. -## slog-to-OTel Bridge +## slog-to-OTel bridge Gatekeeper uses a `multiHandler` to fan out every slog record to two destinations: diff --git a/docs/content/concepts/07-host-gateway.md b/docs/content/concepts/07-host-gateway.md index fa3cb5c..9f6d979 100644 --- a/docs/content/concepts/07-host-gateway.md +++ b/docs/content/concepts/07-host-gateway.md @@ -4,11 +4,11 @@ description: "How Gatekeeper maps synthetic hostnames to host machine IPs, enabl keywords: ["gatekeeper", "host gateway", "container networking", "loopback equivalence"] --- -# Host Gateway +# Host gateway Gatekeeper's host gateway maps a synthetic hostname (used inside containers) to the host machine's IP address. This enables containers to reach services running on the host while maintaining credential injection and network policy enforcement. -## What Host Gateway Solves +## What host gateway solves Containers cannot reliably address the host machine. Docker provides `host.docker.internal`, but this is not universal — it resolves differently across platforms and may not exist in all runtimes. The host gateway gives each run a consistent hostname that resolves to the host machine's actual IP, allowing gatekeeper to intercept, authorize, and forward the traffic. @@ -21,7 +21,7 @@ The `RunContextData` struct carries two fields for this: When a CONNECT request targets the gateway hostname, gatekeeper rewrites the dial address from the synthetic hostname to `HostGatewayIP` before establishing the upstream connection. -## Synthetic Hostname Mapping +## Synthetic hostname mapping The container's `/etc/hosts` file maps the synthetic hostname to the proxy's IP. When the container connects to `moat-host-gateway:8080`, the request routes to gatekeeper. Gatekeeper recognizes the hostname as a gateway address, applies network policy, and dials the real host IP. @@ -32,7 +32,7 @@ curl http://moat-host-gateway:8080/api/data Gatekeeper intercepts this as a CONNECT (for HTTPS) or plain HTTP request, checks that port 8080 is in `AllowedHostPorts`, and forwards to `{HostGatewayIP}:8080`. -## Loopback Equivalence +## Loopback equivalence When `HostGatewayIP` resolves to a loopback address (`127.0.0.1`, `::1`), gatekeeper treats `localhost`, `127.0.0.1`, and `::1` as equivalent to the gateway hostname. This prevents a bypass: without this equivalence, a container can connect directly to `localhost` or `127.0.0.1` to skip network policy that only checks the gateway hostname. @@ -44,7 +44,7 @@ The equivalence check follows this logic: When loopback equivalence is active, credentials configured for the gateway hostname also match requests to `localhost`, `127.0.0.1`, and `::1`. -## Port-Based Access Control +## Port-based access control Host gateway traffic is not governed by the standard allow/deny list. Instead, each destination port must be explicitly listed in `AllowedHostPorts`. A request to `moat-host-gateway:3000` is allowed only if port 3000 appears in the run's allowed ports. @@ -54,7 +54,7 @@ The `AllowedHostPorts` field in `RunContextData` lists the permitted ports. This Requests to unlisted ports receive a `407` response with an `X-Moat-Blocked: host-service` header and a message indicating which port was denied and how to allow it. -## Credential Matching Across Gateway and Loopback +## Credential matching across gateway and loopback Credentials configured for the gateway hostname apply to all equivalent addresses when loopback equivalence is active. If a credential targets `moat-host-gateway` and the gateway routes to loopback, that credential also applies to requests targeting `localhost`, `127.0.0.1`, or `::1` on an allowed port. diff --git a/docs/content/concepts/08-postgres-data-plane.md b/docs/content/concepts/08-postgres-data-plane.md index 3de3450..4414666 100644 --- a/docs/content/concepts/08-postgres-data-plane.md +++ b/docs/content/concepts/08-postgres-data-plane.md @@ -4,13 +4,13 @@ description: "How Gatekeeper proxies the Postgres wire protocol, authenticating keywords: ["gatekeeper", "postgres", "neon", "SCRAM", "credential injection", "database proxy"] --- -# Postgres Data Plane +# Postgres data plane Gatekeeper runs a second listener that speaks the Postgres wire protocol. A client connects to it with the real database hostname and an authentication token in place of the database password. Gatekeeper resolves the real password, authenticates upstream, and relays the connection. The database password never reaches the client. This mirrors the HTTP credential-injection plane: the client presents a weak identity (its run token), and Gatekeeper substitutes the real credential before talking to the upstream. The two planes share configuration, network policy, per-run context scoping, and audit logging. -## What It Solves +## What it solves A client that connects to a managed Postgres database directly holds a connection string with an embedded password. A process that reads its own environment can copy that string and connect from anywhere; a network allowlist is the only remaining control. @@ -24,7 +24,7 @@ The target endpoint travels in the connection's TLS Server Name Indication (SNI) Routing by SNI is what allows a single listener to reach arbitrary endpoints — including database branches created after Gatekeeper started — without enumerating them in configuration. The embedder arranges DNS so the database hostname resolves to Gatekeeper inside the client's environment; that DNS configuration is outside Gatekeeper's scope. -## Connection Lifecycle +## Connection lifecycle 1. The client sends an `SSLRequest`. Gatekeeper replies `S` and completes a TLS handshake using a certificate minted from the configured CA for the SNI hostname. A client that skips the `SSLRequest` is refused before any credential is requested, so the run token never crosses the wire unencrypted. 2. The client sends a `StartupMessage` carrying the user (role) and database. @@ -46,7 +46,7 @@ A credential with a `postgres` block selects how the upstream password is resolv The API key is itself a credential source, so it can come from an environment variable, AWS Secrets Manager, or GCP Secret Manager. See [Credential Sources](./03-credential-sources.md). -## Security Properties +## Security properties - Run-token comparison uses the same constant-time path as the HTTP plane. - The client's token is read only inside Gatekeeper's TLS tunnel. diff --git a/docs/content/getting-started/01-introduction.md b/docs/content/getting-started/01-introduction.md index 4f9bd53..1b7e7f0 100644 --- a/docs/content/getting-started/01-introduction.md +++ b/docs/content/getting-started/01-introduction.md @@ -12,7 +12,7 @@ Gatekeeper is a standalone credential-injecting TLS-intercepting proxy. It sits - **Credential injection** — Resolve credentials from environment variables, static values, host command output, AWS Secrets Manager, GCP Secret Manager, or GitHub App tokens, then inject them as HTTP headers for matching hosts. - **TLS interception** — Man-in-the-middle proxy with per-host certificate generation from a configured CA. The proxy terminates TLS, reads plaintext requests, injects credentials, and forwards to the real server. -- **Multiple credential sources** — Pluggable backend system. Environment variables, static values, and host commands (`process`) for development and local helpers. AWS Secrets Manager, GCP Secret Manager, and GitHub App tokens for production. RFC 8693 token exchange for multi-user deployments. +- **Multiple credential sources** — Pluggable source system. Environment variables, static values, and host commands (`process`) for development and local helpers. AWS Secrets Manager, GCP Secret Manager, and GitHub App tokens for production. RFC 8693 token exchange for multi-user deployments. - **Network policy** — Allow or deny traffic by host pattern. `permissive` mode allows all traffic. `strict` mode denies all traffic except explicitly allowed hosts. - **Postgres data plane** — A second listener that speaks the Postgres wire protocol, authenticating clients with a run token and resolving the real upstream password (a static value or a per-branch Neon password) so the database secret never reaches the client. - **MCP relay** — Forward Model Context Protocol requests to upstream servers with credential injection and SSE streaming. diff --git a/docs/content/guides/01-ca-setup.md b/docs/content/guides/01-ca-setup.md index 291884f..0c30c26 100644 --- a/docs/content/guides/01-ca-setup.md +++ b/docs/content/guides/01-ca-setup.md @@ -4,7 +4,7 @@ description: "Generate a Certificate Authority for TLS interception and configur keywords: ["gatekeeper", "CA certificate", "TLS setup", "certificate trust"] --- -# CA Certificate Setup +# CA certificate setup Generate a Certificate Authority for TLS interception and trust it on your system. Gatekeeper uses this CA to sign per-host certificates dynamically, enabling credential injection into HTTPS requests. @@ -56,7 +56,7 @@ sudo cp ca.crt /etc/pki/ca-trust/source/anchors/gatekeeper-ca.crt sudo update-ca-trust ``` -## Per-Tool Trust +## Per-tool trust Some tools require explicit CA configuration instead of using the system store. @@ -97,6 +97,6 @@ curl --cacert ca.crt --proxy http://127.0.0.1:9080 -v https://example.com 2>&1 | The output should show the CN of your CA certificate as the issuer. -## Next Steps +## Next steps - [Environment Credentials](./02-environment-credentials.md) — inject your first credential through the proxy diff --git a/docs/content/guides/02-environment-credentials.md b/docs/content/guides/02-environment-credentials.md index 41324f7..749099f 100644 --- a/docs/content/guides/02-environment-credentials.md +++ b/docs/content/guides/02-environment-credentials.md @@ -4,7 +4,7 @@ description: "Read a credential from an environment variable and inject it into keywords: ["gatekeeper", "environment variables", "credential injection", "env source"] --- -# Environment Variable Credentials +# Environment variable credentials Read a credential from an environment variable and inject it into HTTPS requests. This is the simplest credential source. @@ -44,7 +44,7 @@ log: The `env` source reads the credential from the environment variable named in `var`. The variable must be set when the proxy starts. -## Start the Proxy +## Start the proxy Set the token and start gatekeeper: @@ -63,7 +63,7 @@ Gatekeeper resolves the credential at startup. For `Authorization` headers, the Override with the `prefix` field if needed. -## Make a Request +## Make a request In another terminal: @@ -81,7 +81,7 @@ Check the proxy log output. A successful injection produces a line like: level=INFO msg=request http_method=GET http_host=api.github.com http_status=200 credential_injected=true injected_headers=authorization grants=github ``` -## Next Steps +## Next steps - [AWS Secrets Manager](./03-aws-secrets-manager.md) — fetch credentials from AWS instead of environment variables -- [Network Lockdown](./07-network-lockdown.md) — restrict which hosts the proxy can reach +- [Network Lockdown](./07-network-lockdown.md) — restrict which hosts the proxy reaches diff --git a/docs/content/guides/03-aws-secrets-manager.md b/docs/content/guides/03-aws-secrets-manager.md index 3da77a6..87074d7 100644 --- a/docs/content/guides/03-aws-secrets-manager.md +++ b/docs/content/guides/03-aws-secrets-manager.md @@ -4,7 +4,7 @@ description: "Fetch a credential from AWS Secrets Manager at proxy startup and i keywords: ["gatekeeper", "AWS Secrets Manager", "credential source", "cloud secrets"] --- -# AWS Secrets Manager Credentials +# AWS Secrets Manager credentials Fetch a credential from AWS Secrets Manager at proxy startup and inject it into HTTPS requests. @@ -14,7 +14,7 @@ Fetch a credential from AWS Secrets Manager at proxy startup and inject it into - AWS credentials configured (environment variables, IAM role, or `~/.aws/credentials`) - A secret stored in AWS Secrets Manager containing the credential value as a plaintext string -## IAM Permissions +## IAM permissions The IAM principal running gatekeeper needs: @@ -65,7 +65,7 @@ log: The secret value must be a plaintext string (not binary). Gatekeeper fetches it once at startup with a 10-second timeout. -## Start the Proxy +## Start the proxy ```bash gatekeeper --config gatekeeper.yaml @@ -87,7 +87,7 @@ level=INFO msg=request http_host=api.github.com credential_injected=true grants= > **Note:** Gatekeeper fetches the secret once at startup. To pick up a rotated secret, restart the proxy. -## Next Steps +## Next steps - [GCP Secret Manager](./04-gcp-secret-manager.md) — use GCP instead of AWS - [Network Lockdown](./07-network-lockdown.md) — restrict proxy traffic to specific hosts diff --git a/docs/content/guides/04-gcp-secret-manager.md b/docs/content/guides/04-gcp-secret-manager.md index 626f3f5..1367f93 100644 --- a/docs/content/guides/04-gcp-secret-manager.md +++ b/docs/content/guides/04-gcp-secret-manager.md @@ -4,7 +4,7 @@ description: "Fetch a credential from Google Cloud Secret Manager at proxy start keywords: ["gatekeeper", "GCP Secret Manager", "credential source", "cloud secrets"] --- -# GCP Secret Manager Credentials +# GCP Secret Manager credentials Fetch a credential from Google Cloud Secret Manager at proxy startup and inject it into HTTPS requests. @@ -14,7 +14,7 @@ Fetch a credential from Google Cloud Secret Manager at proxy startup and inject - GCP Application Default Credentials configured (`gcloud auth application-default login` or a service account key) - A secret stored in GCP Secret Manager containing the credential value -## IAM Permissions +## IAM permissions The service account or principal running gatekeeper needs the `Secret Manager Secret Accessor` role (`roles/secretmanager.secretAccessor`) on the target secret. @@ -56,7 +56,7 @@ log: Gatekeeper constructs the resource name `projects/{project}/secrets/{secret}/versions/{version}` and fetches the payload at startup with a 10-second timeout. -## Pin a Specific Version +## Pin a specific version To pin to a specific secret version instead of `latest`: @@ -68,7 +68,7 @@ source: version: "3" ``` -## Start the Proxy +## Start the proxy ```bash gatekeeper --config gatekeeper.yaml @@ -90,7 +90,7 @@ level=INFO msg=request http_host=api.github.com credential_injected=true grants= > **Note:** Gatekeeper fetches the secret once at startup. To pick up a rotated secret, restart the proxy. -## Next Steps +## Next steps - [GitHub App Tokens](./05-github-app-tokens.md) — auto-refreshing short-lived tokens - [Network Lockdown](./07-network-lockdown.md) — restrict proxy traffic to specific hosts diff --git a/docs/content/guides/05-github-app-tokens.md b/docs/content/guides/05-github-app-tokens.md index ae62747..f7bbfbc 100644 --- a/docs/content/guides/05-github-app-tokens.md +++ b/docs/content/guides/05-github-app-tokens.md @@ -4,7 +4,7 @@ description: "Generate short-lived GitHub installation tokens from a GitHub App keywords: ["gatekeeper", "GitHub App", "installation tokens", "auto-refresh"] --- -# GitHub App Tokens +# GitHub App tokens Generate short-lived GitHub installation tokens from a GitHub App private key. Tokens refresh automatically in the background. @@ -55,7 +55,7 @@ log: Set either `private_key_path` or `private_key_env`, not both. -### Private Key via Environment Variable +### Private key via environment variable For environments where files are not practical (containers, CI): @@ -71,7 +71,7 @@ source: export GITHUB_APP_PRIVATE_KEY="$(cat github-app-key.pem)" ``` -## Auto-Refresh Behavior +## Auto-refresh behavior GitHub installation tokens expire after one hour. Gatekeeper refreshes them automatically: @@ -82,7 +82,7 @@ GitHub installation tokens expire after one hour. Gatekeeper refreshes them auto When multiple credential entries share the same `github-app` source (e.g., `api.github.com` and `github.com`), a single refresh goroutine updates all of them. -## Start the Proxy +## Start the proxy ```bash gatekeeper --config gatekeeper.yaml @@ -108,7 +108,7 @@ level=DEBUG msg="credential refreshed" host=api.github.com grant=github ttl=1h0m See [`examples/gatekeeper-github-app.yaml`](https://github.com/majorcontext/gatekeeper/blob/main/examples/gatekeeper-github-app.yaml) for a complete working example. -## Next Steps +## Next steps - [Token Exchange](./06-token-exchange.md) — per-user credential resolution via RFC 8693 - [Network Lockdown](./07-network-lockdown.md) — restrict proxy traffic to specific hosts diff --git a/docs/content/guides/06-token-exchange.md b/docs/content/guides/06-token-exchange.md index d29260d..cb892cf 100644 --- a/docs/content/guides/06-token-exchange.md +++ b/docs/content/guides/06-token-exchange.md @@ -4,7 +4,7 @@ description: "Resolve per-user credentials dynamically by calling an external Se keywords: ["gatekeeper", "token exchange", "RFC 8693", "STS", "OAuth"] --- -# Token Exchange (RFC 8693) +# Token exchange (RFC 8693) Resolve per-user credentials dynamically by calling an external Security Token Service (STS). Multiple callers with different identities share a single gatekeeper instance. Each request triggers a token exchange -- or uses a cached token -- scoped to the caller's identity. @@ -16,7 +16,7 @@ This guide covers gatekeeper configuration and the STS endpoint contract. Implem - An STS endpoint that implements RFC 8693 token exchange (see [STS Endpoint Requirements](#sts-endpoint-requirements) below) - Client credentials (`client_id` and `client_secret`) for authenticating gatekeeper to the STS -## How It Works +## How it works 1. A request arrives at gatekeeper with a subject identity (via header or proxy auth username). 2. Gatekeeper checks its cache for a valid token for that subject. @@ -24,11 +24,11 @@ This guide covers gatekeeper configuration and the STS endpoint contract. Implem 4. The STS returns an `access_token` (and optional `expires_in`). 5. Gatekeeper caches the token and injects it into the upstream request. -## Subject Identity Modes +## Subject identity modes Gatekeeper extracts the subject identity from one of two sources. The two modes are mutually exclusive. -### Mode 1: Subject from Request Header +### Mode 1: Subject from request header The subject identity is read from a named HTTP header on each request. Gatekeeper strips the header before forwarding upstream. @@ -54,7 +54,7 @@ curl --proxy http://127.0.0.1:9080 --cacert ca.crt \ https://api.github.com/user ``` -### Mode 2: Subject from Proxy Auth +### Mode 2: Subject from proxy auth The subject identity is extracted from the username in proxy authentication credentials. No request headers are modified. @@ -79,7 +79,7 @@ export HTTP_PROXY="http://alice%40example.com:proxypass@127.0.0.1:9080" curl --cacert ca.crt https://api.github.com/user ``` -## Configuration Reference +## Configuration reference | Field | Required | Default | Description | |----------------------|----------------|--------------------------------------------------------|----------------------------------------------------------| @@ -94,7 +94,7 @@ curl --cacert ca.crt https://api.github.com/user | `actor_token_from` | No | -- | Set to `proxy-auth-password` to forward the proxy auth password as actor token | | `actor_token_type` | No | `urn:ietf:params:oauth:token-type:access_token` | Token type URI for the actor token | -## Actor Token Forwarding +## Actor token forwarding By default, subject identities are self-asserted -- any caller can claim any identity. In shared environments, use **actor token forwarding** to let the STS verify caller identity. @@ -125,7 +125,7 @@ Gatekeeper sends both `subject_token=alice@example.com` and `actor_token=ak_alic When `actor_token_from` is configured on any credential, gatekeeper requires all clients to provide Basic proxy auth with a non-empty password. The password is not checked against a static value -- it is forwarded to the STS. -## Caching Behavior +## Caching behavior Gatekeeper caches tokens per `(subject_token, actor_token)` pair: @@ -140,11 +140,11 @@ The cap exists because a long `expires_in` only means the token *may* live that A consequence: `expires_in` values above the cap no longer reduce STS request volume. Sizing the STS for roughly one exchange per subject per minute is the safe assumption. -## STS Endpoint Requirements +## STS endpoint requirements Gatekeeper sends a `POST` with `Content-Type: application/x-www-form-urlencoded` and HTTP Basic authentication. -### Request Format +### Request format ```http POST /token HTTP/1.1 @@ -155,7 +155,7 @@ Content-Type: application/x-www-form-urlencoded grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&subject_token=alice%40example.com&subject_token_type=urn%3Aietf%3Aparams%3Aoauth%3Atoken-type%3Aaccess_token&resource=https%3A%2F%2Fapi.github.com ``` -### Success Response (HTTP 200) +### Success response (HTTP 200) ```json { @@ -175,7 +175,7 @@ grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange&subject_tok `access_token` must be non-empty. Gatekeeper treats an empty value as an error. -### Error Response (non-200) +### Error response (non-200) Gatekeeper treats any non-200 status as a failure and returns HTTP 502 to the client. Use standard OAuth error format: @@ -188,7 +188,7 @@ Gatekeeper treats any non-200 status as a failure and returns HTTP 502 to the cl > **Note:** Do not echo `actor_token` in error responses. Gatekeeper logs up to 200 bytes of STS error bodies, so sensitive values would appear in proxy logs. -## Implementation Checklist +## Implementation checklist - [ ] Accept `POST` with `Content-Type: application/x-www-form-urlencoded` - [ ] Validate HTTP Basic auth credentials (`client_id` / `client_secret`) @@ -223,7 +223,7 @@ curl --cacert ca.crt --proxy http://127.0.0.1:9080 \ The proxy log shows credential injection with the grant name. -## Next Steps +## Next steps - [Network Lockdown](./07-network-lockdown.md) — combine token exchange with strict network policy - [OpenTelemetry](./08-opentelemetry.md) — trace token exchange calls end-to-end diff --git a/docs/content/guides/07-network-lockdown.md b/docs/content/guides/07-network-lockdown.md index a44a8ff..b29c04b 100644 --- a/docs/content/guides/07-network-lockdown.md +++ b/docs/content/guides/07-network-lockdown.md @@ -4,7 +4,7 @@ description: "Restrict which hosts the proxy forwards traffic to using strict ne keywords: ["gatekeeper", "network lockdown", "strict policy", "allow list"] --- -# Network Lockdown +# Network lockdown Restrict which hosts the proxy forwards traffic to. By default, gatekeeper operates in `permissive` mode -- it proxies requests to any host. Switch to `strict` mode to deny all traffic except explicitly allowed hosts. @@ -13,7 +13,7 @@ Restrict which hosts the proxy forwards traffic to. By default, gatekeeper opera - CA certificate generated ([CA Setup](./01-ca-setup.md)) - A working gatekeeper configuration with at least one credential -## Permissive Mode (Default) +## Permissive mode (default) The default configuration allows traffic to all hosts: @@ -24,7 +24,7 @@ network: All CONNECT and HTTP requests pass through. Credentials are injected only for matching hosts; all other traffic is forwarded without modification. -## Strict Mode +## Strict mode Switch to `strict` to deny all traffic except listed hosts: @@ -38,7 +38,7 @@ network: Requests to unlisted hosts receive an HTTP `407` response with a `Proxy-Authenticate: Moat-Policy` header. -## Glob Patterns +## Glob patterns The `allow` list supports glob patterns for flexible matching: @@ -50,7 +50,7 @@ The `allow` list supports glob patterns for flexible matching: Port numbers are stripped before matching -- `api.github.com:443` matches a rule for `api.github.com`. -## Combined Configuration +## Combined configuration Combine credential injection with network lockdown: @@ -103,6 +103,6 @@ Confirm allowed requests still work: curl --cacert ca.crt --proxy http://127.0.0.1:9080 https://api.github.com/user ``` -## Next Steps +## Next steps - [OpenTelemetry](./08-opentelemetry.md) — monitor denied requests with metrics and traces diff --git a/docs/content/guides/08-opentelemetry.md b/docs/content/guides/08-opentelemetry.md index 324c158..e93f944 100644 --- a/docs/content/guides/08-opentelemetry.md +++ b/docs/content/guides/08-opentelemetry.md @@ -75,7 +75,7 @@ Four instruments are registered under the `gatekeeper` meter: Gatekeeper bridges `slog` output to OTel via `otelslog`. Structured log records are sent to both the configured slog handler (text/JSON to stderr) and the OTel log exporter. Log records carry trace context for correlation. -## Local Collector Example +## Local collector example Run a local OpenTelemetry Collector with Jaeger: @@ -106,6 +106,6 @@ Export failures (for example, no collector listening at the configured `OTEL_EXP > **Note:** Without a registered OTel error handler, export errors fall through to the standard library `log` package, which gatekeeper's logging setup rewires through the configured slog handler at `INFO` level. Gatekeeper registers its own handler so these errors log at `DEBUG` instead. -## Next Steps +## Next steps - [Go Library](./09-go-library.md) — embed gatekeeper in a Go application with custom instrumentation diff --git a/docs/content/guides/09-go-library.md b/docs/content/guides/09-go-library.md index d9ca142..7d3548b 100644 --- a/docs/content/guides/09-go-library.md +++ b/docs/content/guides/09-go-library.md @@ -4,7 +4,7 @@ description: "Import Gatekeeper as a Go module to embed the credential-injecting keywords: ["gatekeeper", "Go library", "embedding", "proxy API"] --- -# Go Library Usage +# Go library usage Import gatekeeper as a Go module to embed the proxy in a custom application. This is how [moat](https://github.com/majorcontext/moat) integrates gatekeeper -- importing the proxy engine and adding per-run credential scoping via a daemon layer. @@ -18,7 +18,7 @@ Import gatekeeper as a Go module to embed the proxy in a custom application. Thi go get github.com/majorcontext/gatekeeper/proxy ``` -## Basic Setup +## Basic setup Create a proxy, load a CA, and set credentials: @@ -109,7 +109,7 @@ p.SetContextResolver(func(token string) (*proxy.RunContextData, bool) { Each caller authenticates via `Proxy-Authorization` (or the username/password in `HTTP_PROXY`). The resolver returns per-caller credentials, network policy, and MCP server configuration. -## How Moat Uses Gatekeeper +## How Moat uses Gatekeeper Moat imports `github.com/majorcontext/gatekeeper/proxy` and layers a daemon on top: @@ -120,7 +120,7 @@ Moat imports `github.com/majorcontext/gatekeeper/proxy` and layers a daemon on t Gatekeeper has no knowledge of moat. It exposes the `ContextResolver` hook and `RunContextData` struct; moat provides the implementation. -## OTel Middleware +## OTel middleware Wrap the proxy with `OTelHandler` for OpenTelemetry instrumentation: @@ -131,6 +131,6 @@ log.Fatal(http.ListenAndServe("127.0.0.1:9080", handler)) This adds request spans, duration histograms, and request counters. See [OpenTelemetry](./08-opentelemetry.md) for details on emitted signals. -## Next Steps +## Next steps - [WebSocket Support](./10-websockets.md) — WebSocket upgrades through the proxy diff --git a/docs/content/guides/10-websockets.md b/docs/content/guides/10-websockets.md index 13dfdac..a6f69e3 100644 --- a/docs/content/guides/10-websockets.md +++ b/docs/content/guides/10-websockets.md @@ -4,11 +4,11 @@ description: "WebSocket connections work through Gatekeeper with credential inje keywords: ["gatekeeper", "WebSocket", "upgrade request", "bidirectional tunneling"] --- -# WebSocket Support +# WebSocket support WebSocket connections work through gatekeeper with no special configuration. The proxy intercepts the TLS connection, injects credentials on the HTTP upgrade request, and then tunnels the bidirectional WebSocket frames transparently. -## How It Works +## How it works 1. The client sends `CONNECT host:443` through the proxy. 2. Gatekeeper terminates TLS and reads the plaintext HTTP request. @@ -70,7 +70,7 @@ After the upgrade completes, the proxy tunnels frames bidirectionally until eith - The proxy does not inspect or modify WebSocket frame content. - Connection lifetime is bounded by the proxy's idle timeout and the underlying TCP keepalive settings. -## Next Steps +## Next steps -- [Network Lockdown](./07-network-lockdown.md) — restrict which WebSocket hosts the proxy can reach +- [Network Lockdown](./07-network-lockdown.md) — restrict which WebSocket hosts the proxy reaches - [OpenTelemetry](./08-opentelemetry.md) — trace WebSocket CONNECT tunnels diff --git a/docs/content/guides/11-load-balancer-proxy-protocol.md b/docs/content/guides/11-load-balancer-proxy-protocol.md new file mode 100644 index 0000000..b5030bf --- /dev/null +++ b/docs/content/guides/11-load-balancer-proxy-protocol.md @@ -0,0 +1,150 @@ +--- +title: "Deploying Behind a TCP Load Balancer" +description: "Recover the real client IP when gatekeeper runs behind a TCP-terminating load balancer, using PROXY protocol v1/v2." +keywords: ["gatekeeper", "load balancer", "PROXY protocol", "GCP", "client IP"] +--- + +# Deploying behind a TCP load balancer + +Recover the real client IP when gatekeeper runs behind a TCP-terminating load balancer, such as GCP's global TCP Proxy load balancer. Without this, every request's `client_ip` log attribute shows the load balancer's hop instead of the actual client. + +## Prerequisites + +- A working gatekeeper deployment (see [Config file](../reference/02-config-file.md)) +- A TCP-terminating load balancer in front of gatekeeper that supports PROXY protocol v1/v2 (for example, GCP's global TCP Proxy load balancer) +- `gcloud` configured against the target GCP project, if following the GCP walkthrough below + +## Why client_ip shows the load balancer + +A TCP-terminating load balancer ends the client's TCP connection at its own edge and opens a new connection to gatekeeper from its own front-end IP range. GCP's global TCP Proxy load balancer dials backends from `35.191.0.0/16`. By default, gatekeeper reads `client_ip` from the raw TCP peer address of the accepted connection — so every request logs the load balancer's address, never the real client, no matter which host actually opened the connection. + +## How it works + +1. The client connects to the load balancer's public IP. +2. The load balancer terminates that connection and opens a new TCP connection to gatekeeper, prepending a PROXY protocol v1 or v2 header that names the original client address. +3. When `network.proxy_protocol: true`, gatekeeper wraps its proxy listener with PROXY protocol parsing. On each accepted connection, it reads the leading header (if present) and substitutes the header's advertised source address for the raw TCP peer address. +4. That substituted address flows through to the `client_ip` request-log attribute on every request path, including CONNECT-intercepted TLS traffic. +5. A connection that opens without a PROXY header — or that doesn't send one within a 10-second read timeout — falls back to the raw TCP peer address instead of being rejected. This fail-open behavior keeps load balancer health checks and direct probes of the port working. + +## Enabling proxy_protocol + +Set `network.proxy_protocol: true` in gatekeeper.yaml: + +```yaml +proxy: + host: 0.0.0.0 + port: 9080 + +tls: + ca_cert: ca.crt + ca_key: ca.key + +credentials: + - host: api.github.com + header: Authorization + grant: github + source: + type: env + var: GITHUB_TOKEN + +network: + policy: permissive + proxy_protocol: true + +log: + level: info + format: text +``` + +`proxy.host` is bound to `0.0.0.0` here because the listener must be reachable from the load balancer, which is not running on the same host. See [Security Requirement](#security-requirement) below before deploying this configuration. + +## Configuring the GCP backend service + +Enabling `proxy_protocol` on gatekeeper is only half the change — the load balancer must also be told to send the header. For a global TCP Proxy load balancer, this is the backend service's `proxyHeader` field. + +Update an existing backend service: + +```bash +gcloud compute backend-services update gatekeeper-backend \ + --global \ + --proxy-header=PROXY_V1 +``` + +Or set it at creation time: + +```bash +gcloud compute health-checks create http gatekeeper-health-check \ + --port=9080 \ + --request-path=/healthz + +gcloud compute backend-services create gatekeeper-backend \ + --global \ + --protocol=TCP \ + --health-checks=gatekeeper-health-check \ + --proxy-header=PROXY_V1 + +gcloud compute backend-services add-backend gatekeeper-backend \ + --global \ + --instance-group=gatekeeper-ig \ + --instance-group-zone=us-central1-a +``` + +`--proxy-header` accepts `NONE` (default) or `PROXY_V1`. Gatekeeper's PROXY protocol parser handles both v1 (text) and v2 (binary) headers, but GCP's global TCP Proxy load balancer sends v1. + +## Safe rollout ordering + +Deploy in this order to avoid an outage: + +1. **Enable `network.proxy_protocol: true` on gatekeeper first, and deploy it.** Because the listener fails open, connections without a PROXY header — which is every connection at this point, since the load balancer isn't sending one yet — fall back to the raw TCP peer address exactly as before. `/healthz` stays green throughout. +2. **Confirm gatekeeper is healthy and serving traffic** with `proxy_protocol` enabled but no header arriving yet. +3. **Update the backend service's `proxyHeader` to `PROXY_V1`.** Once the load balancer starts sending headers, gatekeeper parses them and `client_ip` starts reflecting real clients. + +Reversing this order — turning on `proxyHeader: PROXY_V1` before gatekeeper understands it — sends a raw PROXY header into a listener that doesn't parse it. Gatekeeper would treat the header bytes as the start of an HTTP request or TLS handshake and reject the connection, which is why gatekeeper goes first. + +## Security requirement + +> **Warning:** `network.proxy_protocol` trusts the PROXY header from any peer that can reach the listener. A client that connects directly to gatekeeper's port — bypassing the load balancer — can prepend its own PROXY header and forge its logged `client_ip`. + +Only enable `proxy_protocol` when the port is reachable solely through the load balancer, and never use `client_ip` for security decisions — it's a logging convenience, not an authenticated identity. Restrict direct access to the port with a firewall rule that allows only the load balancer's and health check's source ranges: + +```bash +gcloud compute firewall-rules create allow-gatekeeper-lb \ + --network=default \ + --direction=INGRESS \ + --action=ALLOW \ + --rules=tcp:9080 \ + --source-ranges=130.211.0.0/22,35.191.0.0/16 \ + --target-tags=gatekeeper +``` + +The Postgres data-plane listener does not parse PROXY protocol headers — it is a separate listener and port, unaffected by this setting. + +## Verification + +Before enabling `proxy_protocol`, the canonical log line shows the load balancer's address as `client_ip`: + +```text +level=INFO msg=request http_host=api.github.com client_ip=35.191.12.34 credential_injected=true +``` + +After enabling `network.proxy_protocol: true` on gatekeeper and `--proxy-header=PROXY_V1` on the backend service, the same request logs the real client: + +```text +level=INFO msg=request http_host=api.github.com client_ip=203.0.113.7 credential_injected=true +``` + +A connection whose PROXY header fails to parse — a malformed v1 line or truncated v2 binary header — is dropped, and gatekeeper logs one line at `DEBUG`: + +```text +level=DEBUG msg="dropping connection: malformed PROXY protocol header" peer=35.191.12.34:51422 err="proxyproto: ..." +``` + +## Health check notes + +Gatekeeper's fail-open policy means the load balancer's HTTP health check against `/healthz` keeps working whether or not it carries a PROXY header: a probe that includes one is parsed normally, and a probe that doesn't falls back to the raw TCP peer address. Neither case is rejected, so flipping `proxy_protocol` on and off does not require a matching change to the health check configuration. + +## Next steps + +- [Observability](../concepts/06-observability.md) — how `client_ip` and other request attributes flow into logs, traces, and metrics +- [Config file](../reference/02-config-file.md) — full `network.proxy_protocol` field reference +- [OpenTelemetry](./08-opentelemetry.md) — correlate `client_ip` with traces and metrics diff --git a/docs/content/guides/12-docker-deployment.md b/docs/content/guides/12-docker-deployment.md new file mode 100644 index 0000000..822ed8a --- /dev/null +++ b/docs/content/guides/12-docker-deployment.md @@ -0,0 +1,191 @@ +--- +title: "Running the Container Image" +description: "Pull and run the published gatekeeper container image, mount config and CA material, and wire clients to trust the proxy." +keywords: ["gatekeeper", "Docker", "container", "ghcr.io", "deployment"] +--- + +# Running the container image + +Gatekeeper publishes a multi-arch container image to `ghcr.io/majorcontext/gatekeeper` on every `v*` tag (`.github/workflows/release.yml`), built for `linux/amd64` and `linux/arm64` from `cmd/gatekeeper/Dockerfile`. The image is `gcr.io/distroless/static-debian12` with a single static binary at `/gatekeeper` — no shell, no package manager, no other executables. + +## Prerequisites + +- Docker (or another OCI-compatible runtime) +- A CA certificate and key for TLS interception ([CA Setup](./01-ca-setup.md)) +- A gatekeeper.yaml config file ([Config file](../reference/02-config-file.md)) + +## Pulling the image + +Pull by version tag: + +```bash +docker pull ghcr.io/majorcontext/gatekeeper:0.17.0 +``` + +Or pin to an immutable digest for reproducible deployments: + +```bash +docker pull ghcr.io/majorcontext/gatekeeper@sha256:1238110cc242a14719182f695c785d298e5a841ecec33f68dc70f88af57192b3 +``` + +A tag like `0.17.0` can be reassigned to a different digest if the release is ever rebuilt; a digest cannot. Use the tag for local development and the digest when the deployment pipeline needs to guarantee exactly which image is running. `major.minor` tags (`0.17`) and a `latest` tag are also published, alongside `sha-` tags for every build. + +## Mounting config and CA material + +The image ships no config and no CA — both must be mounted read-only at container start. The binary reads its config path from `--config` or the `GATEKEEPER_CONFIG` environment variable; the Dockerfile's `ENTRYPOINT` is `["/gatekeeper"]` with no default arguments, so one of the two must be supplied. + +Write a config that references the mounted paths: + +```yaml +proxy: + host: 0.0.0.0 + port: 9080 + +tls: + ca_cert: /etc/gatekeeper/ca.crt + ca_key: /etc/gatekeeper/ca.key + +credentials: + - host: api.github.com + header: Authorization + grant: github + source: + type: env + var: GITHUB_TOKEN + +log: + level: info + format: text +``` + +`proxy.host` is `0.0.0.0` because the proxy listens inside the container's own network namespace — binding to `127.0.0.1` would make it unreachable from outside the container even with the port published. + +## Running the container + +```bash +docker run -d --name gatekeeper \ + -p 127.0.0.1:9080:9080 \ + -v "$(pwd)/gatekeeper.yaml:/etc/gatekeeper/gatekeeper.yaml:ro" \ + -v "$(pwd)/ca.crt:/etc/gatekeeper/ca.crt:ro" \ + -v "$(pwd)/ca.key:/etc/gatekeeper/ca.key:ro" \ + -e GATEKEEPER_CONFIG=/etc/gatekeeper/gatekeeper.yaml \ + -e GITHUB_TOKEN \ + ghcr.io/majorcontext/gatekeeper:0.17.0 +``` + +`-p 127.0.0.1:9080:9080` publishes the proxy port to the host's loopback interface only. Widen this (or route it through a load balancer — see [Deploying Behind a TCP Load Balancer](./11-load-balancer-proxy-protocol.md)) only once the port's exposure is deliberate. + +## Wiring clients + +Clients reach the proxy the same way they would a non-containerized gatekeeper: set `HTTPS_PROXY` and trust the CA certificate. + +```bash +export HTTPS_PROXY=http://127.0.0.1:9080 +curl --cacert ca.crt https://api.github.com/user +``` + +See [CA Setup](./01-ca-setup.md#per-tool-trust) for trusting the CA in the system store, Node.js, Python, and Go, instead of passing it per-command. + +## Verification + +Pulling `0.17.0` and running it with a scratch config (reusing `examples/ca.crt` and `examples/ca.key`) confirms the full flow: + +```bash +$ curl -s http://127.0.0.1:9080/healthz +{"status":"ok"} + +$ curl --cacert ca.crt --proxy http://127.0.0.1:9080 https://httpbin.org/headers +{ + "headers": { + "Accept": "*/*", + "Accept-Encoding": "gzip", + "Authorization": "Bearer test-token-xxxx", + "Host": "httpbin.org", + "User-Agent": "curl/8.7.1", + "X-Amzn-Trace-Id": "Root=1-..." + } +} +``` + +The container's log line confirms the injection: + +```text +time=2026-07-15T12:46:19.438Z level=INFO msg=request request_id=req_01kxjwz81jf7mt3v9jnfss33pa http_method=GET http_host=httpbin.org http_path=/headers http_status=200 proxy_type=connect client_ip=172.17.0.1 credential_injected=true injected_headers=authorization grants=test +``` + +`client_ip` shows the Docker bridge network's gateway address (`172.17.0.1`), not a real external client — expected for a container reached directly from the host. Behind a load balancer, see [Deploying Behind a TCP Load Balancer](./11-load-balancer-proxy-protocol.md) to recover the actual client address. + +## Health checks + +`/healthz` is served on the proxy port and returns `{"status":"ok"}` with HTTP 200 for any `GET` request, unauthenticated. It's suitable for a container orchestrator's liveness or readiness probe. + +> **Note:** The image has no shell and no `curl`/`wget` binary, so a Docker `HEALTHCHECK` instruction or a Compose `healthcheck.test` — both of which exec a command *inside* the container — cannot invoke `curl` the way a typical image's health check does. Attempting it fails with `executable file not found in $PATH`. Probe `/healthz` from outside the container instead: an orchestrator's HTTP-based probe (Kubernetes `livenessProbe`/`readinessProbe` with `httpGet`, which the kubelet performs from outside the container) or an external monitoring check against the published port both work without needing a shell inside the image. + +```yaml +# Kubernetes probe example -- httpGet runs from the kubelet, not inside the container. +livenessProbe: + httpGet: + path: /healthz + port: 9080 + initialDelaySeconds: 2 + periodSeconds: 10 +``` + +## Exposing the Postgres data-plane port + +The optional Postgres data-plane listener ([Postgres Data Plane](../concepts/08-postgres-data-plane.md)) is a second, separate listener. Publish its port alongside the proxy port and add a `postgres` section to the config: + +```yaml +postgres: + host: 0.0.0.0 + port: 5432 +``` + +```bash +docker run -d --name gatekeeper \ + -p 127.0.0.1:9080:9080 \ + -p 127.0.0.1:5432:5432 \ + -v "$(pwd)/gatekeeper.yaml:/etc/gatekeeper/gatekeeper.yaml:ro" \ + -v "$(pwd)/ca.crt:/etc/gatekeeper/ca.crt:ro" \ + -v "$(pwd)/ca.key:/etc/gatekeeper/ca.key:ro" \ + -e GATEKEEPER_CONFIG=/etc/gatekeeper/gatekeeper.yaml \ + ghcr.io/majorcontext/gatekeeper:0.17.0 +``` + +The `postgres` section requires `tls.ca_cert` and `tls.ca_key` — gatekeeper refuses to start without them. + +## Environment variables that matter in containers + +| Variable | Description | +|---|---| +| `GATEKEEPER_CONFIG` | Path to gatekeeper.yaml inside the container. Required unless `--config` is passed as the container command instead. | +| `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP collector endpoint. Defaults to `localhost:4318` if unset — inside a container, `localhost` is the container's own network namespace, not the host, so gatekeeper attempts (and, absent a collector, fails to reach) a collector inside the container unless this is set to a reachable address. | +| `OTEL_SDK_DISABLED` | Set to `true` to skip OTel exporter setup entirely. Useful when no collector is reachable from the container and the resulting DEBUG-level connection-refused log lines aren't wanted. | +| Any credential source env var (e.g. `GITHUB_TOKEN`) | Referenced by `source.var` in credential config entries; pass with `-e VAR` or `-e VAR=value`. | + +`SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt` is already set in the image by the `distroless/static-debian12` base and points Go's HTTP client at the base image's system root CAs — relevant to credential sources that call out over HTTPS (AWS Secrets Manager, GCP Secret Manager, token exchange, GitHub App installation tokens). It does not need to be set or changed for gatekeeper's own interception CA, which is configured separately via `tls.ca_cert`/`tls.ca_key`. + +## Compose example + +```yaml +services: + gatekeeper: + image: ghcr.io/majorcontext/gatekeeper:0.17.0 + ports: + - "127.0.0.1:9080:9080" + volumes: + - ./gatekeeper.yaml:/etc/gatekeeper/gatekeeper.yaml:ro + - ./ca.crt:/etc/gatekeeper/ca.crt:ro + - ./ca.key:/etc/gatekeeper/ca.key:ro + environment: + GATEKEEPER_CONFIG: /etc/gatekeeper/gatekeeper.yaml + GITHUB_TOKEN: ${GITHUB_TOKEN} + OTEL_SDK_DISABLED: "true" + restart: unless-stopped +``` + +## Next steps + +- [Deploying Behind a TCP Load Balancer](./11-load-balancer-proxy-protocol.md) — recover real client IPs when the container sits behind a TCP-terminating load balancer +- [OpenTelemetry](./08-opentelemetry.md) — point the container at a collector instead of disabling the SDK +- [Postgres Data Plane](../concepts/08-postgres-data-plane.md) — how the optional Postgres listener resolves upstream passwords diff --git a/docs/content/guides/13-postgres-neon.md b/docs/content/guides/13-postgres-neon.md new file mode 100644 index 0000000..0d75bc9 --- /dev/null +++ b/docs/content/guides/13-postgres-neon.md @@ -0,0 +1,163 @@ +--- +title: "Postgres Data Plane with Neon" +description: "Run gatekeeper's Postgres listener against a Neon project so clients connect with a run token instead of a database password." +keywords: ["gatekeeper", "postgres", "neon", "database proxy", "credential injection"] +--- + +# Postgres data plane with Neon + +Run gatekeeper's Postgres listener against a Neon project. Clients connect with their run token in place of the database password; gatekeeper resolves the real per-branch password from the Neon API and completes the upstream connection. + +See [Postgres Data Plane](../concepts/08-postgres-data-plane.md) for how the listener works internally. + +## Prerequisites + +- CA certificate generated ([CA Setup](./01-ca-setup.md)) — the Postgres listener requires it for TLS termination +- A Neon API key ([neon.tech/docs/manage/api-keys](https://neon.tech/docs/manage/api-keys)) +- A Neon project with at least one branch and compute endpoint + +## Configuration + +Add a `postgres` listener block and a credential with a `postgres` resolver to `gatekeeper.yaml`: + +```yaml +proxy: + host: 127.0.0.1 + port: 9080 + auth_token: local-test-token + +postgres: + port: 5432 + host: 127.0.0.1 + +tls: + ca_cert: ca.crt + ca_key: ca.key + +credentials: + - host: "*.neon.tech" + grant: neon-databases + postgres: + resolver: neon + source: + type: env + var: NEON_API_KEY + +network: + policy: strict + allow: + - "*.neon.tech" + +log: + level: info + format: text +``` + +| Field | Required | Description | +|-------------------|----------|---------------------------------------------------------------------------| +| `postgres.port` | Yes | Port for the Postgres-protocol listener | +| `postgres.host` | No | Bind address. Defaults to the same host as `proxy.host` | +| `credentials[].postgres.resolver` | Yes | `neon` or `static` | +| `credentials[].postgres.project` | No | Neon project ID; required for project-scoped API keys | + +Omit the top-level `postgres` block to run gatekeeper without the data plane. A `postgres` credential entry bypasses the `header`/`prefix`/`format` fields used for HTTP credential injection — the listener authenticates with SCRAM-SHA-256, not an HTTP header. + +Gatekeeper refuses to start if `postgres` is configured without `tls.ca_cert` and `tls.ca_key`: the listener terminates client TLS with a CA-minted certificate for the connection's SNI hostname, so there is no way to run it without a CA. + +## Account-scoped vs. project-scoped API keys + +Gatekeeper locates the project and branch that own a Neon endpoint before it can fetch that branch's password. How it does this depends on the API key's scope: + +- **Account-scoped key** — Gatekeeper lists the key's projects (`GET /api/v2/projects`) and searches each project's endpoints for a match. No further configuration needed. +- **Project-scoped key** — Neon's project-scoped keys cannot call `/api/v2/projects`; a listing attempt is rejected. Set `project` on the credential's `postgres` block so gatekeeper queries that project directly instead of enumerating: + +```yaml +credentials: + - host: "*.neon.tech" + grant: neon-databases + postgres: + resolver: neon + project: falling-river-38863773 + source: + type: env + var: NEON_API_KEY +``` + +With an account-scoped key and more than 99 projects, the project listing is not paginated — gatekeeper logs a warning and returns an error naming the endpoint if it isn't found on the first page. Setting `project` avoids the enumeration (and the pagination limit) entirely, so it's the better choice even for account-scoped keys once you know which project a credential targets. + +## How clients connect + +The client's Postgres password is gatekeeper's run token — the same token used for `Proxy-Authorization` on the HTTP plane. In standalone mode that's the static `proxy.auth_token`; nothing else about the connection string changes from a direct Neon connection except the password value. + +`sslmode=require` is mandatory. The run token is read only inside gatekeeper's TLS tunnel — a plaintext connection is refused before any credential is requested — so a client that skips TLS never gets to authenticate. + +```bash +PGPASSWORD=local-test-token psql \ + "host=ep-cool-darkness-123456.us-east-2.aws.neon.tech \ + dbname=neondb user=neondb_owner sslmode=require" +``` + +For this to reach gatekeeper instead of Neon directly, `ep-cool-darkness-123456.us-east-2.aws.neon.tech` must resolve to the host running gatekeeper. Arranging that DNS is outside gatekeeper's scope — an embedder typically does it per run. See the local-testing trick below if you don't want to touch DNS. + +## Local testing without DNS + +libpq accepts `host` and `hostaddr` as separate connection parameters. When both are set, `hostaddr` is used for the actual TCP connection — no DNS lookup occurs — while `host` still supplies the TLS Server Name Indication and the value checked during certificate verification. + +Gatekeeper's Postgres listener routes entirely on SNI, so this combination points the TCP connection at gatekeeper without editing `/etc/hosts` or resolvers, while still handing gatekeeper the real Neon endpoint hostname to route on: + +```bash +PGPASSWORD=local-test-token psql \ + "host=ep-cool-darkness-123456.us-east-2.aws.neon.tech \ + hostaddr=127.0.0.1 \ + dbname=neondb user=neondb_owner sslmode=require" +``` + +`host` never touches DNS here — it only sets SNI — so this works even for endpoint hostnames that don't resolve from your machine at all. + +## Static resolver alternative + +For a non-Neon Postgres server, or to pin a single fixed password instead of calling the Neon API, use `resolver: static`. The source supplies the password directly and gatekeeper fetches it once at startup: + +```yaml +credentials: + - host: db.internal + grant: internal-db + postgres: + resolver: static + source: + type: env + var: DB_PASSWORD +``` + +`static` does not re-fetch: rotating the password requires a restart, the same limitation as the `env`/`static`/secret-manager sources on the HTTP plane. + +## Verification + +Start gatekeeper: + +```bash +gatekeeper --config gatekeeper.yaml +``` + +Connect with `psql` (adjust `host`/`hostaddr`/`dbname`/`user` for your project): + +```bash +PGPASSWORD=local-test-token psql \ + "host=ep-cool-darkness-123456.us-east-2.aws.neon.tech \ + hostaddr=127.0.0.1 \ + dbname=neondb user=neondb_owner sslmode=require" \ + -c "select 1" +``` + +The proxy log confirms the connection and credential resolution: + +```text +level=INFO msg=request http_method=STARTUP http_host=ep-cool-darkness-123456.us-east-2.aws.neon.tech proxy_type=postgres user_id=neondb_owner credential_injected=true grants=postgres:ep-cool-darkness-123456.us-east-2.aws.neon.tech request_messages=4 response_messages=6 +``` + +> **Note:** Some Neon projects restrict access with an IP allowlist. If gatekeeper's egress IP isn't allowlisted, the upstream connection fails and the client sees a generic `could not authenticate to upstream database` error — identical to what a genuinely wrong password produces, since gatekeeper never echoes upstream failure detail to the client. Check the debug log (`postgres upstream connection failed`, logged with host and user but never the password) and the project's IP Allow settings in the Neon console. + +## Next steps + +- [Postgres Data Plane](../concepts/08-postgres-data-plane.md) — connection lifecycle, SNI routing, and security properties +- [Network Lockdown](./07-network-lockdown.md) — restrict proxy traffic to specific hosts diff --git a/docs/content/guides/14-process-credentials.md b/docs/content/guides/14-process-credentials.md new file mode 100644 index 0000000..c60b9ee --- /dev/null +++ b/docs/content/guides/14-process-credentials.md @@ -0,0 +1,128 @@ +--- +title: "Host Command Credentials (process source)" +description: "Run a host command and inject its output as a credential, so gatekeeper can use any secret manager that has a CLI." +keywords: ["gatekeeper", "process source", "credential_process", "1Password", "pass", "credential source"] +--- + +# Host command credentials (process source) + +Run a host command and use its trimmed stdout as the credential value. The `process` source is the escape hatch for secret managers gatekeeper has no dedicated integration for: any CLI that prints a credential — a password manager, an OS keychain tool, a corporate credential helper — can back a grant without writing Go code. + +## When to use it + +Use `process` when the credential lives behind a command-line tool rather than an API gatekeeper speaks natively (AWS Secrets Manager, GCP Secret Manager, GitHub Apps). This covers bring-your-own secret managers: 1Password, `pass`, a Vault CLI wrapper, or an internal `credential_process`-style helper. + +If the backend has a stable HTTP API, prefer a dedicated source ([AWS Secrets Manager](./03-aws-secrets-manager.md), [GCP Secret Manager](./04-gcp-secret-manager.md)) — those fetch over HTTPS directly instead of shelling out. + +## Prerequisites + +- CA certificate generated ([CA Setup](./01-ca-setup.md)) +- A command, runnable by the user gatekeeper runs as, that prints a credential to stdout and exits 0 + +## Configuration + +Add a `process` credential source to `gatekeeper.yaml`: + +```yaml +proxy: + host: 127.0.0.1 + port: 9080 + +tls: + ca_cert: ca.crt + ca_key: ca.key + +credentials: + - host: api.example.com + header: x-api-key + grant: example-api + source: + type: process + command: "op read op://vault/example/api-key" + ttl: 10m + +network: + policy: permissive + +log: + level: info + format: text +``` + +| Field | Required | Default | Description | +|-----------|----------|---------|----------------------------------------------------------------------| +| `command` | Yes | -- | Shell command, run with `sh -c`. Trimmed stdout becomes the credential | +| `ttl` | No | `5m` | Refresh interval when the output carries no expiry (Go duration string) | + +## Worked example: 1Password CLI + +Store the credential in a 1Password vault, then reference it by its `op://` path: + +```yaml +credentials: + - host: api.github.com + grant: github + source: + type: process + command: "op read op://Engineering/github-pat/credential" +``` + +`op` must be signed in (`op signin`) in whatever session gatekeeper's process inherits, or configured with a service account token via its own environment variable — gatekeeper only runs the command, it does not manage `op`'s auth state. + +## Worked example: pass + +[`pass`](https://www.passwordstore.org/) prints the decrypted secret to stdout: + +```yaml +credentials: + - host: api.example.com + header: x-api-key + grant: example-api + source: + type: process + command: "pass show api/example-api-key" +``` + +`pass` shells out to GPG, so gatekeeper's process needs access to the same `gpg-agent` (and cached passphrase, or an unattended key) that an interactive `pass show` would need. A command that blocks on a passphrase prompt blocks the credential fetch — and, on a refresh, the background refresh goroutine — until it times out. + +## Refresh semantics + +`process` implements background refresh the same way [GitHub App tokens](./05-github-app-tokens.md) do: gatekeeper re-runs the command at 75% of the credential's TTL (floored at 30 seconds) and hot-swaps the value without downtime. + +How the TTL is determined depends on the command's output: + +- **Expiry-aware.** If stdout is JSON matching the AWS `credential_process` shape — exact-case `Version`, `AccessKeyId`, and `Expiration` (RFC 3339) keys — gatekeeper reads `Expiration` and schedules the refresh from it. The configured `ttl` is ignored in this case. The JSON is passed through verbatim as the credential value; gatekeeper does not extract `AccessKeyId` or any other field for you. +- **Plain string or other JSON.** Anything else — a bare token, an API key, JSON without that exact key set — is treated as an opaque value with no expiry, and the configured `ttl` (default 5 minutes) sets the refresh interval. + +```bash +# Expiry-aware: gatekeeper schedules refresh from "Expiration", ignores ttl. +{"Version":1,"AccessKeyId":"...","SecretAccessKey":"...","Expiration":"2026-07-13T18:00:00Z"} + +# Opaque: gatekeeper refreshes every ttl (or every 5m by default). +ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +``` + +## Failure behavior + +**At startup**, a failing command fails gatekeeper's startup: the credential is fetched once before the proxy begins serving, and a non-zero exit or empty output is a fatal configuration error. + +**On refresh**, a failing command does not take gatekeeper down or drop the previously-injected credential — the last good value stays in place while the refresh loop retries with exponential backoff (starting at 1 second, capped at 60 seconds, with jitter). + +A few specific failure modes: + +- **Non-zero exit.** The fetch fails; stderr (truncated to 256 bytes) is included in the error so a diagnosable failure — an expired SSO session, a locked keychain — shows up in logs. Stdout is never included, since it may hold a partial credential. +- **Empty output.** Treated as an error even on exit 0. +- **Already-expired `Expiration`.** If the sniffed `credential_process` JSON reports an `Expiration` already in the past, the fetch fails instead of installing a credential that would be rejected upstream on first use. +- **Control characters in output.** Bytes invalid in HTTP header values (RFC 7230) are stripped automatically. If any non-whitespace control byte was present, gatekeeper logs a warning with a count — never the value — so a helper emitting garbage is diagnosable without exposing the credential. + +## Security notes + +> **Warning:** The command runs on the host with gatekeeper's own OS privileges, not the client's. Only configure commands from config files you trust — an embedder that lets an untrusted party supply `gatekeeper.yaml` (or just the `command` field) is granting that party arbitrary code execution as the gatekeeper process. + +- Don't echo secrets to logs. Gatekeeper never logs the fetched value, but a command that itself writes the credential to stderr (for debugging) will have that stderr echoed into gatekeeper's error log on failure — keep debug output out of commands used as credential sources. +- Prefer a command that reads from an already-unlocked store over one that prompts interactively. A prompt blocks the calling goroutine, and on a background refresh, there's no terminal attached to answer it. + +## Next steps + +- [GitHub App Tokens](./05-github-app-tokens.md) — a source with the same 75%-of-TTL refresh model, built in +- [Network Lockdown](./07-network-lockdown.md) — restrict proxy traffic to specific hosts diff --git a/docs/content/guides/15-gcp-service-account.md b/docs/content/guides/15-gcp-service-account.md new file mode 100644 index 0000000..77874a2 --- /dev/null +++ b/docs/content/guides/15-gcp-service-account.md @@ -0,0 +1,159 @@ +--- +title: "GCP Service Account Tokens" +description: "Mint short-lived GCP OAuth2 access tokens from a service account key and inject them into requests to Google APIs, with automatic background refresh." +keywords: ["gatekeeper", "GCP", "service account", "OAuth2", "credential source"] +--- + +# GCP service account tokens + +Mint short-lived OAuth2 access tokens from a GCP service account key and inject them into requests to Google APIs. Tokens refresh automatically in the background before they expire. + +Unlike [GCP Secret Manager](./04-gcp-secret-manager.md), which fetches a literal secret value once, this source holds a service account key and continuously exchanges it for fresh access tokens — the credential injected into requests is never the key itself. + +## Prerequisites + +- CA certificate generated ([CA Setup](./01-ca-setup.md)) +- A GCP service account with the IAM role(s) needed for the target API +- The service account's key JSON (`gcloud iam service-accounts keys create key.json --iam-account=...@...iam.gserviceaccount.com`), or that key JSON stored in GCP Secret Manager + +## Key location modes + +The service account key JSON can come from exactly one of three places: + +| Field | Where the key lives | +|---------------------|---------------------------------------------------------| +| `private_key_path` | A file on disk, read once at startup | +| `private_key_env` | An environment variable, read once at startup | +| `secret` + `project` | GCP Secret Manager, read lazily on first use and re-read on key rotation | + +Set exactly one. `secret` also requires `project` (the GCP project containing the secret); `project` and `version` are only valid alongside `secret`. + +### File or environment variable + +```yaml +credentials: + - host: storage.googleapis.com + grant: gcs + source: + type: gcp-service-account + private_key_path: ./gcs-uploader-key.json +``` + +```yaml +source: + type: gcp-service-account + private_key_env: GCS_SERVICE_ACCOUNT_KEY +``` + +```bash +export GCS_SERVICE_ACCOUNT_KEY="$(cat gcs-uploader-key.json)" +``` + +With either mode, the key is read once at startup and held in memory for the life of the process. It is not re-read — rotating the key file or environment variable requires a restart. + +### GCP Secret Manager + +```yaml +credentials: + - host: storage.googleapis.com + grant: gcs + source: + type: gcp-service-account + project: my-gcp-project + secret: gcs-uploader-key +``` + +The secret's payload must be the full service account key JSON. Reading it uses Application Default Credentials, same as the [GCP Secret Manager source](./04-gcp-secret-manager.md) directly. Unlike the file/env modes, this mode re-reads the secret when the key is rotated — see [Key Rotation](#key-rotation) below. + +## Scopes + +```yaml +source: + type: gcp-service-account + private_key_path: ./key.json + scopes: https://www.googleapis.com/auth/devstorage.read_write +``` + +`scopes` is a space-separated list of OAuth scope URIs requested when minting the token. It defaults to `https://www.googleapis.com/auth/cloud-platform`, which is broad — most Google Cloud APIs accept it, but scope the token down to what the target host actually needs when you know it (e.g. `devstorage.read_write` for Cloud Storage rather than the blanket `cloud-platform` scope). + +## How the token mint and refresh work + +Gatekeeper signs a JWT asserting the service account's identity (`iss`, requested `scope`, the token endpoint as `aud`) with the key's RSA private key, and exchanges it at the key's `token_uri` (from the key JSON, defaulting to `https://oauth2.googleapis.com/token`) for an access token using the `urn:ietf:params:oauth:grant-type:jwt-bearer` grant. The response's `expires_in` sets the token's TTL. + +Background refresh follows the same schedule as every other refreshing source in gatekeeper: **75% of TTL, floored at 30 seconds**. For a typical one-hour Google access token, that's a refresh roughly every 45 minutes. Refresh is atomic — requests see either the old token or the new one, never a partial state. + +## Key rotation + +When the token endpoint rejects the signed JWT with `400`, `401`, or `403`, that's a strong signal the key itself is stale (rotated or revoked, not just an expired token — tokens don't get JWT-assertion errors, keys do). Gatekeeper responds differently depending on where the key came from: + +- **`secret` mode.** The cached parsed key is dropped. The next fetch attempt re-reads the key JSON from Secret Manager, so a key rotated there is picked up without a gatekeeper restart. +- **`private_key_path` / `private_key_env` mode.** There is no backing source to re-read from — the key was loaded once from a file or environment variable at startup — so this drop-and-retry has no effect. Rotating a file- or env-sourced key requires a restart. + +> **Note:** This key-drop is scoped to the *token endpoint* rejecting the signing key — it does not react to the *target API* (e.g. `storage.googleapis.com`) returning 401/403 for an otherwise-valid token. Gatekeeper's proxy-level "evict a credential when the destination rejects it" mechanism ([Credential invalidation](../reference/03-credential-sources.md#credential-invalidation)) is wired up for `token-exchange` credentials only; a `gcp-service-account` token that a Google API rejects is not proactively evicted — the background refresh loop replaces it on its normal 75%-of-TTL schedule regardless. + +## Worked example: Cloud Storage + +```yaml +proxy: + host: 127.0.0.1 + port: 9080 + +tls: + ca_cert: ca.crt + ca_key: ca.key + +credentials: + - host: storage.googleapis.com + header: Authorization + grant: gcs + source: + type: gcp-service-account + project: my-gcp-project + secret: gcs-uploader-key + scopes: https://www.googleapis.com/auth/devstorage.read_write + +network: + policy: permissive + +log: + level: info + format: text +``` + +## IAM setup + +- The service account needs whatever IAM role the target API requires — for example `roles/storage.objectAdmin` for read/write access to Cloud Storage objects. Grant the narrowest role that covers the request pattern. +- If the key comes from Secret Manager (`secret` + `project`), the principal running gatekeeper additionally needs `roles/secretmanager.secretAccessor` on that secret — the same permission described in the [GCP Secret Manager guide](./04-gcp-secret-manager.md#iam-permissions). +- The key JSON itself grants whatever the service account can do — treat it as a credential. Prefer Secret Manager over a key file on disk where practical, since Secret Manager access is itself auditable and revocable without rotating the key. + +## Start the proxy + +```bash +gatekeeper --config gatekeeper.yaml +``` + +If the key JSON is malformed, missing required fields, or ADC is not configured for Secret Manager access, gatekeeper exits with an error at startup. + +## Verification + +```bash +curl --cacert ca.crt --proxy http://127.0.0.1:9080 \ + https://storage.googleapis.com/storage/v1/b/my-bucket/o +``` + +The proxy log confirms credential injection: + +```text +level=INFO msg=request http_host=storage.googleapis.com credential_injected=true grants=gcs +``` + +At debug level, refresh events appear: + +```text +level=DEBUG msg="credential refreshed" host=storage.googleapis.com grant=gcs ttl=1h0m0s +``` + +## Next steps + +- [GCP Secret Manager](./04-gcp-secret-manager.md) — fetch a literal secret value instead of minting OAuth2 tokens +- [Network Lockdown](./07-network-lockdown.md) — restrict proxy traffic to specific hosts diff --git a/docs/content/guides/16-mcp-relay.md b/docs/content/guides/16-mcp-relay.md new file mode 100644 index 0000000..6203086 --- /dev/null +++ b/docs/content/guides/16-mcp-relay.md @@ -0,0 +1,169 @@ +--- +title: "MCP Relay Setup" +description: "Configure gatekeeper's MCP relay end-to-end using the Go library, since MCP servers are not configurable in gatekeeper.yaml." +keywords: ["gatekeeper", "MCP relay", "model context protocol", "Go library", "SSE streaming"] +--- + +# MCP relay setup + +Relay Model Context Protocol (MCP) requests to remote MCP servers with credential injection. This guide walks through configuring the relay end-to-end. For how the relay works internally — request flow, SSE mechanics, the two credential-injection modes — see [MCP Relay](../concepts/05-mcp-relay.md). + +> **Note:** MCP servers are not configurable in `gatekeeper.yaml`. `MCPServerConfig` is a Go-library type, set via `Proxy.SetMCPServers` or `RunContextData.MCPServers` — there is no `mcp:` section in the standalone config file. This is how [moat](https://github.com/majorcontext/moat)'s daemon layer wires up MCP servers per run. This guide shows the Go-library path, which is the only path that exists. + +## Prerequisites + +- Go 1.25+ +- Familiarity with embedding gatekeeper as a library ([Go Library Usage](./09-go-library.md)) + +## Minimal setup + +Create a proxy, register an MCP server, and supply a credential store that resolves grants to values: + +```go +package main + +import ( + "fmt" + "log" + "net/http" + + "github.com/majorcontext/gatekeeper/proxy" +) + +// credStore looks up MCP grant credentials by name. +type credStore struct { + tokens map[string]string +} + +func (s *credStore) GetToken(grant string) (string, error) { + token, ok := s.tokens[grant] + if !ok { + return "", fmt.Errorf("no credential for grant %q", grant) + } + return token, nil +} + +func main() { + p := proxy.NewProxy() + + p.SetCredentialStore(&credStore{ + tokens: map[string]string{ + "mcp-context7": "real-api-key-123", + }, + }) + + p.SetMCPServers([]proxy.MCPServerConfig{ + { + Name: "context7", + URL: "https://mcp.context7.com/mcp", + Auth: &proxy.MCPAuthConfig{ + Grant: "mcp-context7", + Header: "Authorization", + }, + }, + }) + + log.Fatal(http.ListenAndServe("127.0.0.1:9080", p)) +} +``` + +`SetCredentialStore` and `SetMCPServers` are the standalone/single-run fallback: the proxy checks `RunContextData` first (daemon mode) and falls back to these proxy-level values when no per-run context is set. `proxy.Proxy` implements `http.Handler`, so this needs no CA and no TLS interception — the relay endpoint is a plain HTTP handler on the proxy's own listener. + +## Sending a request + +An MCP client sends requests directly to the relay endpoint, bypassing `HTTP_PROXY`: + +```bash +curl -X POST http://127.0.0.1:9080/mcp/context7/v1/endpoint \ + -H "Content-Type: application/json" \ + -d '{"method": "tools/list"}' +``` + +Gatekeeper: + +1. Matches `context7` against the configured `MCPServerConfig` entries. +2. Builds the target URL from the server's `URL` field, appending any path after `/mcp/context7` (here, `/v1/endpoint`) and preserving the query string. +3. Resolves the credential for grant `mcp-context7` — first from `RunContextData.Credentials` (daemon mode), then from the `CredentialStore` — and sets it on the configured `Auth.Header`. +4. Forwards the request to `https://mcp.context7.com/mcp/v1/endpoint` and streams the response back. + +Path building trims exactly the `/mcp/{server-name}` prefix and appends what remains to the server's configured URL path (trimming a trailing slash first), so `/mcp/context7` (no sub-path) forwards to the bare `https://mcp.context7.com/mcp`. + +## Multi-tenant setup (RunContextData) + +For a daemon that scopes MCP servers and credentials per caller, populate `RunContextData.MCPServers` and `RunContextData.Credentials` (or `RunContextData.CredStore`) inside a `ContextResolver` instead of calling `SetMCPServers`/`SetCredentialStore` on the proxy directly: + +```go +p.SetContextResolver(func(token string) (*proxy.RunContextData, bool) { + run, ok := registry.Get(token) + if !ok { + return nil, false + } + return &proxy.RunContextData{ + RunID: run.ID, + MCPServers: []proxy.MCPServerConfig{ + { + Name: "context7", + URL: "https://mcp.context7.com/mcp", + Auth: &proxy.MCPAuthConfig{ + Grant: "mcp-context7", + Header: "Authorization", + }, + }, + }, + Credentials: map[string][]proxy.CredentialHeader{ + "mcp.context7.com": { + {Name: "Authorization", Value: "Bearer " + run.Context7Key, Grant: "mcp-context7"}, + }, + }, + }, true +}) +``` + +Each run's `MCPServers` list and credentials are isolated — `getMCPServersForRequest` reads `RunContextData.MCPServers` when a `ContextResolver` resolved the run, and only falls back to the proxy-level list set by `SetMCPServers` when no run context is present. See [Go Library Usage](./09-go-library.md#custom-contextresolver) for the full `ContextResolver` pattern. + +### Daemon-mode direct access + +When a `ContextResolver` is set, gatekeeper also serves `/mcp/{token}/{server-name}[/path]` for clients that connect directly (not through the proxy mechanism) and therefore cannot send a `Proxy-Authorization` header. The run's proxy auth token is embedded in the URL instead; gatekeeper extracts it, resolves the run, strips the token from the path, and dispatches to the same relay handling described above. Standalone gatekeeper (no `ContextResolver`) never serves this form — see [MCP Relay](../concepts/05-mcp-relay.md#daemon-mode-token-embedded-path) for the full mechanics. + +## Credential injection modes + +The relay endpoint above is one of two ways credentials reach an MCP server. + +**Relay mode** (`/mcp/{server-name}`, shown above): the client talks to gatekeeper's relay endpoint directly. Gatekeeper resolves the grant and sets the header on the outgoing request unconditionally. + +**Stub-replacement mode**: when an MCP client instead routes through the CONNECT proxy (`HTTP_PROXY`) to an MCP server's real hostname, gatekeeper inspects the configured `Auth.Header` on the intercepted request. If its value exactly matches `moat-stub-{grant}`, gatekeeper replaces it with the real credential; any other value (including a credential the client already holds) is left untouched. This mode requires a CA — TLS interception is what lets gatekeeper read and rewrite the header. See [CA Setup](./01-ca-setup.md). + +```bash +# Client sends a stub value through the CONNECT proxy: +curl --cacert ca.crt --proxy http://127.0.0.1:9080 \ + -H "Authorization: moat-stub-mcp-context7" \ + https://mcp.context7.com/mcp/v1/endpoint +``` + +Gatekeeper matches the request host against a configured server's `URL` host, sees the stub value, and swaps in the real credential before forwarding. + +### oauth: grant prefix + +When a grant name starts with `oauth:` (e.g., `oauth:notion`), gatekeeper prepends `Bearer ` to the resolved credential value automatically, in both modes. Grants that don't start with `oauth:` are injected as-is — set `Header` to `Authorization` and give the credential value its own scheme prefix if one is needed. + +## SSE streaming + +MCP responses that use Server-Sent Events stream through the relay as they arrive: gatekeeper reads the upstream body in 4096-byte chunks and flushes the `http.ResponseWriter` after each one, rather than buffering the full response. The relay's HTTP client has no client-level timeout, since MCP SSE connections may stay open indefinitely. + +## Error handling + +| Status | Cause | +|--------|-------| +| `400` | Direct daemon-mode request (`/mcp/{token}/{server}`) with no server name after the token | +| `401` | Direct daemon-mode request with an unrecognized proxy auth token | +| `404` | `{server-name}` doesn't match any configured `MCPServerConfig` | +| `403` | A `mcp-{server-name}` Keep policy denied the request, or the request body failed fail-closed inspection (non-JSON, or a redaction step failed) — see [Keep Policy Scopes](./18-keep-policy-scopes.md) | +| `500` | The server's configured `URL` failed to parse, reading the request body for Keep evaluation failed, the credential for the grant resolved empty, or building the outgoing request failed | +| `502` | Gatekeeper reached out to the real MCP server and the connection failed | + +The `404` and `500` bodies include diagnostic detail (available server count, the grant name, a suggested `moat grant` command) meant for an operator or coding agent reading the response, not for display to an end user. + +## Next steps + +- [Keep Policy Scopes](./18-keep-policy-scopes.md) — evaluate `tools/call` requests against policy before forwarding +- [Credential Caching, Refresh, and Invalidation](./17-credential-lifecycle.md) — how the grants referenced here are resolved and rotated diff --git a/docs/content/guides/17-credential-lifecycle.md b/docs/content/guides/17-credential-lifecycle.md new file mode 100644 index 0000000..1b4ebdb --- /dev/null +++ b/docs/content/guides/17-credential-lifecycle.md @@ -0,0 +1,111 @@ +--- +title: "Credential Caching, Refresh, and Invalidation" +description: "How gatekeeper fetches, caches, refreshes, and evicts credentials across their lifecycle, from startup through 401/403-triggered invalidation." +keywords: ["gatekeeper", "credential refresh", "credential caching", "invalidation", "backoff", "token exchange"] +--- + +# Credential caching, refresh, and invalidation + +Credentials move through several distinct lifecycle stages depending on source type: a one-time fetch at startup, a background refresh loop, a per-request cache, or an eviction triggered by an upstream rejection. This guide consolidates the timing and failure behavior of each stage in one place. For the source types themselves, see [Credential Sources](../reference/03-credential-sources.md) (reference) and [Sources](../concepts/03-credential-sources.md) (concepts); for token exchange specifically, see [Token Exchange](./06-token-exchange.md). + +## Prerequisites + +- A working gatekeeper configuration with at least one credential ([CA Setup](./01-ca-setup.md)) + +## Lifecycle overview + +| Stage | Applies to | Trigger | Behavior | +|-------|-----------|---------|----------| +| Startup fetch | All sources | Server startup | Single `Fetch`, 10s timeout, failure is fatal | +| Background refresh | `RefreshingSource` (`process`, `github-app`, `gcp-service-account`) | Timer at 75% of TTL, floor 30s | Hot-swaps the credential; failures retry with backoff | +| Per-request cache | `token-exchange` | Cache miss on incoming request | Exchanges with the STS, caches up to 1 minute, coalesced via singleflight | +| Invalidation | Sources with an `Invalidate` hook (`token-exchange`) | Upstream `401`/`403` on an injected credential | Drops the cache entry so the next request re-resolves; rate-limited to one eviction per key per 10s | + +## Startup: fetch once, fail fast + +Every credential in `gatekeeper.yaml` is fetched once when the server starts, with a 10-second timeout per fetch. If any `Fetch` call fails or times out, gatekeeper refuses to start: + +```yaml +credentials: + - host: api.github.com + grant: github + source: + type: github-app + app_id: "12345" + installation_id: "67890" + private_key_path: /etc/gatekeeper/github-app.pem +``` + +If the GitHub App's private key file is missing or the installation token request fails, the process exits with an error naming the host — it does not start with a missing credential and fail requests later. `static` and `env` sources resolve instantly; `aws-secretsmanager`, `gcp-secretmanager`, `process`, `github-app`, and `gcp-service-account` all make a network or subprocess call during this window, so a slow or unreachable backend delays startup up to the 10s timeout per credential. + +Credentials that share an identical `source` block (same type and fields) are deduplicated: one `Fetch`, applied to every host that references it. See [Source Deduplication](../concepts/03-credential-sources.md#source-deduplication). + +## Background refresh + +Sources whose credentials expire implement `RefreshingSource`, adding a `TTL() time.Duration` method to `CredentialSource`. Gatekeeper schedules a background refresh goroutine per unique source: + +- **Refresh interval:** 75% of the reported TTL, floored at 30 seconds. A GitHub App installation token (1-hour TTL) refreshes every 45 minutes. A `process` source with no expiry-aware output and the default 5-minute `ttl` refreshes every 3m45s. +- **Hot-swap:** the refreshed value replaces the credential on the proxy immediately (`SetCredentialWithGrant`) for every host sharing that source. In-flight requests already holding the old value are unaffected; the next request picks up the new one. +- **Failure backoff:** on a failed refresh, gatekeeper retries starting at 1 second, doubling on each subsequent failure, capped at 60 seconds. A random jitter of up to 25% of the (already-capped) backoff is added on top, so a run of consecutive failures can wait up to ~75 seconds between attempts, not a strict 60s ceiling. The previous credential value stays in use throughout — a failed refresh never blanks a working credential. + +```yaml +credentials: + - host: api.example.com + header: x-api-key + grant: helper + source: + type: process + command: "op read op://vault/example/api-key" + ttl: 10m +``` + +With `ttl: 10m` and no expiry-aware JSON output, this refreshes every 7m30s; a failing `op` invocation (an expired session, say) retries at 1s, 2s, 4s... up to the ~60-75s ceiling until the helper succeeds again. + +### gcp-service-account and github-app: two different rotation behaviors + +Both are `RefreshingSource`s with a one-hour token TTL, but they react differently to a rejection: + +- **github-app** parses `expires_at` from each GitHub API response and schedules the next refresh from it. There is no key-rotation-specific handling beyond the standard backoff — a rejected JWT (e.g., the App's private key was revoked) simply keeps failing and retrying. +- **gcp-service-account** additionally detects a rejected assertion — but only in Secret Manager mode (`secret` + `project`): if the GCP token endpoint returns `400`, `401`, or `403` for the signed JWT, gatekeeper drops its in-memory copy of the parsed service account key and re-reads it from Secret Manager on the next attempt, picking up a rotated key without a restart as long as the *new* key was already written there. Keys loaded from a file (`private_key_path`) or environment variable (`private_key_env`) have no backing source to re-read — rotating them requires a restart. A `429` or `5xx` from the token endpoint is treated as transient and does not drop the cached key. See [GCP Service Account Tokens](./15-gcp-service-account.md) for the full rotation semantics. + +This distinction only matters when the key itself rotates (e.g., a new GCP service account key replaces the old one in Secret Manager); for both sources, an expired *token* is handled by the ordinary refresh-before-expiry schedule, not by this fallback path. + +## Per-request caching: token exchange + +`token-exchange` does not fit the fetch-once/refresh-on-a-timer model above — it resolves a credential per request, scoped to the caller's identity, and has its own cache: + +- Tokens are cached per `(subject_token, actor_token)` pair. +- Cache TTL is the STS's `expires_in`, **capped at 1 minute** regardless of what the STS advertises. If `expires_in` is `0` or omitted, the 1-minute cap is used directly. +- Concurrent requests for the same subject are coalesced into a single STS call via singleflight — the 1-minute cap does not mean an STS call every minute per active subject, it means at most one exchange per subject per minute even under concurrent load. +- There is no proactive refresh: a cache miss (first request, or the entry aged out) triggers a synchronous exchange on that request's path. + +See [Token Exchange: Caching Behavior](./06-token-exchange.md#caching-behavior) for the full mechanics and the reasoning behind the 1-minute cap. + +## Invalidation on 401/403 + +When the destination rejects a forwarded request with `401 Unauthorized` or `403 Forbidden`, gatekeeper evicts every credential that was injected into that specific request — not every credential configured for the host, only the ones this request actually used. The next request re-resolves rather than replaying a credential the destination just refused. + +This is evict-only: the failed request itself is never retried. Its body may have already been consumed by the time the response arrives, and the operations that surface this (a `git push`, an API mutation) are frequently not safe to replay automatically. + +Only sources that expose an `Invalidate` hook participate. Concretely, that means `token-exchange` today — eviction drops the cached token for that `(subject_token, actor_token)` pair. Static sources (`env`, `static`) and sources fetched once at startup with no cache to drop have no `Invalidate` hook and are unaffected by a 401/403; the value simply stays as configured until the process restarts or a background refresh (for `RefreshingSource`s) replaces it on its own schedule. + +Evictions are rate-limited to one per cache key per 10 seconds. A client looping on a request that fails for a reason unrelated to a stale credential (a secondary rate limit, a permissions gap) triggers at most one eviction, and therefore at most one extra STS call, per 10-second window — not one per failed request. The trade-off: a genuinely rotated credential can take up to 10 seconds after the first eviction to be picked up, since evictions inside the cooldown are no-ops. + +A `5xx` response is left alone entirely — it says nothing about whether the injected credential is valid. + +## Summary table + +| Source type | Startup fetch | Background refresh | Per-request cache | Participates in 401/403 invalidation | +|---|---|---|---|---| +| `env`, `static` | Yes | No | No | No | +| `aws-secretsmanager`, `gcp-secretmanager` | Yes | No | No | No | +| `process` | Yes | Yes (75% of TTL, floor 30s) | No | No | +| `github-app` | Yes | Yes (75% of TTL, floor 30s) | No | No | +| `gcp-service-account` | Yes | Yes (75% of TTL, floor 30s); Secret Manager mode also drops the cached key on assertion rejection | No | No | +| `token-exchange` | No (resolved per request) | No | Yes (STS `expires_in`, capped at 1 minute) | Yes (10s cooldown per subject/actor key) | + +## Next steps + +- [Credential Sources](../reference/03-credential-sources.md) — field-by-field reference for every source type +- [Token Exchange](./06-token-exchange.md) — configuring the per-request resolver and its STS contract +- [MCP Relay Setup](./16-mcp-relay.md) — credentials referenced by MCP server grants follow the same lifecycle diff --git a/docs/content/guides/18-keep-policy-scopes.md b/docs/content/guides/18-keep-policy-scopes.md new file mode 100644 index 0000000..45c8074 --- /dev/null +++ b/docs/content/guides/18-keep-policy-scopes.md @@ -0,0 +1,136 @@ +--- +title: "Keep Policy Scopes" +description: "The three Keep policy scopes gatekeeper evaluates — http, mcp-, and llm-gateway — and how to read their denial logs." +keywords: ["gatekeeper", "Keep", "policy", "scopes", "http", "mcp", "llm-gateway"] +--- + +# Keep policy scopes + +Gatekeeper evaluates [Keep](https://github.com/majorcontext/keep) policy engines at three distinct points in the request lifecycle, keyed by three distinct scope names: `http`, `mcp-`, and `llm-gateway`. They inspect different things, deny in different ways, and fail closed under different conditions. This guide is for two audiences: embedders (like moat) wiring engines into `RunContextData.KeepEngines`, and operators reading policy denial logs and trying to tell which layer produced them. + +> **Note:** None of this is configurable in `gatekeeper.yaml`. `KeepEngines` is a field on `RunContextData` — a Go-library type — populated by whatever constructs the struct returned from a `ContextResolver`. Standalone gatekeeper never sets it; this feature exists only for embedders. See [Go Library Usage](./09-go-library.md). + +## The KeepEngines map + +```go +type RunContextData struct { + // ... + KeepEngines map[string]*keeplib.Engine + // ... +} +``` + +There is no setter method — an embedder populates the map directly when building the `RunContextData` it returns from its `ContextResolver`. The map is keyed by scope name, and gatekeeper looks up exactly three keys: + +| Key | Looked up | Evaluates | +|---|---|---| +| `"http"` | On every intercepted HTTPS request, after network policy allows it | The request itself (method, host, path, body) | +| `"mcp-" + serverName` | On `POST` requests to `/mcp/{serverName}` whose body is a `tools/call` | The tool name and arguments | +| `"llm-gateway"` | On `200` responses from `api.anthropic.com` | The response body (JSON or SSE) | + +A run with no entry for a given key simply skips that check — Keep evaluation is opt-in per scope, not a blanket policy. A run can set only `"llm-gateway"`, only `"mcp-github"`, all three, or none. + +## `http` scope + +Runs after [network policy](../concepts/04-network-policy.md) allows a request, on the plaintext request produced by TLS interception. Unlike network policy, it can inspect the request body, not just the host. + +**Denial:** `403 Forbidden`, header `X-Moat-Blocked: keep-policy`, `Content-Type: text/plain`, a plaintext body naming the host (and, for evaluation-error denials, a generic "policy evaluation error" — the specific evaluator error is not echoed to the client). + +**Fail-closed conditions.** The engine's rules determine whether a body is needed at all (`eng.RequiresBody("http")`); when it is, gatekeeper reads and parses it before evaluation, and any of the following denies the request rather than passing it through unevaluated: + +- `Content-Encoding` is set on the request (e.g., `gzip`) — the body is not decompressed here, so a compressed body can't be inspected and is denied outright. This differs from the `llm-gateway` scope below, which does decompress gzip responses. +- `Content-Type` is not `application/json` or does not end in `+json`. +- The body exceeds 10 MiB (10,485,760 bytes). +- The body is not valid JSON. +- The body has duplicate keys in a JSON object at any nesting depth (e.g., two `"model"` keys) — rejected because `encoding/json` silently keeps the last value, but a downstream parser that keeps the first would see a different value than the one the policy evaluated, a bypass vector. + +A `nil` or zero-`Content-Length` body, or a body that is empty/whitespace after trimming, is treated as body-less rather than a failure — path-only rules still apply, body rules just don't match. + +## `mcp-` scope + +Evaluated inside the [MCP relay](./16-mcp-relay.md) handler, before the request is forwarded to the real MCP server, and only for a `tools/call` method with a non-empty tool name. Requests for other MCP methods (`tools/list`, `initialize`, etc.) are not evaluated even when an engine is configured. + +**Decisions:** + +- `Deny` — the request is blocked with `403 Forbidden` and a plaintext body. If the engine set a message, it's appended to the client-facing response. +- `Redact` — the engine's mutation rules rewrite the tool call's `arguments`, the request body is re-encoded with the mutated arguments, and the modified request is forwarded. A failure at any step of re-encoding (unmarshal, missing `params` map, marshal) denies with `403` rather than forwarding a partially-mutated or original body. +- Anything else (allow) — forwarded unchanged. + +**Fail-closed condition:** once an engine is configured for a server, *every* request body to that server must parse as JSON, not just `tools/call` bodies — gatekeeper reads and unmarshals the body before it can even check the method name, and denies with `403` if that unmarshal fails. Only a body that does parse as JSON goes on to the method check above, where non-`tools/call` requests (and `tools/call` requests with an empty tool name) pass through without Keep evaluation. + +> **Note:** This scope does not set `X-Moat-Blocked`. All of its denial responses use a plain `http.Error` call with no policy header, unlike the `http` and `llm-gateway` scopes below. Don't rely on the header's presence to detect an MCP-scope denial in logs or client-side handling — use the response body and the `mcp.tool_call` operation in the policy log instead (see [Observability](#observability)). + +## `llm-gateway` scope + +Evaluated on the response side, inside `ModifyResponse`, and gated on two hardcoded conditions: the response status must be exactly `200`, and the request host must be exactly `api.anthropic.com` (a literal string comparison, not a configurable pattern). A non-200 response, or a response from any other host, is never evaluated even when `KeepEngines["llm-gateway"]` is set. + +**Denial:** `400 Bad Request` (not `403` — this scope replaces the response body rather than blocking a forward-in-progress), `Content-Type: application/json`, header `X-Moat-Blocked: llm-policy`, JSON body: + +```json +{ + "type": "error", + "error": { + "type": "policy_denied", + "message": "Policy denied: rule-name. Human-readable message." + } +} +``` + +This matches the error shape Keep's own LLM gateway uses, so a client like Claude Code handles a gatekeeper denial the same way it handles a Keep-gateway denial. + +**Fail-closed conditions:** + +- The response body exceeds 10 MiB (10,485,760 bytes) — denied with rule `size-limit` before evaluation runs. +- The body can't be read from the upstream connection — rule `read-error`. +- The body is `Content-Encoding: gzip` and decompression fails — rule `evaluation-error`. Unlike the `http` scope, a gzip-compressed body here *is* decompressed and evaluated on success; Claude Code sends `Accept-Encoding: gzip`, and Go's transport does not auto-decompress a response the client explicitly asked to receive compressed, so gatekeeper does it itself before evaluation. +- The response is SSE (`Content-Type: text/event-stream`) and the stream fails to parse — rule `evaluation-error`. SSE events are read up to and including `message_stop`; anything after that (pings, keepalives) is not policy-relevant and isn't parsed. +- The Keep evaluation call itself errors — rule `evaluation-error`. + +A `Deny` decision from the engine uses the rule and message the engine returned, tagged with operation `llm.tool_use` in the policy log (see below). + +This scope is documented in full in [LLM policy](../reference/05-llm-policy.md), including the codec and evaluation-flow details this guide doesn't repeat. + +## Observability + +Every denial across all three scopes calls the same policy logger, configured once via `Proxy.SetPolicyLogger`. Gatekeeper's own standalone binary wires it to a `slog.Warn` call with exactly five fields: + +| Field | Description | +|---|---| +| `run_id` | The run's ID from `RunContextData`, empty in standalone mode | +| `scope` | `"http"`, `"mcp-"`, or `"llm-gateway"` (also `"network"` for network-policy denials, which are a separate layer — see [Network Policy](../concepts/04-network-policy.md)) | +| `operation` | See the table below | +| `rule` | The Keep rule name that triggered the denial, or an internal reason like `evaluation-error`/`size-limit`/`read-error` for a fail-closed denial that never reached rule evaluation | +| `message` | Human-readable detail from the engine or the fail-closed condition | + +This logger fires only on denials — there is no corresponding "allow" log line from this path. Allowed requests still appear in the canonical per-request log (`RequestLogData`), which is a separate log stream covering every request regardless of policy outcome. + +### scope / operation combinations + +| scope | operation | rule examples | +|---|---|---| +| `network` | `http.request` / `http.connect` | (empty — network policy denials carry no rule name) | +| `http` | `http.request` | Keep rule name, or `body-inspection-error` / `evaluation-error` | +| `mcp-` | `mcp.tool_call` | Keep rule name, or `evaluation-error` / `redaction-error` | +| `llm-gateway` | `llm.tool_use` | Keep rule name | +| `llm-gateway` | `llm.read_error` | `read-error` | +| `llm-gateway` | `llm.response_too_large` | `size-limit` | + +### OTel signals + +Each denial also, when a trace context is present, adds a `policy.denial` span event (attributes: `scope`, `operation`, `rule`, `message`) and increments the `proxy.policy.denials` counter with `proxy.policy.scope` and `proxy.policy.rule` attributes. See [OpenTelemetry](./08-opentelemetry.md). + +### X-Moat-Blocked header values + +| Value | Scope/layer | Status | +|---|---|---| +| `request-rule` | Network policy | `407` | +| `host-service` | Host gateway | `407` | +| `keep-policy` | `http` scope | `403` | +| `llm-policy` | `llm-gateway` scope | `400` | +| *(none)* | `mcp-` scope | `403` | + +## Next steps + +- [LLM policy](../reference/05-llm-policy.md) — full reference for the `llm-gateway` scope +- [Network Policy](../concepts/04-network-policy.md) — how the `http` scope relates to host/path-level network policy +- [MCP Relay Setup](./16-mcp-relay.md) — configuring the servers that the `mcp-` scope evaluates