Unbounded fastmcp constraint: image rebuilds pick up FastMCP 3.4.3 host-origin protection, kai-api returns 421 for all proxied requests

[ibolton336]

Summary

Since July 8, the kai-api MCP server (quay.io/konveyor/kai-solution-server:latest) returns HTTP 421 Misdirected Request for every request when deployed behind a proxy/ingress (minikube, and likely OpenShift routes). The server is completely unreachable through the ingress.

Root cause

Two compounding problems:

1. The image build ignores the lockfile. tools/deploy/Containerfile runs pip3.12 install --no-cache-dir . against pyproject.toml only — requirements.txt / uv.lock (which pin fastmcp==2.14.3, mcp==1.25.0) are never copied into the image. pyproject declares only a floor: fastmcp>=2.8.0, so every rebuild resolves the newest fastmcp from PyPI.
2. FastMCP 3.4.3 enabled host/origin (DNS-rebinding) protection by default — http_host_origin_protection: bool = True with http_allowed_hosts: None, i.e. an effectively empty allowlist. The security middleware validates the Host header and returns 421 "Invalid Host header" on mismatch. Same failure mode hit other projects the same week: modelcontextprotocol/python-sdk#1798, CrowdStrike/falcon-mcp#291, blockscout/mcp-server#333, write-up.

The merge of #932 triggered the fresh :latest build that picked up fastmcp ≥3.4.3 — but the dependency bumps in #932 are themselves incidental; any merge would have produced the same broken image. (Earlier drafts of this issue blamed the starlette bump; that was wrong — starlette has no 421 code path.)

Behind a proxy, the forwarded Host header is the external host/IP (e.g. 192.168.49.2 via tackle-hub's /hub/services/kai/api route), which never matches the empty/localhost allowlist → every request is rejected before reaching the MCP session layer.

Evidence

kai-api pod logs — the server itself returns 421, not nginx:

INFO:     Starting MCP server 'KaiSolutionServer' with transport 'streamable-http' on http://0.0.0.0:8000/api
INFO:     10.244.0.16:45922 - "POST /api HTTP/1.1" 421 Misdirected Request
INFO:     10.244.0.16:37090 - "POST /api HTTP/1.1" 421 Misdirected Request
... (every request, deterministic)

Impact

• All editor-extensions CI infrastructure tests (@requires-minikube) are broken
• Any deployment behind an ingress/route that preserves the external Host header is unreachable

Reproduction

1. Deploy kai-api via the konveyor operator on minikube with ingress enabled
2. curl -k -X POST https://$(minikube ip)/hub/services/kai/api -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","method":"initialize","id":0,"params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"0.0.0"}}}'
3. Observe HTTP 421

Suggested fixes (increasing durability)

• Immediate mitigation for existing deployments: set FASTMCP_HTTP_HOST_ORIGIN_PROTECTION=false on the kai-api container
• Make the image reproducible: install from the lock (uv sync --frozen, or copy requirements.txt into the image and pip install -r) instead of resolving floors at build time
• Cap the constraint (fastmcp>=2.8.0,<3) until a deliberate 3.x migration, and/or configure http_allowed_hosts explicitly for proxied deployments

[ibolton336]

Warning

Superseded — this analysis was wrong. Starlette is not the cause (it has no 421 code path), and the CVE cited below doesn't check out. See the corrected root cause in the issue body and the follow-up comment: the unbounded fastmcp>=2.8.0 constraint let the image rebuild pick up FastMCP 3.4.3's host-origin protection. Original text kept below for history.

Root Cause Confirmed

The starlette bump 0.51.0 → 1.3.1 in #932 brought in the fix for CVE-2026-48710 ("BadHost"). This introduced DNS rebinding protection in the MCP Python SDK's StreamableHTTP transport — the transport_security.py module now validates the Host header and returns Response("Invalid Host header", status_code=421) when it doesn't match the server's allowed hosts.

When deployed behind an nginx ingress (minikube or OpenShift), the forwarded Host header is the external IP/hostname (e.g. 192.168.49.2), which doesn't match the kai-api's listening address (0.0.0.0:8000). Every request is rejected.

This is a known issue in the MCP Python SDK: modelcontextprotocol/python-sdk#1798.

Fix

The kai-api MCP server needs to be configured with the correct allowed origins/hosts, or the DNS rebinding protection needs to be relaxed for proxy deployments. Options:

• Pass allowed_origins when creating the StreamableHTTPTransport to include the ingress host
• Set the FASTMCP_ALLOWED_HOSTS environment variable in the kai-api deployment
• Or use the MCP SDK's mechanism to disable host validation for behind-proxy scenarios

[ibolton336]

Correction after deeper investigation: starlette is not the cause. Starlette has no 421 code path at all. The actual root cause is an unbounded fastmcp constraint combined with a FastMCP 3.4.3 behavior change:

1. The solution-server image build (tools/deploy/Containerfile) runs pip3.12 install --no-cache-dir . against pyproject.toml only — requirements.txt / uv.lock (which pin fastmcp==2.14.3, mcp==1.25.0) are never copied into the image. pyproject declares only a floor: fastmcp>=2.8.0.
2. FastMCP 3.4.3 enabled host/origin (DNS-rebinding) protection by default (http_host_origin_protection: bool = True, http_allowed_hosts: None → effectively empty allowlist). See e.g. https://dev.to/toyama0919/how-an-unbounded-fastmcp-version-constraint-took-down-production-with-421-misdirected-request-1mh1 and Guide: Resolving "421 Invalid Host Header" (DNS Rebinding Protection) modelcontextprotocol/python-sdk#1798 for the identical failure mode in other projects.
3. PR Bump dependencies to remediate CVEs (aiohttp, starlette, pyjwt, urllib3, python-dotenv, python-multipart) #932's merge triggered a fresh :latest image build, which resolved fastmcp to ≥3.4.3. Any merge would have had the same effect — the dependency bumps in Bump dependencies to remediate CVEs (aiohttp, starlette, pyjwt, urllib3, python-dotenv, python-multipart) #932 are incidental.
4. In deployments behind a proxy (e.g. tackle-hub routing /hub/services/kai/api), the forwarded Host header is the external host/IP, which never matches the empty/localhost allowlist → the security middleware rejects every request with 421 Misdirected Request / "Invalid Host header" before it reaches the MCP session layer. This is exactly what the uvicorn access logs show.

Fixes, in increasing order of durability:

• Immediate mitigation for deployments: set FASTMCP_HTTP_HOST_ORIGIN_PROTECTION=false on the kai-api container.
• Make the image build reproducible: install from the lock (uv sync --frozen or copy requirements.txt and pip install -r) instead of resolving floors at build time.
• Cap the constraint (fastmcp>=2.8.0,<3) until the 3.x migration is done deliberately, and/or configure http_allowed_hosts explicitly for proxied deployments.

Happy to retitle this issue to reflect the corrected root cause.

[ibolton336]

Fix opened: #937 — builds the image from requirements.txt (the lock) instead of resolving pyproject floors, so it stops drifting onto fastmcp 3.4.3.

Two clarifications from testing the failure end to end:

• The default flipped between patch releases: fastmcp 3.4.3 sets http_host_origin_protection = True; 3.4.4 reverts it to False. The :latest image was built before 3.4.4 existed, so it baked in 3.4.3 with the guard on. The 421 comes from fastmcp's own HostOriginGuardMiddleware (Response("Misdirected Request", status_code=421)), not the MCP SDK's ("Invalid Host header").
• A deployment-level env override does not work in the konveyor-operator setup: kubectl set env deployment/kai-api FASTMCP_HTTP_HOST_ORIGIN_PROTECTION=false gets reconciled back to the CR spec by the operator. So the durable fix has to be in the image (this PR), not a runtime patch.

[ibolton336]

Correction: the superseding fix is #935 (@fabianvf), not my #937 (now closed). #935 pins fastmcp<3 and builds the image from the lockfile, and adds a host-header regression test + an image-test CI workflow to prevent recurrence.