From 94506a14d9dbdff1bc17a2eb6e89e2f69da973c1 Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 8 Jul 2026 11:32:44 -0700 Subject: [PATCH] Fix external-provider ModelEndpoint example's double /v1 prefix The "Route to External Providers" doc set rewritePath: /v1/ for a Together-backed ModelEndpoint. A ModelService's HTTPRoute only strips the /// prefix from the client's request, leaving the client's own v1/... segment as the preserved remainder. Prepending /v1/ on top of that produces /v1/v1/..., which 404s against the backend. rewritePath should replace only the namespace/service prefix, same as the reference example for composed replicas (rewritePath: /ml-team/qwen-72b/, no added segment). Set it to / here so the client's v1/... passes through unchanged to Together's own /v1/... endpoint. Fixes #327. Signed-off-by: Nic Cope --- docs/content/models/model-endpoint.md | 7 ++++--- docs/manifests/concepts/model-endpoint.yaml | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/content/models/model-endpoint.md b/docs/content/models/model-endpoint.md index b5a7832a..1ecce28f 100644 --- a/docs/content/models/model-endpoint.md +++ b/docs/content/models/model-endpoint.md @@ -31,9 +31,10 @@ spec: # 2. The provider's base URL. url: https://api.together.xyz/ # 3. The path to rewrite requests to. A ModelService receives requests at - # ///v1/... and rewrites them to this prefix, so an - # OpenAI-compatible provider that serves /v1/... takes /v1/. - rewritePath: /v1/ + # ///v1/... and strips only the /// + # prefix, so an OpenAI-compatible provider that already serves /v1/... + # takes just /. + rewritePath: / ``` Then point a [`ModelService`]({{< ref "model-service.md" >}}) at it. Selecting diff --git a/docs/manifests/concepts/model-endpoint.yaml b/docs/manifests/concepts/model-endpoint.yaml index 8a7b9020..84529aac 100644 --- a/docs/manifests/concepts/model-endpoint.yaml +++ b/docs/manifests/concepts/model-endpoint.yaml @@ -14,4 +14,4 @@ metadata: modelplane.ai/external-provider: together spec: url: https://api.together.xyz/ - rewritePath: /v1/ + rewritePath: /