[Proposal][Design] Per-Operation Upstream Override on OperationRequest

[mehara-rothila]

Background

The current OperationRequest schema cannot direct individual operations to different backend services: every operation in a REST API shares one API-level upstream. This proposal adds per-operation upstream routing so a single API can front multiple microservices without fragmenting the API contract.

Proposed Solution

A new optional upstream field on OperationRequest, an OperationUpstream wrapper with main and sandbox sub-fields. Each sub-field is an OperationUpstreamTarget that is ref-only: it names an entry in an API-level upstreamDefinitions pool. Inline URLs are not used at the operation level; every backend URL is declared once in upstreamDefinitions and referenced by name. Omitted sub-fields fall back to the API-level upstream for that environment.

upstreamDefinitions (the backend pool)

A new API-level array of named backends. Each entry has a name, an optional basePath, an optional connect timeout, and one or more weighted upstreams (each a backend url). Both API-level and operation-level refs resolve against this pool, so a backend URL is defined in exactly one place and reused by name.

Example

upstreamDefinitions:
  - name: user-service
    basePath: /user-svc
    upstreams:
      - url: http://user-service:8080
  - name: user-service-sandbox
    basePath: /user-svc
    upstreams:
      - url: http://user-service-test:5000
  - name: order-service
    basePath: /order-svc
    timeout:
      connect: 5s
    upstreams:
      - url: http://order-service:8080
upstream:
  main:
    url: http://default-backend:8080   # API-level default (accepts url or ref)
operations:
  - name: listUsers
    request:
      method: GET
      path: /users
      upstream:
        main:
          ref: user-service
        sandbox:
          ref: user-service-sandbox
  - name: createOrder
    request:
      method: POST
      path: /orders
      upstream:
        main:
          ref: order-service
  - name: health
    request:
      method: GET
      path: /health
      # no per-op upstream: falls back to the API-level default-backend

Key Design Features

• Ref-only at the operation level. An operation never carries an inline url; its main/sandbox name an upstreamDefinitions entry.
• Fallback mechanism. An omitted main/sandbox falls back to the API-level upstream of the same environment.
• Precedence. A dynamic-endpoint policy takes precedence over the per-operation ref, which takes precedence over the API-level upstream.
• Non-disruptive URL edits. API-level upstream URL changes propagate as endpoint updates rather than cluster recreates, so they do not drain in-flight connections.
• No auth support. The schema intentionally excludes auth at the operation level.
• Validation. A ref must resolve to a unique, well-formed definition; ref and definition names share the contract ^[a-zA-Z0-9\-_]+$ (max 100); upstream urls are host-only http/https; connect timeout must be positive; an empty upstream wrapper and an empty main/sandbox leaf are rejected; invalid configs return HTTP 400.
• Environment-scoped. Separate resolution for the main and sandbox vhosts.

Implementation Scope

Platform API: new OpenAPI schemas for the upstreamDefinitions pool and the ref-only OperationUpstream/OperationUpstreamTarget; ref resolution and validation; JSON persistence (no database migration required); emits the deployment artifact the gateway consumes.

Gateway Controller: resolves the per-operation ref and points the operation route at the referenced definition's cluster (one cluster per definition, reused by every operation that shares the ref); API-level clusters use identity-based naming so a URL edit is an endpoint update; validation and xDS translation.

Policy Engine (gateway runtime): resolves the per-operation base path so it is applied exactly once, and honors dynamic-endpoint precedence over the static per-operation ref.

Backward Compatibility

The feature maintains full backward compatibility. The upstream field is optional, and existing APIs remain unaffected.