Skip to content

fix: support connectrpc 0.11 property-form request_headers (FLYTE-SDK-5T)#1253

Open
EngHabu wants to merge 1 commit into
mainfrom
fix/sentry-5t-connectrpc-headers
Open

fix: support connectrpc 0.11 property-form request_headers (FLYTE-SDK-5T)#1253
EngHabu wants to merge 1 commit into
mainfrom
fix/sentry-5t-connectrpc-headers

Conversation

@EngHabu

@EngHabu EngHabu commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Problem (FLYTE-SDK-5T)

connectrpc 0.11 changed RequestContext.request_headers (and the other accessors) from methods into @property attributes. Our auth interceptors call ctx.request_headers() as a method, so any user who resolved connectrpc>=0.11 — which is permitted by our connectrpc>=0.9.0,<1.0.0 pin — crashes with:

TypeError: 'Headers' object is not callable
  at flyte/remote/_client/auth/_interceptors/default_metadata.py:36 on_start

Because the failure is in the metadata/auth interceptor that runs on every outgoing control-plane RPC, the user can't SelectCluster, upload, or run anything — it surfaces in Sentry as RuntimeSystemError: Upload failed ... SelectCluster failed. Seen on the latest release 2.5.4.

Fix

Add a tiny _compat.request_headers(ctx) accessor that resolves both API shapes:

headers = ctx.request_headers
return headers() if callable(headers) else headers
  • connectrpc <0.11: request_headers is a bound method → callable → invoked.
  • connectrpc >=0.11: request_headers is a Headers property → not callable → used directly.

All four interceptor call sites (default_metadata.py ×2, auth.py ×2) now route through it. This keeps the SDK working across the entire pinned connectrpc range rather than betting on one API shape.

Tests

tests/flyte/remote/test_connectrpc_interceptors.py:

  • TestRequestHeadersCompat — unit tests for both method-form and property-form contexts.
  • TestConnectrpc011PropertyForm — DefaultMetadata + Auth interceptors against a property-form context (regression guard for 5T).

47/47 pass locally.

fixes FLYTE-SDK-5T

@EngHabu EngHabu added the sentry-fix Fix for an issue surfaced by Sentry label Jun 25, 2026

async def on_start(self, ctx) -> None:
existing_rid = ctx.request_headers().get("x-request-id")
headers = request_headers(ctx)

@pingsutw pingsutw Jun 25, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

related to https://unionai.slack.com/archives/C07QTRXMH8C/p1782404943001369. connect rpc just fixed this breaking change today, we will bump the version next week

@EngHabu EngHabu force-pushed the fix/sentry-5t-connectrpc-headers branch 2 times, most recently from 4e496ea to 372a020 Compare June 28, 2026 16:07
connectrpc 0.11 turned RequestContext.request_headers (and the other
accessors) from methods into @Property attributes. Our auth interceptors
called ctx.request_headers() as a method, so users who resolved
connectrpc>=0.11 (allowed by the connectrpc>=0.9.0,<1.0.0 pin) crashed with
"TypeError: 'Headers' object is not callable" on every control-plane RPC
(SelectCluster, uploads, runs).

Add a small _compat.request_headers(ctx) accessor that resolves both the
method form (connectrpc <0.11) and the property form (>=0.11), and route the
four interceptor call sites through it. Adds tests covering both API shapes.

fixes FLYTE-SDK-5T

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
@EngHabu EngHabu force-pushed the fix/sentry-5t-connectrpc-headers branch from 372a020 to e8b8310 Compare June 29, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sentry-fix Fix for an issue surfaced by Sentry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants