Add XCom read access to callback supervisor comms channel#66611
Add XCom read access to callback supervisor comms channel#66611seanghaeli wants to merge 5 commits into
Conversation
|
What is the plan for plumbing the dag_id, run_id, etc through here? You don't seem to have addressed thr "motivation" part of your description. The reason I didn't include this one in the initial batch of comms channels was because GetXCom requires those and I didn't want users to have to manually provide those like |
|
Good question. The plan is for PR #66608 to provide the context plumbing — once a callback has access to So this PR gives the mechanism (XCom comms channel), and #66608 gives the identifiers. Once #66608 lands, we can add a convenience wrapper that pre-fills |
90f3323 to
91e5a68
Compare
Callbacks running inside the CallbackSubprocess can now read XCom values via the same supervisor IPC mechanism used for Connections and Variables. The caller must supply explicit dag_id, run_id, and task_id since callbacks have no implicit task context. This adds GetXCom to the CallbackToSupervisor union and routes it through a new shared handle_get_xcom handler in request_handlers.py. Only read access (GetXCom) is exposed; SetXCom and DeleteXCom remain out of scope for callbacks.
…I routes Callback subprocesses use workload-scoped JWT tokens, but the connections and variables endpoints defaulted to execution-only token acceptance. Any callback using Connection.get() or Variable.get() would receive a 403. Add Security(require_auth, scopes=[...workload]) at the router level to allow both token types, matching the fix applied to dag_runs in apache#66608.
4a33ab3 to
454244a
Compare
Add read-only XCom access to the callback supervisor IPC channel, allowing
callbacks to fetch XCom values from upstream tasks via the same mechanism
used for Connections and Variables.
Motivation
PR #65269 added Connection, Variable, and MaskSecret comms channels to the
callback supervisor. XCom was deferred because it requires explicit dag_id,
run_id, and task_id (callbacks have no implicit task context). This PR adds
that missing piece.
Changes
GetXComto theCallbackToSupervisorunion typehandle_get_xcomshared handler inrequest_handlers.pyGetXCommessages inCallbackSubprocess._handle_requestOnly read access (GetXCom) is exposed. SetXCom and DeleteXCom remain out of
scope for callbacks since they should not be producing XCom values.
related: #65269
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.6) following the guidelines