[https://nvbugs/6003113][fix] BREAKING CHANGE: Authenticate disagg request to verify encoded_opaque_state and ctx_info_endpoint by yibinl-nvidia · Pull Request #16002 · NVIDIA/TensorRT-LLM

yibinl-nvidia · 2026-07-06T20:50:27Z

Summary

This change protects disaggregated serving’s internal proxy-to-worker handoff by requiring an internal authentication signature on requests that carry trusted handoff fields ctx_info_endpoint and encoded_opaque_state. The goal is to prevent external clients from directly injecting those internal fields while preserving normal proxy-generated disaggregated requests. This PR fixes 6003113 opaque-state and 6030763 ctx_info_endpoint protections behind one internal disaggregated HMAC auth path.

Validation

Updated tests

Summary by CodeRabbit

New Features
- Added optional authentication for internal disaggregated serving requests.
- Protected internal request metadata is signed and validated between the proxy and workers.
- Added configuration support for sharing an authentication key across disaggregated servers and workers.
- Requests with missing or invalid authentication are rejected.
Documentation
- Updated disaggregated serving examples with authentication and dynamic scaling configuration guidance.
Tests
- Added coverage for signing, validation, tampering, missing keys, and configuration handling.

yibinl-nvidia · 2026-07-06T21:04:32Z

/bot help

github-actions · 2026-07-06T21:04:40Z

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

Details

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental) --high-priority]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Supports wildcard * for pattern matching (e.g., "*PerfSanity*" matches all stages containing PerfSanity). Examples: "A10-PyTorch-1, xxx", "PerfSanity". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Supports wildcard * for pattern matching. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx", --extra-stage "Post-Merge".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

--high-priority (OPTIONAL) : Run the pipeline with high priority. This option is restricted to authorized users only and will route the job to a high-priority queue.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

yibinl-nvidia · 2026-07-06T21:10:55Z

/bot run --add-multi-gpu-test

tensorrt-cicd · 2026-07-06T21:16:28Z

PR_Github #57826 [ run ] triggered by Bot. Commit: 0b43581 Link to invocation

yibinl-nvidia · 2026-07-06T21:17:05Z

/bot kill

tensorrt-cicd · 2026-07-06T21:22:35Z

PR_Github #57827 [ kill ] triggered by Bot. Commit: 0b43581 Link to invocation

tensorrt-cicd · 2026-07-06T21:28:39Z

PR_Github/16002-0b43581 #57826 was force-killed by a newer pipeline run.
L0 job information not available (job may not have been triggered yet).

Link to superseding invocation

tensorrt-cicd · 2026-07-06T21:28:40Z

PR_Github #57827 [ kill ] completed with state SUCCESS. Commit: 0b43581
Successfully killed previous jobs for commit 0b43581

Link to invocation

yibinl-nvidia · 2026-07-07T03:51:40Z

/bot run --add-multi-gpu-test --disable-fail-fast

tensorrt-cicd · 2026-07-07T03:57:51Z

PR_Github #57925 [ run ] triggered by Bot. Commit: 0853c0e Link to invocation

tensorrt-cicd · 2026-07-07T09:12:50Z

PR_Github #57925 [ run ] completed with state FAILURE. Commit: 0853c0e
/LLM/main/L0_MergeRequest_PR pipeline #46614 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-07T18:50:23Z

/bot run --add-multi-gpu-test --disable-fail-fast

tensorrt-cicd · 2026-07-07T18:56:19Z

PR_Github #58065 [ run ] triggered by Bot. Commit: 8db5aa4 Link to invocation

tensorrt-cicd · 2026-07-08T03:23:21Z

PR_Github #58065 [ run ] completed with state SUCCESS. Commit: 8db5aa4
/LLM/main/L0_MergeRequest_PR pipeline #46732 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-08T20:33:27Z

/bot run --add-multi-gpu-test --disable-fail-fast

tensorrt-cicd · 2026-07-08T20:40:05Z

PR_Github #58297 [ run ] triggered by Bot. Commit: 0f7134d Link to invocation

tensorrt-cicd · 2026-07-09T10:35:43Z

PR_Github #58297 [ run ] completed with state SUCCESS. Commit: 0f7134d
/LLM/main/L0_MergeRequest_PR pipeline #46932 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-09T22:21:19Z

/bot run --add-multi-gpu-test --disable-fail-fast

tensorrt-cicd · 2026-07-09T22:29:55Z

PR_Github #58518 [ run ] triggered by Bot. Commit: 6557be6 Link to invocation

tensorrt-cicd · 2026-07-10T19:04:07Z

PR_Github #58518 [ run ] completed with state SUCCESS. Commit: 6557be6
/LLM/main/L0_MergeRequest_PR pipeline #47125 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

Signed-off-by: Yibin-Li <109242046+yibinl-nvidia@users.noreply.github.com>

tensorrt-cicd · 2026-07-23T22:00:08Z

PR_Github #61385 [ run ] completed with state FAILURE. Commit: f5b5ba9
/LLM/main/L0_MergeRequest_PR pipeline #49612 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-23T22:00:54Z

/bot run

tensorrt-cicd · 2026-07-23T22:07:20Z

PR_Github #61413 [ run ] triggered by Bot. Commit: b3be300 Link to invocation

tensorrt-cicd · 2026-07-23T22:23:42Z

PR_Github #61413 [ run ] completed with state FAILURE. Commit: b3be300
/LLM/main/L0_MergeRequest_PR pipeline #49640 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-23T22:40:13Z

/bot run

tensorrt-cicd · 2026-07-23T22:45:46Z

PR_Github #61420 [ run ] triggered by Bot. Commit: b3be300 Link to invocation

tensorrt-cicd · 2026-07-23T23:01:32Z

PR_Github #61420 [ run ] completed with state FAILURE. Commit: b3be300
/LLM/main/L0_MergeRequest_PR pipeline #49645 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-23T23:17:47Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-23T23:23:26Z

PR_Github #61425 [ run ] triggered by Bot. Commit: b3be300 Link to invocation

tensorrt-cicd · 2026-07-24T02:49:10Z

PR_Github #61425 [ run ] completed with state SUCCESS. Commit: b3be300
/LLM/main/L0_MergeRequest_PR pipeline #49650 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-24T03:01:55Z

/bot run --disable-fail-fast

yibinl-nvidia · 2026-07-24T03:10:23Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-24T03:16:36Z

PR_Github #61483 [ run ] triggered by Bot. Commit: b3be300 Link to invocation

tensorrt-cicd · 2026-07-24T07:38:47Z

PR_Github #61483 [ run ] completed with state SUCCESS. Commit: b3be300
/LLM/main/L0_MergeRequest_PR pipeline #49703 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-24T08:01:27Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-24T08:07:10Z

PR_Github #61538 [ run ] triggered by Bot. Commit: b3be300 Link to invocation

Shixiaowei02

Thanks for the effort. My biggest concern is the rollout design: because enforcement keys off field presence and both sides hard-fail when the key is unset, existing ctx-first deployments break at request time on upgrade. Could we add an explicit disabled/warn/enforce policy, default to warn initially, and validate key consistency at startup? Thanks!

tensorrt-cicd · 2026-07-24T11:16:44Z

PR_Github #61538 [ run ] completed with state FAILURE. Commit: b3be300
/LLM/main/L0_MergeRequest_PR pipeline #49750 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-07-24T11:41:31Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-24T11:47:01Z

PR_Github #61569 [ run ] triggered by Bot. Commit: b3be300 Link to invocation

github-actions Bot assigned yibinl-nvidia Jul 6, 2026

yibinl-nvidia changed the title ~~fix(disaggregated): authenticate internal handoff~~ [https://nvbugs/6003113 https://nvbugs/6030763][fix] Authenticate disagg request to verify disaggregated opaque state tag and context endpointpoint Jul 6, 2026

yibinl-nvidia changed the title ~~[https://nvbugs/6003113/https://nvbugs/6030763][fix] Authenticate disagg request to verify opaque state tag and context endpoint~~ [https://nvbugs/6003113][fix] Authenticate disagg request to verify opaque state tag and context endpoint Jul 6, 2026

yibinl-nvidia force-pushed the fix/6003113-6030763-authenticate-disag-request branch 2 times, most recently from 02c64ed to fba77db Compare July 6, 2026 23:57

yibinl-nvidia force-pushed the fix/6003113-6030763-authenticate-disag-request branch from 8db5aa4 to a8e84e0 Compare July 8, 2026 19:37

yibinl-nvidia force-pushed the fix/6003113-6030763-authenticate-disag-request branch 2 times, most recently from 375aa31 to 6557be6 Compare July 9, 2026 21:58

yibinl-nvidia added 9 commits July 23, 2026 14:59

chore: trim disaggregated auth changes

61c2866

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

test(disaggregated): update receiver endpoint test

9caea87

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

test(perf): inject disagg auth key in sanity harness

e2384cb

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

fix: narrow disagg auth key requirement

e9df536

Signed-off-by: Yibin-Li <109242046+yibinl-nvidia@users.noreply.github.com>

[ci watcher] fix disagg auth validation

c28a031

Signed-off-by: Yibin-Li <109242046+yibinl-nvidia@users.noreply.github.com>

[ci watcher] fix disagg auth and proxy cleanup

8ffb046

Signed-off-by: Yibin-Li <109242046+yibinl-nvidia@users.noreply.github.com>

[ci watcher] require disagg receiver ctx endpoint

2a5093e

Signed-off-by: Yibin-Li <109242046+yibinl-nvidia@users.noreply.github.com>

[ci watcher] share perf disagg auth key

a195c46

Signed-off-by: Yibin-Li <109242046+yibinl-nvidia@users.noreply.github.com>

[ci watcher] satisfy pre-commit after rebase

b3be300

Signed-off-by: Yibin-Li <109242046+yibinl-nvidia@users.noreply.github.com>

yibinl-nvidia force-pushed the fix/6003113-6030763-authenticate-disag-request branch from f5b5ba9 to b3be300 Compare July 23, 2026 22:00

Shixiaowei02 reviewed Jul 24, 2026

View reviewed changes

Uh oh!

Conversation

yibinl-nvidia commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Summary by CodeRabbit

Uh oh!

yibinl-nvidia commented Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

GitHub Bot Help

kill

skip

reuse-pipeline

Uh oh!

yibinl-nvidia commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

yibinl-nvidia commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

yibinl-nvidia commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

yibinl-nvidia commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

yibinl-nvidia commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

yibinl-nvidia commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 10, 2026

Uh oh!

tensorrt-cicd commented Jul 23, 2026

Uh oh!

yibinl-nvidia commented Jul 23, 2026

Uh oh!

tensorrt-cicd commented Jul 23, 2026

Uh oh!

tensorrt-cicd commented Jul 23, 2026

Uh oh!

yibinl-nvidia commented Jul 23, 2026

Uh oh!

tensorrt-cicd commented Jul 23, 2026

Uh oh!

tensorrt-cicd commented Jul 23, 2026

Uh oh!

yibinl-nvidia commented Jul 23, 2026

Uh oh!

tensorrt-cicd commented Jul 23, 2026

Uh oh!

tensorrt-cicd commented Jul 24, 2026

Uh oh!

yibinl-nvidia commented Jul 24, 2026

Uh oh!

yibinl-nvidia commented Jul 24, 2026

Uh oh!

tensorrt-cicd commented Jul 24, 2026

Uh oh!

tensorrt-cicd commented Jul 24, 2026

Uh oh!

yibinl-nvidia commented Jul 6, 2026 •

edited

Loading