[https://nvbugs/6503299][fix] Default fabric memory KV pool for Python cache transceiver

[chuangz0]

Description

When the Python cache transceiver (cache_transceiver_config.transceiver_runtime='PYTHON') is used together with the C++ V1 KV cache manager, the KV cache pool should be allocated with fabric memory so KV blocks can be transferred over MNNVL.

This PR:

1. Defaults TRTLLM_KVCACHE_POOL_USE_FABRIC_MEMORY=1 in KvCacheCreator.__init__ when:

   • the cache transceiver is enabled with transceiver_runtime='PYTHON', and
   • the resolved KV cache manager is the C++ V1 route (KVCacheManager and its hybrid subclasses; KVCacheManagerV2 is excluded).

   The default is applied before any pool allocation (the C++ env getter caches on first read). An explicit user setting is always respected, and platforms without fabric memory support fall back to standard GPU allocation in C++ with a warning.

2. Sets kv_cache_bounce_size_mb: 512 in the GPT-OSS disaggregated perf configs (from [None][perf] Validate GPT-OSS transceiver v2 performance #15765) where the ctx/gen KV-head sharding differs (ctx TP1 → gen TP4, no attention DP), which requires the bounce buffer to regroup KV heads during transfer. Configs with matching per-rank KV-head layout (equal TP, or gen with attention DP) are intentionally left unchanged.

Test Coverage

• Covered by existing disaggregated serving tests using transceiver_runtime: PYTHON.
• The updated perf configs are exercised by the disaggregated perf-sanity/perf pipelines.

PR Checklist

• PR title and description are self-explanatory
• Coding guidelines followed
• Test coverage considered

🤖 Generated with Claude Code

Dev Engineer Review

• Added automatic fabric-memory pooling for Python KV-cache transceivers using the C++ V1 KV cache manager.
• Preserves explicit TRTLLM_KVCACHE_POOL_USE_FABRIC_MEMORY settings and falls back with a warning on unsupported platforms.
• Added kv_cache_bounce_size_mb: 512 to GPT-OSS disaggregated configurations requiring KV-head regrouping; matching-layout configurations remain unchanged.
• Configuration updates are scoped consistently across context and generation workers.

QA Engineer Review

• No test functions or test-list files were modified.
• Existing disaggregated serving tests and performance pipelines cover the changes.
• Verdict: sufficient.

[coderabbitai]

Walkthrough

The change enables Fabric-memory pooling for compatible Python/V1 KV-cache setups and adds a 512 MB KV-cache bounce-size setting to eight GB200 GPT-OSS NIXL benchmark configurations.

Changes

KV-cache Fabric memory configuration

Layer / File(s)	Summary
Fabric memory defaulting tensorrt_llm/_torch/pyexecutor/_util.py	KvCacheCreator conditionally sets TRTLLM_KVCACHE_POOL_USE_FABRIC_MEMORY=1 for Python transceivers using the V1 KVCacheManager, without overriding explicit environment settings.
Benchmark cache bounce settings tests/scripts/perf*/disaggregated/gb200_*NIXL.yaml	Generation and context cache_transceiver_config sections now set kv_cache_bounce_size_mb: 512.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise, correctly formatted, and matches the main change: defaulting fabric memory for the Python cache transceiver.
Description check	✅ Passed	The description includes the required sections and clearly explains the change and test coverage, with only minor checklist brevity.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tensorrt_llm/_torch/pyexecutor/_util.py (1)

347-347: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add the required return annotation.

As per coding guidelines, every Python function must be annotated; declare this helper as returning None.

Proposed fix

-def _maybe_enable_fabric_memory_for_python_transceiver(self):
+def _maybe_enable_fabric_memory_for_python_transceiver(self) -> None:

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/pyexecutor/_util.py` at line 347, Update
_maybe_enable_fabric_memory_for_python_transceiver to include an explicit None
return annotation, preserving its existing behavior.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tensorrt_llm/_torch/pyexecutor/_util.py`:
- Line 347: Update _maybe_enable_fabric_memory_for_python_transceiver to include
an explicit None return annotation, preserving its existing behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e54489eb-50c5-42cb-9963-81b94b2899ed

📥 Commits

Reviewing files that changed from the base of the PR and between 8514fa3 and 442491c.

📒 Files selected for processing (9)

• tensorrt_llm/_torch/pyexecutor/_util.py
• tests/scripts/perf-sanity/disaggregated/gb200_gpt-oss-120b-fp4_1k1k_con64_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL.yaml
• tests/scripts/perf-sanity/disaggregated/gb200_gpt-oss-120b-fp4_8k1k_con1024_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL.yaml
• tests/scripts/perf-sanity/disaggregated/gb200_gpt-oss-120b-fp4_8k1k_con128_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL.yaml
• tests/scripts/perf-sanity/disaggregated/gb200_gpt-oss-120b-fp4_8k1k_con4_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL.yaml
• tests/scripts/perf/disaggregated/gb200_gpt-oss-120b-fp4_8k1k_con1024_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL.yaml
• tests/scripts/perf/disaggregated/gb200_gpt-oss-120b-fp4_8k1k_con128_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL.yaml
• tests/scripts/perf/disaggregated/gb200_gpt-oss-120b-fp4_8k1k_con4_ctx1_tp1_gen1_tp4_eplb0_mtp0_ccb-NIXL.yaml
• tests/scripts/perf/disaggregated/gb200_stress-gpt-oss-120b-fp4_8k1k_ctx1_tp1_gen1_tp4_eplb0_eagle3_ccb-NIXL.yaml