[https://nvbugs/6327147][fix] Lower kv_cache_config.free_gpu_memory_fraction from 0.7 to 0.5 in the…

[tensorrt-cicd]

Summary

• Root cause: free_gpu_memory_fraction=0.7 over-budgets the paged KV cache (9.0 GB) so the post-resize CUDA-graph capture (~5.4 GB) and runtime fp32 lm_head logits cast (~3.9 GB) push the L40S 48 GB GPU over the limit.
• Fix: Lower kv_cache_config.free_gpu_memory_fraction from 0.7 to 0.5 in the non-chunked-prefill branch of TestNemotronH.get_default_kwargs; remove the matching waives.txt entries.
• Automated fix generated by repair-bot

Test plan

• Verify fix on the same GPU type as the original failure
• Check for regressions in related tests

Links

• Bug: https://nvbugs/6327147

Summary by CodeRabbit

• Tests
  • Adjusted memory configuration settings for accuracy test execution to improve reliability based on prefill configuration.
  • Re-enabled two accuracy tests that were previously waived, allowing them to run in the test suite.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c60fbad3-3ecb-40d6-b5cb-c3ffdef836eb

📥 Commits

Reviewing files that changed from the base of the PR and between 42a3e55 and 6d63c43.

📒 Files selected for processing (2)

• tests/integration/defs/accuracy/test_llm_api_autodeploy.py
• tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

• tests/integration/test_lists/waives.txt

---

📝 Walkthrough

Walkthrough

TestNemotronH.get_default_kwargs now selects free_gpu_memory_fraction as 0.7 when chunked prefill is enabled and 0.5 otherwise to avoid OOM from the synthetic forward profile. Two corresponding L40S waive entries for test_auto_dtype with trtllm-flashinfer_ssm-False and trtllm-triton_ssm-False are removed.

Changes

NemotronH OOM fix and waive cleanup

Layer / File(s)	Summary
Conditional free_gpu_memory_fraction and waive removal tests/integration/defs/accuracy/test_llm_api_autodeploy.py, tests/integration/test_lists/waives.txt	get_default_kwargs computes free_gpu_memory_fraction as 0.7 (chunked prefill enabled) or 0.5 (disabled) and uses it in kv_cache_config; the two L40S waive entries for TestNemotronH::test_auto_dtype parameterized with trtllm-flashinfer_ssm-False and trtllm-triton_ssm-False are removed.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#15214: Also edits tests/integration/test_lists/waives.txt to remove accuracy-test skip entries for test_llm_api_autodeploy.py.
• NVIDIA/TensorRT-LLM#15389: Originally added the waive entries for TestNemotronH::test_auto_dtype with trtllm-flashinfer_ssm-False and trtllm-triton_ssm-False that this PR removes.

Suggested reviewers

• bmarimuthu-nv
• nvchenghaoz
• suyoggupta
• tcherckez-nvidia

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	❓ Inconclusive	The title is incomplete and truncated with an ellipsis, making it unclear what the fix entails beyond the parameter name.	Complete the title to clearly state what the fix accomplishes, e.g., '[https://nvbugs/6327147][fix] Lower kv_cache_config.free_gpu_memory_fraction to 0.5 for non-chunked-prefill to fix L40S OOM.'

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description check	✅ Passed	The PR description addresses the root cause, fix, and includes test verification, but the PR checklist items are not explicitly confirmed as completed.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[suyoggupta]

/bot run

[tensorrt-cicd]

PR_Github #54886 [ run ] triggered by Bot. Commit: 6d63c43 Link to invocation

[tensorrt-cicd]

PR_Github #54886 [ run ] completed with state FAILURE. Commit: 6d63c43
/LLM/main/L0_MergeRequest_PR pipeline #43891 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation