Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 3 additions & 1 deletion .claude/skills/exec-local-compile/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ git checkout main && git pull
Run the build command (**incremental by default** — omit `-c`/`--clean` unless explicitly requested or the incremental build fails):

```bash
./scripts/build_wheel.py --use_ccache -a "<arch>" -f --nvtx
./scripts/build_wheel.py --trt_root /usr/local/tensorrt --benchmarks --use_ccache -a "<arch>" -f --nvtx
```

Replace `<arch>` with the target GPU architecture (see Architecture Reference below). If not specified by the user, auto-detect from `nvidia-smi`.
Expand All @@ -66,6 +66,8 @@ python3 -c "import tensorrt_llm; print(tensorrt_llm.__version__)"

| Flag | Description |
|------|-------------|
| `--trt_root /usr/local/tensorrt` | TensorRT installation path (standard in NVIDIA containers) |
| `--benchmarks` | Build the C++ benchmarks |
| `-a "<arch>"` | Target GPU architecture(s) |
| `--nvtx` | Enable NVTX markers for profiling |
| `--use_ccache` | Use ccache for faster recompilation |
Expand Down
3 changes: 3 additions & 0 deletions .claude/skills/exec-slurm-compile/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,8 @@ A successful build ends with a message like `Successfully built tensorrt_llm` or

| Flag | Description |
|------|-------------|
| `--trt_root /usr/local/tensorrt` | TensorRT installation path (standard in NVIDIA containers) |
| `--benchmarks` | Build the C++ benchmarks |
| `-a "100-real"` | Target architecture — `100` for Blackwell, `90` for Hopper, etc. |
| `--nvtx` | Enable NVTX markers for profiling |
| `--no-venv` | Skip virtual environment creation |
Expand All @@ -226,6 +228,7 @@ Common architecture values:
| `sbatch: error: invalid partition` | Verify partition name with `sinfo -s` |
| `sbatch: error: invalid account` | Check available accounts with `sacctmgr show assoc user=$USER` |
| Container image not found | Verify the `.sqsh` path exists and is readable |
| Build fails with missing TensorRT | Ensure `--trt_root` points to the correct path inside the container |
| Build OOM (out of memory) | Reduce parallelism with `-j <N>` flag to `build_wheel.py` |
| `srun: error: Unable to create step` | The node may lack enroot/pyxis — check with cluster admin |
| Job stuck in `PD` state | Check `squeue -j <id> -o %R` for the reason (e.g., resource limits, priority) |
Expand Down
4 changes: 3 additions & 1 deletion .claude/skills/exec-slurm-compile/scripts/compile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
# Usage: compile.sh <repo_dir> [build_wheel_args...]
#
# Default build_wheel.py flags:
# -a "100-real" --nvtx --no-venv
# --trt_root /usr/local/tensorrt --benchmarks -a "100-real" --nvtx --no-venv
# Any extra arguments after repo_dir are forwarded to build_wheel.py,
# overriding the defaults above.

Expand All @@ -36,6 +36,8 @@ if [[ $# -gt 0 ]]; then
else
echo "[compile.sh] Running default build command"
python3 ./scripts/build_wheel.py \
--trt_root /usr/local/tensorrt \
--benchmarks \
-a "100-real" \
--nvtx
fi
39 changes: 4 additions & 35 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,26 +22,7 @@ reviews:
auto_title_placeholder: '@coderabbitai title'
auto_title_instructions: 'Format: "[<category>] <title>". Category must be one of: fix, feat, doc, infra, style, refactor, perf, test, chore, revert. Enclose the category in square brackets. Title should be concise (<= 60 chars). Example: "[feat] Add logit_bias support".'
commit_status: false
high_level_summary_instructions: |
Always produce two review sections in the summary:

**Dev Engineer Review**
Review all changes for correctness and consistency, including:
- Code changes: correctness, performance, API consistency (CODING_GUIDELINES.md), error handling, regressions.
- Config files: valid values, no typos, consistency with related configs, no unintended scope changes.
- Test list files (test-db/, qa/, waives.txt): correct format, valid test paths, appropriate bug references, no duplicates.

**QA Engineer Review**
Always include this section when any files under tests/ are touched.
For test-list-only changes (only tests/integration/test_lists/ files):
- List which test-db/ or qa/ files were modified and what entries were added or removed.
- Verdict: "needs follow-up" if CBTS coverage data is unavailable, otherwise "sufficient" or "insufficient".
For test-code changes (files outside tests/integration/test_lists/):
- List test functions added, modified, or removed.
- State whether each is covered in tests/integration/test_lists/ (test-db/ for CI, qa/ for manual QA).
- Verdict: sufficient, insufficient, or needs follow-up.
If no test files are touched, write "No test changes."
collapse_walkthrough: false
collapse_walkthrough: true
assess_linked_issues: true
related_issues: true
related_prs: true
Expand All @@ -50,27 +31,15 @@ reviews:
poem: false
review_status: false
auto_review:
auto_incremental_review: true
auto_incremental_review: false
drafts: false
base_branches: ["main", "release/.+"]
path_instructions:
- path: "tests/**"
instructions: |
Act as a QA engineer reviewing test changes and coverage for TensorRT-LLM.
Always produce a test coverage summary, even if no issues are found.

If the change touches ONLY files under tests/integration/test_lists/ (no test-code changes):
- Report which test-db/ or qa/ list files were modified and what entries were added or removed.
- Do NOT require changed test functions for this path.
- Use verdict "needs follow-up" when cbts_touchmap.sqlite or a CBTS coverage report is unavailable
to confirm the impacted test scope; otherwise use "sufficient" or "insufficient".

If the change includes test-code files (outside tests/integration/test_lists/), the summary must include:
1. Which test functions were added, modified, or removed.
2. Whether each changed test is listed in the appropriate test list files under
tests/integration/test_lists/ (test-db/ for CI, qa/ for manual QA).
3. A coverage verdict: sufficient, insufficient, or needs follow-up.
Keep feedback actionable: reference concrete list file names when suggesting additions.
Keep feedback actionable: suggest concrete list file names and whether
coverage is sufficient, insufficient, or needs follow-up outside the PR.
- path: "tests/integration/test_lists/qa/**"
instructions: |
Files here are manually-triggered QA perf/regression lists, maintained
Expand Down
3 changes: 1 addition & 2 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,4 @@ docs/source/blogs/media/tech_blog10_full_strategy_performance.png filter=lfs dif
docs/source/blogs/media/tech_blog10_context_wait_performance.png filter=lfs diff=lfs merge=lfs -text
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo_cubin.cpp filter=lfs diff=lfs merge=lfs -text
cpp/tensorrt_llm/kernels/decoderMaskedMultiheadAttention/cubin/xqa_kernel_cubin.cpp filter=lfs diff=lfs merge=lfs -text
docs/source/blogs/media/tech_blog26_deepseek_v4_hybrid_attention.png filter=lfs diff=lfs merge=lfs -text
docs/source/blogs/media/tech_blog26_deepseek_v4_mhc_moe.png filter=lfs diff=lfs merge=lfs -text
tensorrt_llm/_torch/visual_gen/cute_dsl_kernels/blackwell/attention/cubins/*/*/*.so filter=lfs diff=lfs merge=lfs -text
21 changes: 5 additions & 16 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
/tests/unittest @NVIDIA/trt-llm-devs

# ===== TensorRT backend (will be deprecated soon) — also on the trt-llm-devs fallback =====
/cpp/include/tensorrt_llm/plugins @NVIDIA/trt-llm-devs
/cpp/tensorrt_llm/plugins @NVIDIA/trt-llm-devs
/tensorrt_llm/builder.py @NVIDIA/trt-llm-devs
/tensorrt_llm/commands/build.py @NVIDIA/trt-llm-devs
/tensorrt_llm/commands/prune.py @NVIDIA/trt-llm-devs
Expand Down Expand Up @@ -103,6 +105,7 @@
/cpp/tensorrt_llm/batch_manager @NVIDIA/trt-llm-runtime-devs
/cpp/tensorrt_llm/common @NVIDIA/trt-llm-runtime-devs
/cpp/tensorrt_llm/executor @NVIDIA/trt-llm-runtime-devs
/cpp/tensorrt_llm/executor_worker @NVIDIA/trt-llm-runtime-devs
/cpp/tensorrt_llm/layers @NVIDIA/trt-llm-runtime-devs
/cpp/tensorrt_llm/nanobind @NVIDIA/trt-llm-runtime-devs
/cpp/tensorrt_llm/runtime @NVIDIA/trt-llm-runtime-devs
Expand Down Expand Up @@ -166,6 +169,7 @@
/examples/llm-api/quickstart_multimodal.py @NVIDIA/trt-llm-models-devs @NVIDIA/trt-llm-doc-owners
/examples/models @NVIDIA/trt-llm-models-devs @NVIDIA/trt-llm-doc-owners
/examples/serve/*multimodal* @NVIDIA/trt-llm-models-devs @NVIDIA/trt-llm-doc-owners
/scripts/build_cpp_examples.py @NVIDIA/trt-llm-models-devs
/scripts/generate_config_database_tests.py @NVIDIA/trt-llm-models-devs @NVIDIA/trt-llm-doc-owners
/scripts/generate_config_table.py @NVIDIA/trt-llm-models-devs @NVIDIA/trt-llm-doc-owners
/tensorrt_llm/_torch/models @NVIDIA/trt-llm-models-devs
Expand Down Expand Up @@ -217,11 +221,6 @@
/tensorrt_llm/_torch/pyexecutor/mamba_cache_manager.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tensorrt_llm/_torch/pyexecutor/resource_manager.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tensorrt_llm/runtime/kv_cache_manager_v2 @NVIDIA/trt-llm-kv-cache-manager-devs
/tests/unittest/_torch/executor/test_dual_pool_kv_cache.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tests/unittest/_torch/executor/test_kv_cache* @NVIDIA/trt-llm-kv-cache-manager-devs
/tests/unittest/_torch/executor/test_kv_pool_rebalance.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tests/unittest/_torch/executor/test_kvcache_aware_router.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tests/unittest/_torch/executor/test_mamba_cache_manager.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tests/unittest/kv_cache_manager_v2_tests @NVIDIA/trt-llm-kv-cache-manager-devs

# ===== DISAGGREGATED SERVING =====
Expand All @@ -241,17 +240,7 @@
/tensorrt_llm/_torch/pyexecutor/kv_cache_transceiver.py @NVIDIA/trt-llm-disagg-devs
/tensorrt_llm/disaggregated_params.py @NVIDIA/trt-llm-disagg-devs
/tensorrt_llm/serve/openai_disagg_server.py @NVIDIA/trt-llm-disagg-devs
# Disagg tests: co-own with the owning team so disagg-devs review disagg-test changes.
/tests/integration/defs/accuracy/*disagg* @NVIDIA/trt-llm-disagg-devs @NVIDIA/trt-llm-qa
/tests/integration/defs/disaggregated @NVIDIA/trt-llm-disagg-devs @NVIDIA/trt-llm-qa
/tests/integration/defs/stress_test/disagg_cancel @NVIDIA/trt-llm-disagg-devs @NVIDIA/trt-llm-qa
/tests/scripts/perf-sanity/disaggregated @NVIDIA/trt-llm-perf-devs @NVIDIA/trt-llm-disagg-devs
/tests/scripts/perf/disaggregated @NVIDIA/trt-llm-perf-devs @NVIDIA/trt-llm-disagg-devs
/tests/unittest/_torch/executor/*disagg* @NVIDIA/trt-llm-runtime-devs @NVIDIA/trt-llm-disagg-devs
/tests/unittest/_torch/multimodal/*disagg* @NVIDIA/trt-llm-models-devs @NVIDIA/trt-llm-disagg-devs
/tests/unittest/disaggregated @NVIDIA/trt-llm-disagg-devs
/tests/unittest/llmapi/*disagg* @NVIDIA/trt-llm-runtime-devs @NVIDIA/trt-llm-disagg-devs
/tests/unittest/llmapi/apps/*disagg* @NVIDIA/trt-llm-runtime-devs @NVIDIA/trt-llm-disagg-devs

# ===== ATTENTION =====
# Where a kernel is both a dir and sibling .cu/.h, keep the bare dir (subtree) AND a trailing-* (siblings).
Expand Down Expand Up @@ -437,7 +426,6 @@
/cpp/tensorrt_llm/batch_manager/allocateKvCache.cpp @NVIDIA/trt-llm-kv-cache-manager-devs
/cpp/tests/unit_tests/batch_manager/kvCacheManagerTest.cpp @NVIDIA/trt-llm-kv-cache-manager-devs
/cpp/tests/unit_tests/batch_manager/kvCacheUtilsTest.cpp @NVIDIA/trt-llm-kv-cache-manager-devs
/tensorrt_llm/_torch/attention_backend/sparse/*/cache_manager.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py @NVIDIA/trt-llm-kv-cache-manager-devs
/tensorrt_llm/_torch/pyexecutor/resource_manager.py @NVIDIA/trt-llm-kv-cache-manager-devs
/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManager.h @NVIDIA/trt-llm-kv-cache-manager-devs
Expand All @@ -455,6 +443,7 @@
# ===== DISAGG BLAST-RADIUS CO-OWNS (Tier-1) =====
# Last-match cross-cutting: adds disagg-devs to shared files that can silently break disagg e2e.
/tensorrt_llm/_torch/pyexecutor/resource_manager.py @NVIDIA/trt-llm-kv-cache-manager-devs @NVIDIA/trt-llm-disagg-devs
/tensorrt_llm/_torch/modules/mla.py @NVIDIA/trt-llm-torch-attention-devs @NVIDIA/trt-llm-disagg-devs
/tensorrt_llm/_torch/pyexecutor/py_executor.py @NVIDIA/trt-llm-runtime-devs @NVIDIA/trt-llm-disagg-devs
/tensorrt_llm/_torch/pyexecutor/model_engine.py @NVIDIA/trt-llm-runtime-devs @NVIDIA/trt-llm-disagg-devs

Expand Down
Loading
Loading