Remove triton optimization config, causing error for multi gpu inference#15
Draft
tzemicheal wants to merge 90 commits into
Draft
Remove triton optimization config, causing error for multi gpu inference#15tzemicheal wants to merge 90 commits into
tzemicheal wants to merge 90 commits into
Conversation
## Description * Manually resolving forward merger issues from PR #1966 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes.
## Description Manually resolving forward merge conflicts from PR #1983 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes.
…not just rows (#2040) Prevents false positives when columns are missing in output ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #2040
* Replace `Broadcast` with `Router` * Remove unneeded filter node from `dfp_preproc` module * Fix DFP file & class paths in documentation * Remove python blocks from documentation which didn't offer much to readers Closes #2034 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #2050
* Set CMake policy #`CMP0144` to `NEW` in C++ examples, avoids a deprecation warning * Set `-lto=auto` to avoid a warning about falling back to serialized compilation. Related to nv-morpheus/MRC#518 . Closes #1988 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #2054
## Description Manually resolving conflicts from forwar-merger PR #2060 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes.
) * Allow the user to optionally run something other than `bash` when using `docker/run_container_release.sh` and `docker/run_container_dev.sh` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #2061
Add support for writing DataFrames to parquet files in `write_to_file_stage.py`. Closes #980 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #1937
* Ensure each example pipeline produces a unique output file name * Add expected example output filenames to `.gitignore` * Update `ransomware_detection` pipeline to include a unique column * Update `ransomware_detection` adding a `--pipeline_batch_size` flag defaulting to the `model_max_batch_size` avoiding a config warning * Update `Config` to call `_validate_config` from `Config.freeze` instead of the attribute setters, avoids issue where setting attributes becomes order dependent ex: ```python from morpheus.config import Config config = Config() config.model_max_batch_size = 1024 # This line triggers a warning config.pipeline_batch_size = 1024 ``` Includes changes from PR #2040 Closes #2033 Closes #2035 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #2049
Some minor improvements to make `SharedProcessPool` related tests to run faster. Those are mostly combining multiple tests into one unit test function to reduce the number of times that the process pool being initialized and reset. The total execution time of `test_shared_process_pool.py` and `test_multi_processing_stage.py` should be improved from ~60s to ~30s (when running locally). Closes #1951 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #1950
Implemented C++ `MonitorStage` based on [indicators](https://github.com/p-ranav/indicators) library (supports progress bar display) - Each `MonitorStage` displays a progress bar with customized description tag, message unit and (optional) message count method. Customized font is supported. - Supports `ControlMessage` and `MessageMeta` as input type - Able to run multiple `MonitorStage`s within the same pipeline A quick demo:  - **Note**: `indicators` library does not have official support for displaying the progress bars with other log output. This PR uses this [workaround](p-ranav/indicators#107) to enable displaying progress bars alongside other log outputs. Need to merge nv-morpheus/utilities#78 Closes [#804](#804) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) - Christopher Harris (https://github.com/cwharris) URL: #1908
Forward-merge triggered by push to branch-24.10 that creates a PR to keep branch-25.02 up-to-date. If this PR is unable to be immediately merged due to conflicts, it will remain open for the team to manually merge. See [forward-merger docs](https://docs.rapids.ai/maintainers/forward-merger/) for more info.
Forward-merge triggered by push to branch-24.10 that creates a PR to keep branch-25.02 up-to-date. If this PR is unable to be immediately merged due to conflicts, it will remain open for the team to manually merge. See [forward-merger docs](https://docs.rapids.ai/maintainers/forward-merger/) for more info.
For ControlMessage, msg.tensors().count is a common pattern, calling msg.tensors() might require a bit more cost than we think. Add a `tensor_count` property to avoid the overhead. Closes #1876 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #2078
* Work-around known openai incompatibility with httpx v0.28 (openai/openai-python#1915) * Templatize the llm pip dependencies * Replace deprecated imports of openai from langchain and langchain_community Closes #2084 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: #2083
In anticipation of this downstream cuDF PR removing some functionality in `cudf._lib.utils` rapidsai/cudf#17586, this PR replaces that usage with equivalent usage from the stable `pylibcudf` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Matthew Roeschke (https://github.com/mroeschke) - David Gardner (https://github.com/dagardner-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #2082
Current time limit is 60 minutes, recent builds have come close to hitting this limit, and Conda builds are often time sensitive. Closes #2073 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: #2075
* Update CR year * IWYU fixes
) * Perform an `apt update` and `apt upgrade` during the Docker build picking up any security fixes available. * Add `urllib3` to `ci/release/download_deps.py` (unrelated fix) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) - Tad ZeMicheal (https://github.com/tzemicheal) URL: #2174
* Direct users to the main branch of https://github.com/nv-morpheus/morpheus_third_party_oss not a versioned branch * Add a third party notice to the models container * Remove `ci/release/download_deps.py` in favor of adopting the Rapids tools. ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Yuchen Zhang (https://github.com/yczhang-nv) - Tad ZeMicheal (https://github.com/tzemicheal) URL: #2176
Closes #2177 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #2178
* Update the `external/morpheus-visualizations` submodule to adopt fixes from nv-morpheus/morpheus-visualizations#50 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Yuchen Zhang (https://github.com/yczhang-nv) URL: #2186
* Update conda env files to ensure pulling pytorch from pypi Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * remove comments * update * WIP: Tue 03 Jun 2025 10:41:58 AM PDT Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update doc * WIP: Tue 03 Jun 2025 11:03:14 AM PDT [skip ci] Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> --------- Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
* Update dependencies for Morpheus `24.06` container (#2217) * Update conda env files to ensure pulling pytorch from pypi Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * remove comments * update * WIP: Tue 03 Jun 2025 10:41:58 AM PDT Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update doc * WIP: Tue 03 Jun 2025 11:03:14 AM PDT [skip ci] Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> --------- Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * Fix CR year Signed-off-by: David Gardner <dagardner@nvidia.com> * Update test runner Signed-off-by: David Gardner <dagardner@nvidia.com> * Update CR year Signed-off-by: David Gardner <dagardner@nvidia.com> * Remove broken link Signed-off-by: David Gardner <dagardner@nvidia.com> * Pin click version to avoid test breakage Signed-off-by: David Gardner <dagardner@nvidia.com> * Update CR Year Signed-off-by: David Gardner <dagardner@nvidia.com> * Pin setuptools Signed-off-by: David Gardner <dagardner@nvidia.com> * Pin numba to fix compat Signed-off-by: David Gardner <dagardner@nvidia.com> * Pin numba to fix compat Signed-off-by: David Gardner <dagardner@nvidia.com> * Move to examples section: Signed-off-by: David Gardner <dagardner@nvidia.com> --------- Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> Signed-off-by: David Gardner <dagardner@nvidia.com> Co-authored-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
* Update dependencies for Morpheus `24.06` container (#2217) * Update conda env files to ensure pulling pytorch from pypi * remove comments * update * WIP: Tue 03 Jun 2025 10:41:58 AM PDT * update doc * WIP: Tue 03 Jun 2025 11:03:14 AM PDT [skip ci] --------- * Fix CR year * Update test runner * Update CR year * Remove broken link * Pin click version to avoid test breakage * Update CR Year * Pin setuptools * Pin numba to fix compat * Pin numba to fix compat * Move to examples section: --------- Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> Signed-off-by: David Gardner <dagardner@nvidia.com> Co-authored-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
## Description * Resolving conflicts from #2299 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes.
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
## Description * Manually resolving conflicts from #2301 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes.
Signed-off-by: David Gardner <dagardner@nvidia.com>
* Set GPU runner to an earlier driver version * Document #2305 as a known issue * Pin `npy-append-array` to avoid version conflict with nvtabular * Update rapids shared workflows to 24.10 to fix the check action * Pin click version to <8.2 to avoid #2219 * Ensure the logger is reset in `test_cli.py` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) - https://github.com/Salonijain27 Approvers: - Will Killian (https://github.com/willkill07) URL: #2306
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
* Remove pin for npy-append-array, this work-around is not needed in 24.10 Signed-off-by: David Gardner <dagardner@nvidia.com> * Remove old 24.06 work-around Signed-off-by: David Gardner <dagardner@nvidia.com> * Pin the version of faiss to 1.9 to avoid a numpy dependency Signed-off-by: David Gardner <dagardner@nvidia.com> * Numba should be upper-cased Signed-off-by: David Gardner <dagardner@nvidia.com> * Remove npy-append-array pin from conda recipe Signed-off-by: David Gardner <dagardner@nvidia.com> --------- Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
## Description Manually resolving conflicts from #2308 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes.
Signed-off-by: David Gardner <dagardner@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fixes error caused when running Triton inference for multi-GPU for SID pipeline.
Description
Closes
When running Triton inference for SID pipeline using multi-gpu. It result on segment fault.
2024-12-09 23:24:38.378753895 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running TRTKernel_graph_torch_jit_3139280210422962738_0 node. Name:'TensorrtExecutionProvider_TRTKernel_graph_torch_jit_3139280210422962738_0_0' Status Message: TensorRT EP execution context enqueue failed.By Submitting this PR I confirm: