NVIDIA AI Blueprint: Vulnerability Analysis for Container Security

Overview

This repository is what powers the build experience, showcasing vulnerability analysis for container security using NVIDIA NIM microservices and NVIDIA NeMo Agent Toolkit.

The NVIDIA AI Blueprint demonstrates accelerated analysis on common vulnerabilities and exposures (CVE) at an enterprise scale, reducing mitigation from days and hours to just seconds. While traditional methods require substantial manual effort to pinpoint solutions for vulnerabilities, these technologies enable quick, automatic, and actionable CVE risk analysis using large language models (LLMs) and retrieval-augmented generation (RAG). With this blueprint, security analysts can expedite the process of determining whether a software package includes exploitable and vulnerable components using LLMs and event-driven RAG triggered by the creation of a new software package or the detection of a CVE.

Software components

The following are used by this blueprint:

Target audience

This blueprint is for:

Security analysts and IT engineers: People analyzing vulnerabilities and ensuring the security of containerized environments.
AI practitioners in cybersecurity: People applying AI to enhance cybersecurity, particularly those interested in using the NeMo Agent Toolkit and NIMs for faster vulnerability detection and analysis.

Prerequisites

NVAIE developer licence
API keys for vulnerability databases, search engines, and LLM model service(s).
- Details can be found in this later section: Obtain API keys

Hardware requirements

Below are the hardware requirements for each component of the vulnerability analysis workflow.

The overall hardware requirements depend on selected workflow configuration. At a minimum, the hardware requirements for workflow operation must be met. The LLM NIM and Embedding NIM hardware requirements only need to be met if self-hosting these components. See Using self-hosted NIMs, Customizing the LLM models and Customizing the embedding model sections for more information.

(Optional) LLM NIM: Meta Llama 3.1 70B Instruct Support Matrix
- This workflow makes heavy use of parallel LLM calls to accelerate processing. For improved parallel performance (for example, in production workloads), we recommend 8x or more H100s for LLM inference.
(Optional) Embedding NIM: NV-EmbedQA-E5-v5 Support Matrix

API definition

OpenAPI Specification

Use case description

Determining the impact of a documented CVE on a specific project or container is a labor-intensive and manual task, especially as the rate of new reports into the CVE database accelerates. This process involves the collection, comprehension, and synthesis of various pieces of information to ascertain whether immediate remediation is necessary upon the identification of a new CVE.

Current challenges in CVE analysis:

Information collection: The process involves significant manual labor to collect and synthesize relevant information.
Decision complexity: Decisions on whether to update a library impacted by a CVE often hinge on various considerations, including:
- Scan false positives: Occasionally, vulnerability scans may incorrectly flag a library as vulnerable, leading to a false alarm.
- Mitigating factors: In some cases, existing safeguards within the environment may reduce or negate the risk posed by a CVE.
- Lack of required environments or dependencies: For an exploit to succeed, specific conditions must be met. The absence of these necessary elements can render a vulnerability irrelevant.
Manual documentation: Once an analyst has determined the library is not affected, a Vulnerability Exploitability eXchange (VEX) document must be created to standardize and distribute the results.

The efficiency of this process can be significantly enhanced through the deployment of an automated LLM agent workflow, leveraging generative AI to improve vulnerability defense while decreasing the load on security teams.

How it works

The workflow operates using a Plan-and-Execute-style LLM pipeline for CVE impact analysis. The process begins with an LLM planner that generates a context-sensitive task checklist. This checklist is then executed by an LLM agent equipped with Retrieval-Augmented Generation (RAG) capabilities. The gathered information and the agent's findings are subsequently summarized and categorized by additional LLM nodes to provide a final verdict.

Tip

The workflow is adaptable, with support for NIM and OpenAI LLM APIs. NIM models can be hosted on build.nvidia.com or self-hosted.

Key components

The detailed architecture consists of the following components:

Security scan result: The workflow begins by inputting the identified CVEs from a container security scan as input. This can be generated from a container image scanner of your choosing such as Anchore.
PreProcessing: All the below actions are encapsulated by multiple NeMo Agent toolkit functions to prepare the data for use with the LLM engine.
- Code repository and documentation: The blueprint pulls code repositories and documentation provided by the user. These repositories are processed through an embedding model, and the resulting embeddings are stored in vector databases (VDBs) for the agent's reference.
  - Vector database: Various vector databases can be used for the embedding. We currently utilize FAISS for the VDB because it does not require an external service and is simple to use. Any vector store can be used, such as NVIDIA cuVS, which would provide accelerated indexing and search.
  - Lexical search: As an alternative, a lexical search is available for use cases where creating an embedding is impractical due to a large number of source files in the target container.
- Software Bill of Materials (SBOM): A Software Bill of Materials (SBOM) is a machine-readable manifest of all the dependencies of a software package or container. The blueprint cross-references every entry in the SBOM for known vulnerabilities and looks at the code implementation to see whether the implementation puts users at risk—just as a security analyst would do. For this reason, starting with an accurate SBOM is an important first step. SBOMs can be generated for any container using the open-source tool Syft. For more information on generating SBOMs for your containers, see the SBOM documentation.
- Web vulnerability intel: The system collects detailed information about each CVE through web scraping and data retrieval from various public security databases, including GHSA, Redhat, Ubuntu, and NIST CVE records, as well as tailored threat intelligence feeds.
Core LLM engine: The below actions comprise the core LLM engine and are each implemented as NeMo Agent toolkit functions within the workflow.
- Checklist generation: Leveraging the gathered information about each vulnerability, the checklist generation node creates a tailored, context-sensitive task checklist designed to guide the impact analysis. (See src/vuln_analysis/functions/cve_checklist.py.)
- Task agent: At the core of the process is an LLM agent iterating through each item in the checklist. For each item, the agent answers the question using a set of tools which provide information about the target container. The tools tap into various data sources (web intel, vector DB, search etc.), retrieving relevant information to address each checklist item. The loop continues until the agent resolves each checklist item satisfactorily. (See src/vuln_analysis/functions/cve_agent.py.)
- Summarization: Once the agent has compiled findings for each checklist item, these results are condensed by the summarization node into a concise, human-readable paragraph. (See src/vuln_analysis/functions/cve_summarize.py.)
- Justification Assignment: Given the summary, the justification status categorization node then assigns a resulting VEX (Vulnerability Exploitability eXchange) status to the CVE. We provided a set of predefined categories for the model to choose from. (See src/vuln_analysis/functions/cve_justify.py.) If the CVE is deemed exploitable, the reasoning category is "vulnerable." If there is no vulnerable packages detected from the SBOM or insufficient intel gathered the agent is bypassed and an appropriate label is provided. If it is not exploitable, there are 10 different reasoning categories to explain why the vulnerability is not exploitable in the given environment:
  - false_positive
  - code_not_present
  - code_not_reachable
  - requires_configuration
  - requires_dependency
  - requires_environment
  - protected_by_compiler
  - protected_at_runtime
  - protected_by_perimeter
  - protected_by_mitigating_control
Output: At the end of the workflow run, an output file including all the gathered and generated information is prepared for security analysts for a final review. (See src/vuln_analysis/functions/cve_file_output.py.)

Warning

All output should be vetted by a security analyst before being used in a cybersecurity application.

NIM microservices

The NeMo Agent toolkit can utilize various embedding model and LLM endpoints, and is optimized to use NVIDIA NIM microservices (NIMs). NIMs are pre-built containers for the latest AI models that provide industry-standard APIs and optimized inference for the given model and hardware. Using NIMs enables easy deployment and scaling for self-hosted model inference.

The current default embedding NIM model is nv-embedqa-e5-v5, which was selected to balance speed and overall workflow accuracy. The current default LLM model is the llama-3.1-70b-instruct NIM, with specifically tailored prompt engineering and edge case handling. Other models are able to be substituted for either the embedding or LLM model, such as smaller, fine-tuned NIM LLM models or other external LLM inference services. Subsequent updates will provide more details about fine-tuning and data flywheel techniques.

Note

Within the NeMo Agent toolkit workflow, the LangChain framework is employed to deploy all LLMs and agents, and the LangGraph framework is used for orchestration, streamlining efficiency and reducing the need for duplicative efforts.

Tip

Routinely checked validation datasets are critical to ensuring proper and consistent outputs. Learn more about our test-driven development approach in the section on testing and validation.

Getting started

Install system requirements

git
git-lfs
bsdtar (from libarchive) - Used for extracting RPM source archives. Install via apt install libarchive-tools (Debian/Ubuntu) or dnf install bsdtar (Fedora/RHEL).
Since the workflow uses NVIDIA NeMo Agent Toolkit, the NeMo Agent toolkit requirements also need to be installed.

Obtain API keys

To run the workflow you need to obtain API keys for the following APIs. These will be needed in a later step to Set up the environment file.

Required API Keys: These APIs are required by the workflow to retrieve vulnerability information from databases, perform online searches, and execute LLM queries.
- GitHub Security Advisory (GHSA) Database
  - Follow these instructions to create a personal access token. No repository access or permissions are required for this API.
  - This will be used in the GHSA_API_KEY environment variable.
- National Vulnerability Database (NVD)
  - Follow these instructions to create an API key.
  - This will be used in the NVD_API_KEY environment variable.
- SerpApi
  - Go to https://serpapi.com/ and create a SerpApi account. Once signed in, navigate to Your Account > Api Key.
  - This will be used in the SERPAPI_API_KEY environment variable.
- NVIDIA Inference Microservices (NIM)
  - There are two possible methods to generate an API key for NIM:
    - Sign in to the NVIDIA Build portal with your email.
      - Click on any model, then click "Get API Key", and finally click "Generate Key".
    - Sign in to the NVIDIA NGC portal with your email.
      - Select your organization from the dropdown menu after logging in. You must select an organization which has NVIDIA AI Enterprise (NVAIE) enabled.
      - Click on your account in the top right, select "Setup" from the dropdown.
      - Click the "Generate Personal Key" option and then the "+ Generate Personal Key" button to create your API key.
  - This will be used in the NVIDIA_API_KEY environment variable.
- REDHAT container registry (Recommended but not compulsory)
  - To get source images from a Red Hat container registry using registry service account tokens. You will need to create a registry service account
  - Steps:
    - Sign in to the registry service accout
    - Press on New service Account Button
    - Fill the Name (ex. 'test-case') and Description fields and Click the Create botton
    - Token user name for example '11008101|test-case' for REGISTRY_REDHAT_USERNAME environment variable
    - Token password a long string for REGISTRY_REDHAT_PASSWORD environment variable

The workflow can be configured to use other LLM services as well, see the Customizing the LLM models section for more info.

Set up the workflow repository

Clone the repository and set an environment variable for the path to the repository root.

export REPO_ROOT=$(git rev-parse --show-toplevel)

All commands are run from the repository root unless otherwise specified.

Set up the environment file

First we need to create an .env file in the REPO_ROOT, and add the API keys you created in the earlier Obtain API keys step.

cd $REPO_ROOT
cat <<EOF > .env
GHSA_API_KEY="your GitHub personal access token"
NVD_API_KEY="your National Vulnerability Database API key"
NVIDIA_API_KEY="your NVIDIA Inference Microservices API key"
SERPAPI_API_KEY="your SerpApi API key"
EOF

These variables need to be exported to the environment:

export $(cat .env | xargs)

Authenticate Docker with NGC

In order to pull images required by the workflow from NGC, you must first authenticate Docker with NGC. You can use same the NVIDIA API Key obtained in the Obtain API keys section (saved as NVIDIA_API_KEY in the .env file).

echo "${NVIDIA_API_KEY}" | docker login nvcr.io -u '$oauthtoken' --password-stdin

Build the Docker containers

Next, build the vuln-analysis container from source using the following command. This ensures that the container includes all the latest changes from the repository.

cd $REPO_ROOT

# Build the vuln-analysis container
docker compose build vuln-analysis

Start the Docker containers

There are two supported configurations for starting the Docker containers. Both configurations utilize docker compose to start the service:

NVIDIA-hosted NIMs: The workflow is run with all computation being performed by NIMs hosted in NVIDIA GPU Cloud. This is the default configuration and is recommended for most users getting started with the workflow.
1. When using NVIDIA-hosted NIMs, only the docker-compose.yml configuration file is required.
Self-hosted NIMs: The workflow is run using self-hosted LLM NIM services. This configuration is more advanced and requires additional setup to run the NIM services locally.
1. When using self-hosted NIMs, both the docker-compose.yml and docker-compose.nim.yml configuration files are required.

These two configurations are illustrated by the following diagram:

Before beginning, ensure that the environment variables are set correctly. Both configurations require the same environment variables to be set. More information on setting these variables can be found in the Obtain API keys section.

Tip

The container binds to port 8080 by default. If you encounter a port collision error (for example, Bind for 0.0.0.0:8080 failed: port is already allocated), you can set the environment variable NGINX_HOST_HTTP_PORT to specify a custom port before launching docker compose. For example:

export NGINX_HOST_HTTP_PORT=8081

#... docker compose commands...

Using NVIDIA-hosted NIMs

When running the workflow in this configuration, only the vuln-analysis service needs to be started since we will utilize NIMs hosted by NVIDIA. The vuln-analysis container can be started using the following command:

cd ${REPO_ROOT}
docker compose up -d

The command above starts the container in the background using the detached mode, -d. We can confirm the container is running via the following command:

docker compose ps

Next, we need to attach to the vuln-analysis container to access the environment where the workflow command line tool and dependencies are installed.

docker compose exec -it vuln-analysis bash

Continue to the Running the workflow section to run the workflow.

Using self-hosted NIMs

To run the workflow using self-hosted NIMs, we use a second docker compose configuration file, docker-compose.nim.yml, which adds the self-hosted NIM services to the workflow. Utilizing a second configuration file allows for easy switching between the two configurations while keeping the base configuration file the same.

Note

The self-hosted NIM services require additional GPU resources to run. With this configuration, the LLM NIM, embedding model NIM, and the vuln-analysis service will all be launched on the same machine. Ensure that you have the necessary hardware requirements for all three services before proceeding (multiple services can share the same GPU).

To use multiple configuration files, we need to specify both configuration files when running the docker compose command. You will need to specify both configuration files for every docker compose command. For example:

docker compose -f docker-compose.yml -f docker-compose.nim.yml [NORMAL DOCKER COMPOSE COMMAND]

For example, to start the vuln-analysis service with the self-hosted NIMs, you would run:

cd ${REPO_ROOT}
docker compose -f docker-compose.yml -f docker-compose.nim.yml up -d

Next, we need to attach to the vuln-analysis container to access the environment where the workflow command line tool and dependencies are installed.

docker compose -f docker-compose.yml -f docker-compose.nim.yml exec -it vuln-analysis bash

Continue to the Running the workflow section to run the workflow.

Running the workflow

Once the services have been started, the workflow can be run using either the Quick start user guide notebook for an interactive step-by-step process, or directly from the command line.

From the quick start user guide notebook

To run the workflow in an interactive notebook, connect to the Jupyter notebook at http://localhost:8000/lab. Once connected, navigate to the notebook located at quick_start/quick_start_guide.ipynb and follow the instructions.

Tip

If you are running the workflow on a remote machine, you can forward the port to your local machine using SSH. For example, to forward port 8000 from the remote machine to your local machine, you can run the following command from your local machine:

ssh -L 8000:127.0.0.1:8000 <remote_host_name>

From the command line

The vulnerability analysis workflow is designed to be run using the aiq command line tool installed within the vuln-analysis container. This section describes how get started using the command line tool. For more detailed information about the command line interface, see the NeMo Agent toolkit Command Line Interface (CLI) documentation.

Workflow configuration file

The workflow settings are controlled using configuration files. These are YAML files that define the functions, tools, and models to use in the workflow. Example configuration files are located in the configs/ folder.

Note

The configs/ and data/ directories are symlinks pointing to the actual file locations in the src/vuln_analysis/configs/ and src/vuln_analysis/data/ directories respectively. The symlinks are available for convenience.

A brief description of each configuration file is as follows:

config.yml: This configuration file defines the functions, tools, and models used by the vulnerability analysis workflow, as described above in Key Components section.
config-tracing.yml: This configuration file is identical to config.yml but adds configuration for observing traces of this workflow in Phoenix.

There are three main modalities that the workflow can be run in using the following commands:

aiq run: The workflow will process the input data, then it will shut down after it is completed. This modality is suitable for rapid iteration during testing and development.
aiq serve: The workflow is turned into a microservice which will run indefinitely, which is suitable for using in production.
aiq eval: Similar to the aiq run command. However, in addition to running the workflow, it is also used for profiling and evaluating accuracy of the workflow.

For a breakdown of the configuration file and available options, see the Configuration file reference section. To customize the configuration files for your use case, see Customizing the workflow.

Example command: `aiq run`

The workflow can be started using the following command:

aiq run --config_file=${CONFIG_FILE} --input_file=data/input_messages/morpheus:23.11-runtime.json

In the command, ${CONFIG_FILE} is the path to the configuration file you want to use. For example, to run the workflow with the config.yml configuration file, you would run:

aiq run --config_file=configs/config.yml --input_file=data/input_messages/morpheus:23.11-runtime.json

When the workflow runs to completion, you should see logs similar to the following:

Vulnerability 'GHSA-3f63-hfp8-52jq' affected status: FALSE. Label: code_not_reachable
Vulnerability 'CVE-2023-50782' affected status: FALSE. Label: requires_configuration
Vulnerability 'CVE-2023-36632' affected status: FALSE. Label: code_not_present
Vulnerability 'CVE-2023-43804' affected status: TRUE. Label: vulnerable
Vulnerability 'GHSA-cxfr-5q3r-2rc2' affected status: TRUE. Label: vulnerable
Vulnerability 'GHSA-554w-xh4j-8w64' affected status: TRUE. Label: vulnerable
Vulnerability 'GHSA-3ww4-gg4f-jr7f' affected status: FALSE. Label: requires_configuration
Vulnerability 'CVE-2023-31147' affected status: FALSE. Label: code_not_present
--------------------------------------------------
Workflow Result:
{"input":{"scan":{"id":"8351fd75-4798-42c9-81d8-a43d7df838fd","type":null,"started_at":"2025-06-25T20:21:19.698253","completed_at":"2025-06-25T20:30:09.667598","vulns":[{"vuln_id":"GHSA-3f63-hfp8-52jq","description":null,"score":null,"severity":null,"published_date":null,"last_modified_date":null,"url":null,"feed_group":null,"package":null,"package_version":null,"package_name":null,"package_type":null},{"vuln_id":"CVE-2023-50782","description":null,"score":null,"severity":null,"published_date":null,"last_modified_date":null,"url":null,"feed_group":null,"package":null,"package_version":null,"package_name":null,"package_type":null},{"vuln_id":"CVE-2023-36632","description":null,"score":null,"severity":null,"published_date":null,"last_modified_date":null,"url":null,"feed_group":null,"package":null,"package_version":null,"package_name":null,
...

Warning

The output you receive from the workflow may not be identical as the output in the example above. The output may vary due to the non-deterministic nature of the LLM models.

Reviewing the output

The full workflow JSON output is also stashed by default at .tmp/output.json. The output JSON includes the following top level fields:

input: contains the inputs that were provided to the workflow, such as the container and repo source information, the list of vulnerabilities to scan, etc.
info: contains additional information collected by the workflow for decision making. This includes paths to the generated VDB files, intelligence from various vulnerability databases, the list of SBOM packages, and any vulnerable dependencies that were identified.
output: contains the output from the core LLM Engine, including the generated checklist, analysis summary, and justification assignment.

In addition to the raw JSON output, you can also view a Markdown-formatted report for each CVE in the .tmp/vulnerability_markdown_reports directory. This view is helpful for human analysts reviewing the results.

Tip

To return detailed steps taken by the LLM agent in the output, set return_intermediate_steps to true for the cve_agent_executor function in the configuration file. This can be helpful for explaining the output, and for troubleshooting unexpected results.

Example command: `aiq serve`

Similarly, to run the workflow with the aiq serve command, you would run:

aiq serve --config_file=configs/config.yml --host 0.0.0.0 --port 26466

This command starts an HTTP server that listens on port 26466 and runs the workflow indefinitely, waiting for incoming data to process. This is useful if you want to trigger the workflow on demand via HTTP requests.

Once the server is running, you can send a POST request to the /generate endpoint with the input parameters in the request body. The workflow will process the input data and return the output in the terminal and the given output path in the config file.

Here's an example using curl to send a POST request. From a new terminal outside of the container, go to the root of the cloned git repository, and run:

curl -X POST --url http://localhost:26466/generate --header 'Content-Type: application/json' --data @data/input_messages/morpheus:23.11-runtime.json

In this command:

http://localhost:26466/generate is the URL of the server and endpoint.
The -d option specifies the data file being sent in the request body. In this case, it's pointing to the input file morpheus:23.11-runtime.json under the data/input_messages/ directory. You can refer to this file as an example of the expected data format.
- Since it uses a relative path, it's important to run the curl command from the root of the git repository. Alternatively, you can modify the relative path in the command to directly reference the example json file.

After processing the request, the server will return the the results from the curl request and save the results to the output path specified in the configuration file. The server will also display log and summary results from the workflow as it's running. Additional submissions to the server will append the results to the specified output file.

The workflow also provides a /generate/async endpoint for sending requests to workflow asynchronously. The following curl command is an example of using this endpoint. The request is processed the same as the /generate endpoint but without blocking until the request has completed processing.

curl -X POST --url http://localhost:26466/generate/async --header 'Content-Type: application/json' --data @data/input_messages/morpheus:23.11-runtime.json

The above command immediately returns a response similar to the following:

{"job_id":"f79951ee-aeea-4a2d-b103-a0a8c15e314b","status":"submitted"}

The status of the request can then be polled using /generate/async/job/{job_id}. For example,

curl --request GET --url http://localhost:26466/generate/async/job/f79951ee-aeea-4a2d-b103-a0a8c15e314b

Example command: `aiq eval`

The workflow configuration file, config.yml, includes settings in the eval section for use with the NeMo Agent toolkit profiler. The profiler can be started using the following command:

aiq eval --config_file=configs/config.yml

Reviewing the profiling results

The profiler collects usage statistics and stores them in the .tmp/eval/cve_agent directory as configured in config.yml. In this directory, you will find the following files:

all_requests_profiler_traces.json
gantt_chart.png
inference_optimization.json
standardized_data_all.csv
workflow_output.json
workflow_profiling_metrics.json
workflow_profiling_report.txt

More information about analyzing the profiling results can be found here in the NeMo Agent toolkit documentation.

Observing traces in Phoenix

A separate workflow configuration file, config-tracing.yml is provided that enables tracing in the workflow using Phoenix.

First, run the following command in separate terminal within the container to start a Phoenix server on port 6006:

phoenix serve

You can now use config-tracing.yml to run the workflow with tracing enabled:

aiq run --config_file=configs/config-tracing.yml --input_file=data/input_messages/morpheus:23.11-runtime.json

Tracing will also be enabled with config-tracing.yml when running workflow with aiq serve and aiq eval.

Open your browser and navigate to http://localhost:6006 to view the traces.

Command line interface (CLI) reference

The main entrypoint is the aiq CLI tool with built-in documentation using the --help command. For example, to see what commands are available, you can run:

(.venv) root@04a3a12a687d:/workspace# aiq --help
Usage: aiq [OPTIONS] COMMAND [ARGS]...

  Main entrypoint for the AIQ Toolkit CLI

Options:
  --version                       Show the version and exit.
  --log-level [debug|info|warning|error|critical]
                                  Set the logging level  [default: INFO]
  --help                          Show this message and exit.

Commands:
  configure  Configure AIQ Toolkit developer preferences.
  eval       Evaluate a workflow with the specified dataset.
  info       Provide information about the local AIQ Toolkit environment.
  mcp        Run an AIQ Toolkit workflow using the mcp front end.
  registry   Utility to configure AIQ Toolkit remote registry channels.
  run        Run an AIQ Toolkit workflow using the console front end.
  serve      Run an AIQ Toolkit workflow using the fastapi front end.
  start      Run an AIQ Toolkit workflow using a front end configuration.
  uninstall  Uninstall an AIQ Toolkit plugin packages from the local...
  validate   Validate a configuration file
  workflow   Interact with templated workflows.

Configuration file reference

The configuration defines how the workflow operates, including pipeline nodes, agent tools, LLMs, embedders, and general settings. More details about the NeMo Agent toolkit workflow configuration file can be found here.

Reference configuration files:

Environment	File
Local development / Docker Compose	`src/vuln_analysis/configs/config-http-openai.yml`
Cloud-native (Kubernetes / Kustomize)	`kustomize/base/exploit-iq-config.yml`

Field meanings for each pipeline node are defined in the corresponding Pydantic config class under src/vuln_analysis/functions/ and src/vuln_analysis/tools/. The workflow graph is wired in src/vuln_analysis/register.py via CVEAgentWorkflowConfig.

Local vs. cloud-native differences

The table below lists deliberate differences between the two reference files. Shared nodes (for example, cve_checklist, semantic search tools, and cve_generate_cvss) are not repeated here even when their parameter values differ.

Area	Local (`config-http-openai.yml`)	Cloud-native (`exploit-iq-config.yml`)
Cache / data paths	`.cache/am_cache/*`	`${EXPLOIT_IQ_DATA_DIR:-/exploit-iq-data/}*` subdirectories (`git`, `pickle`, `rpms`, `vdb`, `code_index`, `checker`)
LLM `_type`	Hard-coded `openai` for all models	`${LLM_TYPE_*:-nim}` per model, with per-model `api_key` and `base_url` env vars (for example, `code_vdb_retriever_llm` at lines 210–217)
`cve_verify_vuln_package`	Defined in `functions` and wired in `workflow` between `cve_process_sbom` and `cve_checklist`	No `cve_verify_vuln_package` entry; after `cve_process_sbom`, the next node is `cve_checklist` (lines 59–63)
`cve_fetch_patches`	Defined in `functions` and wired in `workflow`	Not defined in `functions` or `workflow`
`cve_fetch_intel.intel_plugin_config`	`endpoint: http://localhost:8080/...`; no `token_path` / `verify_path`	`endpoint: CALLBACK_URL_PLACEHOLDER/...`; `token_path` and `verify_path` for Kubernetes service-account auth (lines 47–57)
`cve_fetch_intel.retry_on_client_errors`	Omitted (defaults to `true`)	Explicitly set to `false`
`cve_agent_executor.tool_names`	Includes `Configuration Scanner` and `Import Usage Analyzer` (10 tools total)	Same eight core tools only; no `Configuration Scanner` or `Import Usage Analyzer` in the agent's `tool_names` list (lines 99–110). Both configs still define semantic search tools and reference them from `cve_generate_cvss.tool_names` (lines 120–128)
`cve_http_output`	`url: http://localhost:8080`; no top-level auth fields	`url: CALLBACK_URL_PLACEHOLDER`; `auth_type: bearer` with `token_path` / `verify_path`; `mlops_config.enable_verify: true`
`cve_source_acquisition.base_checker_dir`	Not set (uses model default)	Set to `${EXPLOIT_IQ_DATA_DIR:-/exploit-iq-data/}checker`

1. General configuration (`general`)

Settings for the NeMo Agent toolkit runtime, not specific to any single pipeline node.

front_end._type: Front-end type (fastapi).
front_end.endpoints: HTTP endpoints exposed by the service. Each entry has path, method, description, and function_name (for example, /health → health_check).
use_uvloop: Use the uvloop event loop for performance. Set to false when debugging.
telemetry.tracing.phoenix._type: Tracing backend type (phoenix).
telemetry.tracing.phoenix.endpoint: OTLP traces endpoint (default ${OTEL_TRACES_ENDPOINT:-http://localhost:6006/v1/traces}).
telemetry.tracing.phoenix.project: Phoenix project name (default cve_agent).

2. Functions (`functions`)

Each key is a registered function name; _type selects the implementation. Functions are grouped below by pipeline role.

Preprocessing and early pipeline nodes

cve_clone_and_deps (_type: cve_clone_and_deps): Clones source repositories and installs dependencies early in the pipeline, before vulnerability verification. VDB/indexing is deferred to cve_segmentation.
- base_git_dir: Directory for cloned git repositories.
- base_pickle_dir: Directory for pickle document cache files.
- base_rpm_dir: Directory for downloaded RPM files.
- ignore_errors (optional, default false): Continue on clone/install errors.
cve_fetch_intel (_type: cve_fetch_intel): Fetches CVE intelligence from NVD, CVE Details, and configured intel plugins.
- rpm_user_type: Brew/OSIDB profile — internal (Red Hat VPN, enables OSIDB intel) or external. Default external; both reference configs use ${RPM_USER_TYPE:-internal}.
- retry_on_client_errors (default true): Retry when the HTTP client cannot connect. Cloud config sets this to false.
- intel_plugin_config: Plugin wrapper with plugin_name and plugin_config dict. Both configs use SimpleHttpIntelPlugin:
  - plugin_config.source: Human-readable label for the intel source.
  - plugin_config.endpoint: URL template with {vuln_id} placeholder.
  - plugin_config.api_key (optional): Bearer token sent as Authorization header.
  - plugin_config.token_path (optional): Path to a file containing a bearer token (used in cloud config for Kubernetes service-account auth).
  - plugin_config.verify_path (optional): TLS certificate path for HTTPS verification.
cve_calculate_intel_score (_type: cve_calculate_intel_score): Scores intel quality and can gate downstream analysis.
- llm_name: LLM used for scoring (intel_source_score_llm).
- generate_intel_score (default true): Whether to compute intel scores.
- intel_low_score (default 51): Score threshold below which analysis may be skipped.
- insist_analysis (default false): Continue analysis even when the score is below intel_low_score.
cve_process_sbom (_type: cve_process_sbom): Prepares and validates SBOM input (file, HTTP URL, or manual package list).
- max_retries (default 10): Retries when loading an SBOM from a URL.
cve_verify_vuln_package (_type: cve_verify_vuln_package): For SOURCE analysis, parses lock files and verifies that the CVE's vulnerable package is present and that the installed version is actually vulnerable. Present in the local config only.
- skip (default false): Skip verification and proceed with all CVEs.
- base_git_dir: Base directory for cloned git repositories.
- llm_name (default checklist_llm): LLM for version comparison and package matching.
cve_segmentation (_type: cve_segmentation): Builds vector databases and Tantivy code indexes from already-cloned repositories. Runs after package verification.
- agent_name: Agent executor name (cve_agent_executor). Used to decide which indexes/VDBs to build based on enabled tools.
- embedder_name: Embedder for VDB creation (nim_embedder).
- base_git_dir, base_vdb_dir, base_code_index_dir, base_pickle_dir: Cache directories for repos, VDBs, code indexes, and pickles.
- ignore_code_embedding (default false): Skip building the code vector database. Both reference configs set true.
- ignore_errors (optional, default false): Continue on segmentation errors.
- ignore_code_index (optional, default false): Skip building the Tantivy code index.
cve_source_acquisition (_type: cve_source_acquisition): Downloads source containers, extracts layers, and locates package sources for the RPM CVE Checker pipeline.
- base_git_dir, base_pickle_dir, base_rpm_dir: Shared cache directories.
- base_checker_dir (optional): Root for checker-specific artifacts. Set in cloud config; defaults to .cache/am_cache/checker.
- rpm_user_type: Brew profile (internal or external).
cve_checker_segmentation (_type: cve_checker_segmentation): Builds a scoped Tantivy lexical index from extracted RPM source files.
- base_checker_dir: Root for checker artifacts.
- base_code_index_dir: Tantivy index storage directory.
- include_extensions (optional): File extensions to index.
cve_fetch_patches (_type: cve_fetch_patches): Fetches vulnerability fix patches from intel references and OSV for the details UI. Present in the local config only.
- llm_name (optional): LLM for Chromium CL disambiguation.
- max_concurrency (default 5): Concurrent patch fetch operations.

Core analysis nodes

cve_checklist (_type: cve_checklist): Generates a tailored, context-sensitive checklist for impact analysis.
- llm_name: LLM for checklist generation (checklist_llm).
- agent_name (default cve_agent_executor): Agent whose tool_names drive which checklist questions are generated.
- prompt (optional): Custom prompt text or path to a prompt file.
cve_agent_executor (_type: cve_agent_executor): ReAct agent that iterates through checklist items using the configured tools and gathered intel.
- llm_name: LLM for the agent (cve_agent_executor_llm).
- tool_names: List of tool function names (see Agent tools below).
- max_concurrency (default null): Maximum concurrent agent invocations; null means unlimited.
- max_iterations (default 10): Maximum ReAct iterations per checklist item.
- prompt (optional): Custom agent prompt text or file path.
- prompt_examples (default false): Include few-shot examples in the prompt.
- replace_exceptions (default false in model; true in both reference configs): Replace tool exceptions with a custom message.
- replace_exceptions_value: Message used when replace_exceptions is true.
- return_intermediate_steps (default false): Include intermediate agent steps in output.
- transitive_search_tool_enabled (default true): Enable the Call Chain Analyzer tool.
- cve_web_search_enabled (default true): Enable the CVE Web Search tool.
- uber_jar_file_threshold (default 600): Source file count above which a JAR is treated as an uber-JAR, disabling same-artifact optimizations.
- verbose (default false): Verbose agent logging.
- context_window_token_limit (default 5000): Token threshold for pruning old messages.
cve_package_code_agent (_type: cve_package_code_agent): Level 1 Package Code Agent for the RPM CVE Checker — investigates CVEs using extracted source and the scoped code index.
- llm_name, tool_names, max_iterations (default 10): Agent LLM, tools, and iteration limit.
- base_checker_dir, base_code_index_dir, base_rpm_dir: Artifact and cache directories.
- rpm_user_type: Brew profile for reference-patch SRPM lookup.
- context_window_token_limit (default 5000): Context pruning threshold.
- Additional optional fields (reference mining, git search, repo resolution) are defined in cve_package_code_agent.py.
cve_build_agent (_type: cve_build_agent): Level 2 Build Agent — checks whether vulnerable code is compiled into binaries and whether hardening flags mitigate the CVE.
- llm_name, tool_names, max_iterations (default 5 in model; 10 in both reference configs).
- base_checker_dir: Checker artifact root.
- context_window_token_limit (default 5000).
cve_checker_report (_type: cve_checker_report): Synthesizes the final L1/L2 checker report.
- llm_name: LLM for report generation.
- base_checker_dir: Checker artifact root.
cve_generate_cvss (_type: cve_generate_cvss): Agent that generates CVSS vector strings and scores.
- skip (default false; both reference configs set true): Skip CVSS generation entirely.
- llm_name, tool_names, max_concurrency, max_iterations, prompt, prompt_examples (default true), replace_exceptions (default false), replace_exceptions_value, return_intermediate_steps, verbose: Same semantics as cve_agent_executor.
cve_summarize (_type: cve_summarize): Produces a concise summary paragraph from agent results.
- llm_name: LLM for summarization (summarize_llm).
cve_justify (_type: cve_justify): Assigns a VEX justification label and reason to each CVE.
- llm_name: LLM for justification (justify_llm).
cve_generate_vex (_type: cve_generate_vex): Generates a VEX document for vulnerable components.
- skip (default false): Skip VEX generation.
- vex_format (default csaf): Output format (cloud config shows a commented # vex_format: csaf example).

Output and health

cve_http_output (_type: cve_http_output): Posts workflow results to an HTTP endpoint.
- url: Base URL for the callback service.
- endpoint (default /api/v1/reports): Path appended to url for successful reports.
- auth_type (default disabled): bearer, basic, or disabled. Cloud config uses bearer.
- token (optional): Inline auth token.
- token_path (optional): File path to a bearer token (Kubernetes service account in cloud config).
- verify_path (optional): TLS certificate for HTTPS verification.
- username, password (optional): Basic-auth credentials.
- failure_endpoint (default /api/v1/reports/failed): Endpoint for failure reports when code indexing fails.
- enable_mlops (default false): Forward output to an MLOps endpoint.
- mlops_config: MLOps sub-configuration:
  - mlops_url: MLOps service URL.
  - auth_type (default None): bearer, basic, or None.
  - token_path, verify_path: Auth token and TLS cert paths.
  - enable_verify (default false): Enable TLS verification for MLOps.
health_check (_type: health_check): Health-check function wired to GET /health.

Agent tools

Tool entries in functions are referenced by name from agent tool_names lists. Display names in the reference configs map to _type values as follows:

Config key	`_type`	Key fields
`Code Semantic Search`	`local_vdb_retriever`	`embedder_name`, `llm_name`, `vdb_type` (`code` or `doc`), `return_source_documents` (default `false`)
`Docs Semantic Search`	`local_vdb_retriever`	Same as above with `vdb_type: doc`
`Code Keyword Search`	`lexical_code_search`	`top_k` (default `5`); `base_code_index_dir` (optional)
`CVE Web Search`	`serp_wrapper`	`max_retries`
`Call Chain Analyzer`	`transitive_code_search`	No additional fields in the config model
`Function Caller Finder`	`calling_function_name_extractor`	No additional fields in the config model
`Function Locator`	`package_and_function_locator`	No additional fields in the config model
`Function Library Version Finder`	`calling_function_library_version_finder`	No additional fields in the config model
`Configuration Scanner`	`configuration_scanner`	`max_results` (default `15`), `context_lines` (default `5`). Defined in the local reference config only; wired into `cve_agent_executor.tool_names` locally but not present in `exploit-iq-config.yml`.
`Import Usage Analyzer`	`import_usage_analyzer`	`max_files` (default `20`). Defined in the local reference config only; wired into `cve_agent_executor.tool_names` locally but not present in `exploit-iq-config.yml`.
`Container Analysis Data`	`container_image_analysis_data`	No additional fields; returns pre-analyzed container data from earlier workflow stages
`Source Grep`	`source_grep`	`base_checker_dir`, `max_results` (default `50`), `context_lines` (default `3`)

Note: enable_transitive_search and enable_functions_usage_search appear in the YAML configs under tool entries but are not defined in the tool config models; tool availability is controlled via cve_agent_executor.tool_names and the transitive_search_tool_enabled / cve_web_search_enabled flags on the agent.

Alternate / optional functions (not in reference configs)

These functions are supported by the codebase but not wired in the two reference configs above:

cve_generate_vdbs: Legacy combined clone + VDB/index node (superseded by cve_clone_and_deps + cve_segmentation). Fields: agent_name, embedder_name, base_git_dir, base_vdb_dir, base_code_index_dir, base_pickle_dir, base_rpm_dir, ignore_errors, ignore_code_embedding, ignore_code_index.
cve_check_vuln_deps: Cross-references SBOM packages against known vulnerabilities for IMAGE analysis. Fields: skip (default false), retry_on_client_errors (default true).
cve_file_output: Writes results to a local JSON file and per-CVE markdown reports. Fields: file_path, markdown_dir, overwrite (default false). Use as cve_output_config_name instead of cve_http_output for file-based output.

3. LLMs (`llms`)

Named LLM configurations referenced by llm_name in pipeline nodes.

Configured models: checklist_llm, code_vdb_retriever_llm, doc_vdb_retriever_llm, cve_agent_executor_llm, generate_cvss_llm, summarize_llm, justify_llm, intel_source_score_llm.

Each entry supports:

_type: LLM API type — nim or openai. Local config hard-codes openai; cloud config uses per-model ${LLM_TYPE_*:-nim} env vars.
api_key: API key (local config uses "EMPTY" with NVIDIA API; cloud config uses ${LLM_API_KEY_*} env vars).
base_url: API base URL. Local config uses ${NVIDIA_API_BASE:-...} for all models; cloud config uses per-model base URL env vars (for example CHECKLIST_LLM_API_BASE, AGENT_EXECUTOR_LLM_API_BASE).
model_name: Model identifier, overridable via env vars (for example ${CHECKLIST_MODEL_NAME:-meta/llama-3.1-70b-instruct}).
temperature, max_tokens, top_p: Standard generation parameters.

See NimModelConfig and OpenAIModelConfig for the full parameter set.

4. Embedding models (`embedders`)

nim_embedder (_type: nim):
- base_url: Embedding API URL (default ${NIM_EMBED_BASE_URL:-https://integrate.api.nvidia.com/v1}).
- model_name: Embedding model (default ${EMBEDDER_MODEL_NAME:-nvidia/nv-embedqa-e5-v5}).
- truncate: How to handle over-length inputs — START, END, or NONE (default END).
- max_batch_size: Batch size for embedding requests (default 128).

See NimEmbedderModelConfig for details.

5. Workflow (`workflow`)

Ties pipeline nodes together. _type is cve_agent, registering the workflow in register.py.

Each *_name field references a function key from the functions section:

Field	Function	Required
`cve_clone_and_deps_name`	`cve_clone_and_deps`	Yes
`cve_segmentation_name`	`cve_segmentation`	Yes
`cve_fetch_intel_name`	`cve_fetch_intel`	Yes
`cve_calculate_intel_score_name`	`cve_calculate_intel_score`	Yes
`cve_process_sbom_name`	`cve_process_sbom`	Yes
`cve_verify_vuln_package_name`	`cve_verify_vuln_package`	Optional (local config only)
`cve_checklist_name`	`cve_checklist`	Yes
`cve_agent_executor_name`	`cve_agent_executor`	Yes
`cve_generate_cvss_name`	`cve_generate_cvss`	Yes
`cve_generate_vex_name`	`cve_generate_vex`	Yes
`cve_summarize_name`	`cve_summarize`	Yes
`cve_justify_name`	`cve_justify`	Yes
`cve_output_config_name`	`cve_http_output` (or `cve_file_output`)	Optional (`null` prints to console)
`cve_source_acquisition_name`	`cve_source_acquisition`	Optional (RPM CVE Checker)
`cve_checker_segmentation_name`	`cve_checker_segmentation`	Optional (RPM CVE Checker)
`cve_package_code_agent_name`	`cve_package_code_agent`	Optional (RPM CVE Checker)
`cve_checker_report_name`	`cve_checker_report`	Optional (RPM CVE Checker)
`cve_build_agent_name`	`cve_build_agent`	Optional (RPM CVE Checker)
`cve_fetch_patches_name`	`cve_fetch_patches`	Optional (local config only)

6. Evaluations and profiling (`eval`)

Configures the NeMo Agent toolkit profiler used by aiq eval. See Evaluating Workflows and Profiler.

general.output_dir: Directory for profiling output (default ./.tmp/eval/cve_agent).
general.dataset._type: Dataset format (json).
general.dataset.file_path: Path to evaluation dataset (default data/eval_datasets/eval_dataset.json).
profiler.token_uniqueness_forecast: Compute inter-query token uniqueness.
profiler.workflow_runtime_forecast: Estimate expected workflow runtime.
profiler.compute_llm_metrics: Compute inference optimization metrics.
profiler.csv_exclude_io_text: Omit large I/O text from CSV output.
profiler.prompt_caching_prefixes.enable: Identify common prompt prefixes.
profiler.prompt_caching_prefixes.min_frequency: Minimum prefix frequency threshold (default 0.1).
profiler.bottleneck_analysis.enable_nested_stack: Enable nested-stack bottleneck analysis.
profiler.concurrency_spike_analysis.enable: Enable concurrency spike detection.
profiler.concurrency_spike_analysis.spike_threshold: Spike threshold (default 7).

NGINX caching server

The docker compose file includes an nginx-cache proxy server container that enables caching for API requests made by the workflow. It is highly recommend to route API requests through the proxy server to reduce API calls for duplicate requests and improve workflow speed. This is especially useful when running the workflow multiple times with the same configuration (for example, for debugging) and can help keep costs down when using paid APIs.

The NGINX proxy server is started by default when running the vuln-analysis service. However, it can be started separately using the following command:

cd ${REPO_ROOT}
docker compose up --detach nginx-cache

To use the proxy server for API calls in the workflow, you can set environment variables for each base URL used by the workflow to point to http://localhost:${NGINX_HOST_HTTP_PORT}/. These are set automatically when running the vuln-analysis service, but can be set manually in the .env file as follows:

CWE_DETAILS_BASE_URL="http://localhost:8080/cwe-details"
DEPSDEV_BASE_URL="http://localhost:8080/depsdev"
FIRST_BASE_URL="http://localhost:8080/first"
GHSA_BASE_URL="http://localhost:8080/ghsa"
NGC_API_BASE="http://localhost:8080/nemo/v1"
NIM_EMBED_BASE_URL="http://localhost:8080/nim_embed/v1"
NVD_BASE_URL="http://localhost:8080/nvd"
NVIDIA_API_BASE="http://localhost:8080/nim_llm/v1"
OPENAI_API_BASE="http://localhost:8080/openai/v1"
OPENAI_BASE_URL="http://localhost:8080/openai/v1"
RHSA_BASE_URL="http://localhost:8080/rhsa"
SERPAPI_BASE_URL="http://localhost:8080/serpapi"
UBUNTU_BASE_URL="http://localhost:8080/ubuntu"

Customizing the Workflow

The primary method for customizing the workflow is to generate a new configuration file with new options. The configuration file defines the workflow settings, such as the functions, the LLM models used, and the output format. The configuration file is a YAML file that can be modified to suit your needs.

Customizing the SBOM input

To use an SBOM from a URL (example), update the SBOM info configuration to:

        "sbom_info": {
          "_type": "http",
          "url": "https://raw.githubusercontent.com/NVIDIA-AI-Blueprints/vulnerability-analysis/refs/heads/main/src/vuln_analysis/data/sboms/nvcr.io/nvidia/morpheus/morpheus%3Av23.11.01-runtime.sbom"
        }

Customizing the LLM models

The workflow configuration file also allows customizing the LLM model and parameters for each component of the workflow, as well as which LLM API is used when invoking the model.

In any configuration file, locate the llms section to see the current settings. For example, the following snippet defines the LLM used for the checklist model:

    checklist_llm:
        _type: nim
        base_url: ${NVIDIA_API_BASE:-https://integrate.api.nvidia.com/v1}
        model_name: ${CHECKLIST_MODEL_NAME:-meta/llama-3.1-70b-instruct}
        temperature: 0.0
        max_tokens: 2000
        top_p: 0.01

_type: specifies the LLM API type. Refer to the Supported LLM APIs table for available options.
base_url: Base URL for LLM. Sets to value of NVIDIA_API_BASE environment variable if set. Otherwise, sets to default NIM base URL. base_url can also be omitted in which case, the default base URL is used for type nim. We use an environment variable here so that we can easily set it to proxy server URL when running with docker compose.
model_name: specifies the model name within the LLM API. Sets to value of CHECKLIST_MODEL_NAME environment variable if set. Otherwise, sets to default checklist model, meta/llama-3.1-70b-instruct. This is also applicable to the other LLM models in the workflow, each having their own environment variable for setting the model name. Refer to the LLM API documentation to determine the available models.
temperature, max_tokens, top_p, ...: specifies the model parameters. The available parameters can be found in the NeMo Agent Toolkit NIMModelConfig. Any non-supported parameters provided in the configuration will be ignored.

Supported LLM APIs

Name	`_type`	Auth Env Var(s)	Base URL Env Var(s)	Proxy Server Route
NVIDIA Inference Microservices (NIMs) (Default)	`nim`	`NVIDIA_API_KEY`	`NVIDIA_API_BASE`	`/nim_llm/v1`
OpenAI	`openai`	`OPENAI_API_KEY`	`OPENAI_API_BASE` (used by `langchain`) `OPENAI_BASE_URL` (used by `openai`)	`/openai/v1`

Steps to configure an LLM model

Obtain an API key and any other required auth info for the selected service.
Update the .env file with the auth and base URL environment variables for the service as indicated in the Supported LLM APIs table. If you choose not to use the default LLM models in your workflow (meta/llama-3.1-70b-instruct), you can also add environment variables to override the model names to your .env file. In addition to CHECKLIST_MODEL_NAME, you can also set model_name for the other LLM models using CODE_VDB_RETRIEVER_MODEL_NAME, DOC_VDB_RETRIEVER_MODEL_NAME, CVE_AGENT_EXECUTOR_MODEL_NAME, GENERATE_CVSS_MODEL_NAME, SUMMARIZE_MODEL_NAME, and JUSTIFY_MODEL_NAME.
Update the config file as described above. For example, if you want to use OpenAI's gpt-4o model for checklist generation, update checklist_llm in the llms section to:

    checklist_llm:
        _type: openai
        model_name: ${CHECKLIST_MODEL_NAME:-gpt-4o}
        temperature: 0.0
        seed: 0
        top_p: 0.01
        max_retries: 5

Please note that the prompts have been tuned to work best with the Llama 3.1 70B NIM and that when using other LLM models it may be necessary to adjust the prompting.

Customizing the embedding model

Vector databases are used by the agent to fetch relevant information for impact analysis investigations. The embedding model used to vectorize your documents can significantly affect the agent's performance. The default embedding model used by the workflow is the NIM nvidia/nv-embedqa-e5-v5 model, but you can experiment with different embedding models of your choice.

To test a custom embedding model, modify the workflow configuration file (for example, config.yml) in the embedders section. For example, the following snippet defines the settings for the default embedding model. The full set of available parameters for a NIM embedder can be found here.

    nim_embedder:
        _type: nim
        base_url: ${NVIDIA_API_BASE:-https://integrate.api.nvidia.com/v1}
        model_name: ${EMBEDDER_MODEL_NAME:-nvidia/nv-embedqa-e5-v5}
        truncate: END
        max_batch_size: 128

_type: specifies the LLM API type. Refer to the Supported LLM APIs table for available options.
base_url: Base URL for embedding model. Sets to value of NVIDIA_API_BASE environment variable if set. Otherwise, sets to default NIM base URL. base_url can also be omitted in which case, the default base
model_name: specifies the model name for the embedding provider. Sets to value of EMBEDDER_MODEL_NAME environment variable if set. Otherwise, sets to default embedding model, nvidia/nv-embedqa-e5-v5. Refer to the embedding provider's documentation to determine the available models.
truncate: specifies how inputs longer than the maximum token length of the model are handled. Passing START discards the start of the input. END discards the end of the input. In both cases, input is discarded until the remaining input is exactly the maximum input token length for the model. If NONE is selected, when the input exceeds the maximum input token length an error will be returned.
max_batch_size: specifies the batch size to use when generating embeddings. We recommend setting this to 128 (default) or lower when using the cloud-hosted embedding NIM. When using a local NIM, this value can be tuned based on throughput/memory performance on your hardware.

Steps to configure an alternate embedding provider

If using OpenAI embeddings, first obtain an API key, then update the .env file with the auth and base URL environment variables for the service as indicated in the Supported LLM APIs table. Otherwise, proceed to step 2. If you choose not to use the default embedding model (nvidia/nv-embedqa-e5-v5), you can also add EMBEDDER_MODEL_NAME to your .env file to override the default.
Update the embedders section of the config file as described above.

Example OpenAI embedding configuration:
```
    nim_embedder:
        _type: openai
        model_name: ${EMBEDDER_MODEL_NAME:-text-embedding-3-small}
        max_retries: 5
```
For OpenAI models, only a subset of parameters are supported. The full set of available parameters can be found in the config definitions here. Any non-supported parameters provided in the configuration will be ignored.

The current workflow uses FAISS to create the vector databases. Interested users can customize the source code to use other vector databases such as cuVS.

Customizing the Output

Currently, there are 3 types of outputs supported by the workflow:

File output: The output data is written to a file in JSON format.
HTTP output: The output data is posted to an HTTP endpoint.
Print output: The output data is printed to the console.

To customize the output, modify the workflow configuration file accordingly. Locate the workflow section to see the output destination used by the workflow. For example, in the configuration file configs/config.yml, the following snippet from the functions section defines the function that writes the workflow output as a single json file and individual markdown files per CVE-ID:

    cve_file_output:
      _type: cve_file_output
      file_path: .tmp/output.json
      markdown_dir: .tmp/vulnerability_markdown_reports
      overwrite: True

The following snippet from the workflow section then configures the workflow to use the above function for output:

    workflow:
      _type: cve_agent
      ...
      cve_output_config_name: cve_file_output

To post the output to an HTTP endpoint, you can add the following to the functions section of the config file, replacing the domain, port, and endpoint with the desired destination (note the trailing slash in the "url" field). The output will be sent as JSON data.

    cve_http_output:
      _type: cve_http_output
      url: http://<domain>:<port>/
      endpoint: "<endpoint>"

The workflow can then be updated to use the new function:

    workflow:
      _type: cve_agent
      ...
      cve_output_config_name: cve_http_output

Additional output options will be added in the future.

RPM CVE Checker

Determine if an RPM package is vulnerable to a specific CVE by examining source patches, changelogs, and build artifacts.

When to use:

Investigating a specific CVE in a single RHEL or Fedora RPM package
Need source-level evidence (patches, changelogs) for vulnerability status
Targeted package analysis vs. container-wide scanning

Quick example:

{
  "pipeline_mode": "package_checker",
  "target_package": {
    "name": "libarchive",
    "version": "3.1.2",
    "release": "14.el7_9.2",
    "arch": "x86_64",
    "cve_id": "CVE-2026-5121"
  }
}

For detailed usage, input/output formats, and verdict explanations, see RPM CVE Checker Documentation.

Limitations

The following limitations apply to this workflow:

Python 2 Support: Python 2 scripts and source code are not supported for analysis. The workflow only processes Python 3 code. Python 2 code will be automatically skipped during code repository processing. This limitation exists because Python 2 reached end-of-life in January 2020 and modern AST parsing libraries used by the workflow only support Python 3.

Troubleshooting

Several common issues can arise when running the workflow. Here are some common issues and their solutions.

Git LFS issues

If you encounter issues with Git LFS, ensure that you have Git LFS installed and that it is enabled for the repository. You can check if Git LFS is enabled by running the following command:

git lfs install

Verifying that all files are being tracked by Git LFS can be done by running the following command:

git lfs ls-files

Files which are missing will show a - next to their name. To ensure all LFS files have been pulled correctly, you can run the following command:

git lfs fetch --all
git lfs checkout *

Container build issues

When building containers for self-hosted NIMs, certain issues may occur. Below are common troubleshooting steps to help resolve them.

Device error

If you encounter an error resembling the following during the container build process for self-hosted NIMs:

nvidia-container-cli: device error: {n}: unknown device: unknown

This error typically indicates that the container is attempting to access GPUs that are either unavailable or non-existent on the host. To resolve this, verify the GPU count specified in the docker-compose.nim.yml configuration file:

Navigate to the deploy.resources.reservations.devices section and check the count parameter.
Set the environment variable NIM_LLM_GPU_COUNT to the actual number of GPUs available on the host machine before building the container. Note that the default value is set to 4.

This adjustment ensures the container accurately matches the available GPU resources, preventing access errors during deployment.

Deploy.Resources.Reservations.devices error

If you encounter an error resembling the following during the container build process for self-hosted NIMs process:

1 error(s) decoding:

* error decoding 'Deploy.Resources.Reservations.devices[0]': invalid string value for 'count' (the only value allowed is 'all')

This is likely caused by an outdated Docker Compose version. Please upgrade Docker Compose to at least v2.21.0.

NGINX caching server

Because the workflow makes such heavy use of the caching server to speed up API requests, it is important to ensure that the server is running correctly. If you encounter issues with the caching server, you can reset the cache.

Resetting the entire cache

To reset the entire cache, you can run the following command:

docker compose down -v

This will delete all the volumes associated with the containers, including the cache.

Resetting just the LLM cache or the services cache

If you want to reset just the LLM cache or the services cache, you can run the following commands:

docker compose down

# To remove the LLM cache
docker volume rm ${COMPOSE_PROJECT_NAME:-vuln_analysis}_llm-cache

# To remove the services cache
docker volume rm ${COMPOSE_PROJECT_NAME:-vuln_analysis}_service-cache

Vector databases

We've integrated VDB and embedding creation directly into the workflow with caching included for expediency. However, in a production environment, it's better to use a separately managed VDB service.

NVIDIA offers optimized models and tools like NIMs (build.nvidia.com/explore/retrieval) and cuVS (github.com/rapidsai/cuvs).

Service errors

National Vulnerability Database (NVD)

These typically resolve on their own. Please wait and try running the workflow again later. Example errors:

404

Error requesting [1/10]: (Retry 0.1 sec) https://services.nvd.nist.gov/rest/json/cves/2.0: 404, message='', url=URL('https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2023-6709')

503

Error requesting [1/10]: (Retry 0.1 sec) https://services.nvd.nist.gov/rest/json/cves/2.0: 503, message='Service Unavailable', url=URL('https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2023-50447')

NVIDIA API Catalog / NVIDIA-hosted NIMs

429 Errors - Too Many Requests

429 errors can occur when your requests exceed the rate limit for the model. Try setting the cve_agent_executor.max_concurrency in the config.yml to a low value such as 5 to reduce the rate of requests.

Exception: [429] Too Many Requests

Authentication errors

Authentication errors will occur if required API key(s) are invalid or have not been set as environment variables as described in Set up the environment file. For example, the following error will occur if NVIDIA_API_KEY is not properly set:

Error: [401] Unauthorized
Authentication failed

Note that exporting the required environment variables in a container shell will not persist outside of that shell. Instead, we recommend shutting down the containers (docker compose down), setting the required environment variables, and then starting the containers again.

Running out of credits

If you run out of credits for the NVIDIA API Catalog, you will need to obtain more credits to continue using the API. Please contact your NVIDIA representative to get more credits added.

Testing and validation

Test-driven development is essential for building reliable LLM-based agentic systems, especially when deploying or scaling them in production environments.

In our development process, we use the Morpheus public container as a case study. We perform security scans and collaborate with developers and security analysts to assess the exploitability of identified CVEs. Each CVE is labeled as either vulnerable or not vulnerable. For non-vulnerable CVEs, we provide a justification based on one of the ten VEX statuses. Team members document their investigative steps and findings to validate and compare results at different stages of the system.

We have collected labels for 38 CVEs, which serve several purposes:

Human-generated checklists, findings, and summaries are used as ground truth during various stages of prompt engineering to refine LLM output.
The justification status for each CVE is used as a label to measure end-to-end workflow accuracy. Every time there is a change to the system, such as adding a new agent tool, modifying a prompt, or introducing an engineering optimization, we run the labeled dataset through the updated workflow to detect performance regressions.

As a next step, we plan to integrate this process into our CI/CD pipeline to automate testing. While LLMs' non-deterministic nature makes it difficult to assert exact results for each test case, we can adopt a statistical approach, where we run the workflow multiple times and ensure that the average accuracy stays within an acceptable range.

We recommend that teams looking to test or optimize their CVE analysis system curate a similar dataset for testing and validation. Note that in test-driven development, it's important that the model has not achieved perfect accuracy on the test set, as this may indicate overfitting or that the set lacks sufficient complexity to expose areas for improvement. The test set should be representative of the problem space, covering both scenarios where the model performs well and where further refinement is needed. Investing in a robust dataset ensures long-term reliability and drives continued performance improvements.

Cite

Please consider citing our paper when using this code in a project. You can use the citation BibTeX:

@inproceedings{zemicheal2024llm,
  title={LLM agents for vulnerability identification and verification of CVEs},
  author={ZeMicheal, Tadesse and Chen, Hsin and Davis, Shawn and Allen, Rachel and Demoret, Michael and Song, Ashley},
  booktitle={Proceedings of the Conference on Applied Machine Learning in Information Security (CAMLIS 2024)},
  pages={161--173},
  year={2024},
  publisher={CEUR Workshop Proceedings},
  volume={3920},
  url={https://ceur-ws.org/Vol-3920/}
}

License

By using this software or microservice, you are agreeing to the terms and conditions of the license and acceptable use policy.

Terms of Use

GOVERNING TERMS: The NIM container is governed by the NVIDIA Software License Agreement and Product-Specific Terms for AI Products; and use of this model is governed by the NVIDIA AI Foundation Models Community License Agreement.

ADDITIONAL Terms: Meta Llama 3.1 Community License, Built with Meta Llama 3.1.

Name		Name	Last commit message	Last commit date
Latest commit History 478 Commits
.tekton		.tekton
.tests-automation @ ae5e980		.tests-automation @ ae5e980
.tmp		.tmp
ci		ci
deploy		deploy
exploit-iq-models @ b60baa9		exploit-iq-models @ b60baa9
images		images
kustomize		kustomize
nginx		nginx
quick_start		quick_start
scripts		scripts
src		src
tests		tests
.dockerignore		.dockerignore
.gitattributes		.gitattributes
.gitignore		.gitignore
.gitmodules		.gitmodules
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
RPM-CVE-CHECKER.md		RPM-CVE-CHECKER.md
SECURITY.md		SECURITY.md
configs		configs
data		data
docker-compose.mlops.yml		docker-compose.mlops.yml
docker-compose.nim.yml		docker-compose.nim.yml
docker-compose.yml		docker-compose.yml
otel-config.yaml		otel-config.yaml
pyproject.toml		pyproject.toml
uv.lock		uv.lock
vulnerability-analysis.code-workspace		vulnerability-analysis.code-workspace

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

NVIDIA AI Blueprint: Vulnerability Analysis for Container Security

Table of Contents

Overview

Software components

Target audience

Prerequisites

Hardware requirements

API definition

Use case description

How it works

Key components

NIM microservices

Getting started

Install system requirements

Obtain API keys

Set up the workflow repository

Set up the environment file

Authenticate Docker with NGC

Build the Docker containers

Start the Docker containers

Using NVIDIA-hosted NIMs

Using self-hosted NIMs

Running the workflow

From the quick start user guide notebook

From the command line

Workflow configuration file

Example command: aiq run

Reviewing the output

Example command: aiq serve

Example command: aiq eval

Reviewing the profiling results

Observing traces in Phoenix

Command line interface (CLI) reference

Configuration file reference

Local vs. cloud-native differences

1. General configuration (general)

2. Functions (functions)

Preprocessing and early pipeline nodes

Core analysis nodes

Output and health

Agent tools

Alternate / optional functions (not in reference configs)

3. LLMs (llms)

4. Embedding models (embedders)

5. Workflow (workflow)

6. Evaluations and profiling (eval)

NGINX caching server

Customizing the Workflow

Customizing the SBOM input

Customizing the LLM models

Supported LLM APIs

Steps to configure an LLM model

Customizing the embedding model

Steps to configure an alternate embedding provider

Customizing the Output

RPM CVE Checker

Limitations

Troubleshooting

Git LFS issues

Container build issues

Device error

Deploy.Resources.Reservations.devices error

NGINX caching server

Resetting the entire cache

Resetting just the LLM cache or the services cache

Vector databases

Service errors

National Vulnerability Database (NVD)

NVIDIA API Catalog / NVIDIA-hosted NIMs

429 Errors - Too Many Requests

Authentication errors

Running out of credits

Testing and validation

Cite

License

Example command: `aiq run`

Example command: `aiq serve`

Example command: `aiq eval`

1. General configuration (`general`)

2. Functions (`functions`)

3. LLMs (`llms`)

4. Embedding models (`embedders`)

5. Workflow (`workflow`)

6. Evaluations and profiling (`eval`)

Packages