feat(AWML): introduce new deployment and onnx/tensorRT evaluation pipeline by vividf · Pull Request #115 · tier4/AWML

vividf · 2025-10-08T15:13:51Z

AWML Unified Deployment Framework

Introduced a unified deployment and evaluation framework for AWML, significantly improving consistency, scalability, and maintainability across projects

PR feat(deployment):prototype deployment framework #137: Core deployment framework (ONNX Runtime + TensorRT support)
PR feat(deployment): centerpoint deployment integration #181: CenterPoint integration into the unified pipeline

Key Contributions

1. Unified Deployment Pipeline

Standardized model deployment flow: PyTorch → ONNX → TensorRT
Eliminated duplicated and project-specific deployment scripts
Enabled config-driven, reusable pipelines across multiple models
Supported seamless backend switching (PyTorch / ONNX / TensorRT)

2. Cross-Backend Verification

Implemented automatic output comparison across backends
Ensured numerical consistency between PyTorch, ONNX Runtime, and TensorRT
Prevented silent accuracy regressions during deployment
Improved reliability for production-level inference

3. Integrated Evaluation Framework

Built unified evaluation pipeline based on T4MetricV2
Supports:
- mAP / mAPH
- Per-class metrics
- Threshold-based evaluation
- Enables consistent benchmarking across different backends and models

How to run

python -m deployment.cli.main centerpoint deployment/projects/centerpoint/config/deploy_config.py   projects/CenterPoint/configs/t4dataset/Centerpoint/second_secfpn_8xb16_121m_j6gen2_base_amp_t4metric_v2.py   --rot-y-axis-reference

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

Signed-off-by: vividf <yihsiang.fang@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

Signed-off-by: vividf <yihsiang.fang@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

KSeangTan

Only review the first part, please address the comments accordingly first.

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

…se.py Signed-off-by: vividf <yihsiang.fang@tier4.jp>

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

KSeangTan

LGTM

Copilot

Pull request overview

This PR introduces a unified, project-agnostic deployment framework under deployment/ that standardizes export (PyTorch→ONNX→TensorRT), verification (cross-backend output comparison), and evaluation (metrics + latency), and migrates CenterPoint deployment to this new pipeline.

Changes:

Added a new deployment/ package with CLI, typed config/schema, exporters/pipelines, runtime orchestrators, primitives, evaluation/verification utilities, and shared documentation.
Integrated CenterPoint into the unified framework via a deployment/projects/centerpoint bundle (entrypoint, runner, export decomposition, inference pipelines, evaluator/executor, config).
Removed legacy CenterPoint deploy script/runner and added CPU-only unit tests for key framework behaviors (artifact resolution, export orchestration, ONNX atomic write path, comparator, schema validation).

Reviewed changes

Copilot reviewed 87 out of 92 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
projects/CenterPoint/scripts/deploy.py	Removed legacy CenterPoint deploy entry script (superseded by unified CLI).
projects/CenterPoint/runners/deployment_runner.py	Removed legacy CenterPoint deployment runner (superseded by unified runner/orchestrators).
projects/CenterPoint/README.md	Updated CenterPoint docs to use unified deployment CLI and Docker image.
projects/CenterPoint/models/detectors/centerpoint_onnx.py	Removed legacy ONNX-specific detector implementation from training project tree.
projects/CenterPoint/models/init.py	Dropped exports of legacy ONNX variants now living under `deployment/projects`.
projects/CenterPoint/Dockerfile	Added CenterPoint deployment Docker image with ORT/TensorRT dependencies.
projects/CenterPoint/configs/t4dataset/Centerpoint/second_secfpn_8xb16_121m_j6gen2_base_amp.py	Adjusted `info_directory_path` default.
deployment/tests/test_output_comparator.py	Added unit tests for output comparator pass/fail and mismatch paths.
deployment/tests/test_onnx_exporter.py	Added tests for atomic ONNX export publish logic (incl. external-data sidecars).
deployment/tests/test_export_pipelines.py	Added tests for ONNX/TensorRT export pipeline control flow and validation.
deployment/tests/test_export_orchestrator.py	Added tests for stale-ONNX guard and artifact resolution delegation.
deployment/tests/test_config_schema.py	Added tests for deploy-config schema parsing/validation.
deployment/tests/test_artifacts.py	Added tests for artifact path resolution and ArtifactManager priority order.
deployment/tests/init.py	Added tests package marker.
deployment/runtime/verification_orchestrator.py	Added scenario-based verification orchestration.
deployment/runtime/runner.py	Added unified base deployment runner wiring export/verify/eval stacks.
deployment/runtime/evaluation_orchestrator.py	Added cross-backend evaluation orchestration with device resolution.
deployment/runtime/artifact_manager.py	Added artifact registration + resolution logic across stages.
deployment/runtime/init.py	Added runtime package marker.
deployment/README.md	Added top-level deployment framework README and navigation.
deployment/projects/registry.py	Added project adapter registry for project-agnostic unified CLI dispatch.
deployment/projects/centerpoint/runner.py	Added CenterPoint runner wiring project-specific model load + export pipeline.
deployment/projects/centerpoint/README.md	Added CenterPoint deployment bundle README.
deployment/projects/centerpoint/io/sample_types.py	Added CenterPoint typed sample payloads and export feature bundle types.
deployment/projects/centerpoint/io/model_loader.py	Added CenterPoint ONNX-compatible model config rewriting + checkpoint loading.
deployment/projects/centerpoint/io/data_loader.py	Added MMDet3D-backed CenterPoint deployment dataloader.
deployment/projects/centerpoint/io/init.py	Added IO package marker.
deployment/projects/centerpoint/inference/pytorch_inference_pipeline.py	Added staged PyTorch inference pipeline matching ONNX/TRT component outputs.
deployment/projects/centerpoint/inference/onnx_inference_pipeline.py	Added ORT-based componentized inference pipeline.
deployment/projects/centerpoint/export/sample_extractor.py	Added CenterPoint export sample extractor using model feature extraction.
deployment/projects/centerpoint/export/onnx_models/centerpoint_onnx.py	Added deploy-only CenterPoint ONNX-friendly detector variant.
deployment/projects/centerpoint/export/onnx_models/centerpoint_head_onnx.py	Added deploy-only ONNX head variants with deterministic output ordering.
deployment/projects/centerpoint/export/onnx_models/init.py	Added ONNX models package marker.
deployment/projects/centerpoint/export/component_builder.py	Added CenterPoint component builder (voxel encoder + backbone/neck/head).
deployment/projects/centerpoint/export/init.py	Added export package marker.
deployment/projects/centerpoint/evaluation/executor.py	Added CenterPoint backend executor (pipeline creation + input preparation).
deployment/projects/centerpoint/evaluation/evaluator.py	Added CenterPoint evaluator using shared 3D detection metrics interface.
deployment/projects/centerpoint/evaluation/init.py	Added evaluation package marker.
deployment/projects/centerpoint/entrypoint.py	Added project entrypoint invoked by unified CLI to wire config/runner.
deployment/projects/centerpoint/contexts.py	Added CenterPoint export context for `--rot-y-axis-reference`.
deployment/projects/centerpoint/config/deploy_config.py	Added reference deploy config (export/eval/verify sections + components).
deployment/projects/centerpoint/cli.py	Added CenterPoint-specific CLI flag registration.
deployment/projects/centerpoint/init.py	Registered CenterPoint project adapter into the registry.
deployment/projects/init.py	Added projects package marker.
deployment/primitives/device.py	Added `DeviceSpec` runtime device descriptor and provider helpers.
deployment/primitives/artifacts.py	Added artifact dataclass + safe path resolution helper.
deployment/primitives/init.py	Added primitives package marker.
deployment/metrics/detection_base.py	Added shared base for flattening/summarizing detection metrics.
deployment/metrics/detection_2d_metrics.py	Added 2D detection metrics interface implementation.
deployment/metrics/init.py	Added metrics package marker.
deployment/io/base_data_loader.py	Added base dataloader interface and default shape-sample helper.
deployment/io/init.py	Added IO package marker.
deployment/inference/gpu_resource_mixin.py	Added GPU resource cleanup utilities and TensorRT resource manager.
deployment/inference/base_inference_pipeline.py	Added base inference pipeline contract with latency breakdown.
deployment/inference/init.py	Added inference package marker.
deployment/export/pipelines/tensorrt_export_pipeline.py	Added model-agnostic TensorRT export pipeline (per-component engines).
deployment/export/pipelines/sample_extractor.py	Added sample extractor seam + default whole-model implementation.
deployment/export/pipelines/onnx_export_pipeline.py	Added model-agnostic ONNX export pipeline with injected decomposition seams.
deployment/export/pipelines/component_builder.py	Added component builder seam + default whole-model implementation.
deployment/export/pipelines/init.py	Added export pipelines package marker and documentation.
deployment/export/exporters/onnx_exporter.py	Added ONNX exporter with staging-dir atomic publish behavior.
deployment/export/exporters/model_wrappers.py	Added base/identity model wrappers for ONNX export.
deployment/export/exporters/configs.py	Added typed ONNX/TensorRT exporter config dataclasses.
deployment/export/contexts.py	Added base `ExportContext` type for per-project extensions.
deployment/export/init.py	Added export package marker.
deployment/evaluation/output_comparator.py	Added structured output comparator used by verification.
deployment/evaluation/evaluator_types.py	Added shared evaluator/verifier datatypes (ModelSpec, latency stats, etc.).
deployment/evaluation/base_evaluator.py	Added base evaluator loop for metrics + latency across backends.
deployment/evaluation/backend_executor.py	Added backend executor contract shared by evaluation + verification.
deployment/evaluation/init.py	Added evaluation package marker.
deployment/docs/runbook.md	Added operational runbook for the unified CLI and workflow.
deployment/docs/operations.md	Added ops guidance and troubleshooting.
deployment/docs/contributing.md	Added contribution guide for adding new project bundles.
deployment/docs/architecture.md	Added architecture/contract documentation for framework layers and seams.
deployment/config/enums.py	Added config enums (Backend, ExportMode, PrecisionPolicy).
deployment/config/base.py	Added `BaseDeploymentConfig` with runtime validation and helpers.
deployment/config/init.py	Added config package marker.
deployment/cli/main.py	Added unified CLI that discovers/loads project bundles and dispatches.
deployment/cli/args.py	Added shared CLI args and file logging helpers.
deployment/cli/init.py	Added CLI package marker.
deployment/init.py	Added deployment package root and version.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    @staticmethod
+    def _parse_deploy_log_path(raw: Optional[str]) -> Optional[str]:
+        """Parse deploy_log_path; None or blank disables file logging."""
+        return raw.strip() or None if raw is not None else None


+    def to_ort_provider(self) -> list[str]:
+        """Return ONNX Runtime execution providers for this device."""
+        if self.is_cuda:
+            return ["CUDAExecutionProvider", "CPUExecutionProvider"]
+        return ["CPUExecutionProvider"]


+        to modify them manually.
+        """
+        # Heatmap
+        heatmap_tensors = head_tensors["heatmap"][:, torch.tensor([0, 1, 2, 3, 4], dtype=torch.int), :, :]


+        heatmap_scale_factors = heatmap_scale_factors.view([1, -1, 1, 1])
+        scale_heatmap_tensors = torch.mul(heatmap_tensors, heatmap_scale_factors)
+        # Reg
+        reg_tensors = head_tensors["reg"][:, torch.tensor([0, 1], dtype=torch.int), :, :]


+        scale_reg_tensors = torch.mul(reg_tensors, reg_scale_factors)
+
+        # Height
+        height_tensors = head_tensors["height"][:, torch.tensor([0], dtype=torch.int), :, :]


+
+        # Dim
+        # Swap length, width, height to width, length, height
+        flip_dim_tensors = head_tensors["dim"][:, torch.tensor([1, 0, 2], dtype=torch.int), :, :]


+
+        # Rot
+        # Swap sin(y), cos(x) to cos(x), sin(y)
+        flip_rot_tensors = head_tensors["rot"][:, torch.tensor([1, 0], dtype=torch.int), :, :]


+        scale_flip_rot_tensors = torch.mul(flip_rot_tensors, rot_scale_factors)
+
+        # Vel
+        vel_tensors = head_tensors["vel"][:, torch.tensor([0, 1], dtype=torch.int), :, :]


vividf changed the title ~~feat(AWML): introduce new deployment and onnx/tensorRT evaluation pipeline~~ WIP: feat(AWML): introduce new deployment and onnx/tensorRT evaluation pipeline Nov 14, 2025

vividf force-pushed the feat/new_deployment_and_evaluation_pipeline branch from 68d1700 to 7ac5b8b Compare January 29, 2026 08:08

vividf and others added 10 commits March 11, 2026 13:24

chore: init

c77cf92

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: add deployment framework intro

117dead

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

feat(deployment):prototype deployment framework (#137)

41a3e9a

Signed-off-by: vividf <yihsiang.fang@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

feat(deployment): restructure deployment framework (#160)

d80db0a

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: add missing pipeline directory and docstring

a1edc32

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

feat(deployment): refactor config and clean code (#180)

a362b27

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: refactor base config

d99296c

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean up code: device spec, remove unused fucntion .etc

e23c210

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: refactor export compenent

dd46052

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix more Device spec

2b28f60

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

vividf force-pushed the feat/new_deployment_and_evaluation_pipeline branch from 5256306 to 2b28f60 Compare March 11, 2026 04:27

vividf and others added 14 commits March 25, 2026 20:49

chore: enable only trt evaluation

85f0d64

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix verification

e7439a6

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix main

ca6f730

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix base evaluator

d150f5e

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean up base

23df89b

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: add base logger

36d7568

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: update docs

5bd5b76

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: add more readme

0eed1f1

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: base add -> None for __init__

d706ff7

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: remove task profile

9dfb6a1

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix matrix report base

a75d739

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean metrics

efa71d1

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: update project registry base

0c7c57c

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

feat(deployment): centerpoint deployment integration (#181)

ddc5106

Signed-off-by: vividf <yihsiang.fang@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

vividf self-assigned this Apr 14, 2026

vividf added 2 commits April 16, 2026 16:16

chore: improve docs

d0b491d

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean up docs

5080b40

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chor: clean code

1d4c27f

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

vividf force-pushed the feat/new_deployment_and_evaluation_pipeline branch from 8756669 to 1d4c27f Compare April 20, 2026 13:05

KSeangTan requested changes Apr 21, 2026

View reviewed changes

vividf and others added 22 commits May 19, 2026 20:35

chore:pathlib

cf9157a

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix sequence

7256b29

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix message to dict-like mapping

e242301

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix normalize, validate cuda device, parse deploy log path, ba…

f2e5a94

…se.py Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean code

b70171e

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean up private member in configs base

736a31b

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean up more code

159eab3

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: fix record time, clean funcs

e85d3cf

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: update readme and clean more code

ccb1482

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean more code

63e4e0e

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: clean more code

3cf2438

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: seperate responsiblility of verifier and evaluator

2f38e92

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: refactor export pipeline

72ca1da

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: rm

3c10cf3

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: rename class

04b09cc

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: remove registry

13243a5

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: refactor metrics

38d5234

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: mv backend in enum

53ed723

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: redesign directory naming

a3cb78f

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

chore: rename dir

e64dda5

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

Merge branch 'main' into feat/new_deployment_and_evaluation_pipeline

8001c89

chore: fix readme

07218b5

Signed-off-by: vividf <yihsiang.fang@tier4.jp>

vividf requested a review from KSeangTan July 1, 2026 05:58

KSeangTan requested a review from Copilot July 1, 2026 08:09

Copilot started reviewing on behalf of KSeangTan July 1, 2026 08:09 View session

KSeangTan approved these changes Jul 1, 2026

View reviewed changes

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(AWML): introduce new deployment and onnx/tensorRT evaluation pipeline#115

feat(AWML): introduce new deployment and onnx/tensorRT evaluation pipeline#115
vividf wants to merge 53 commits into
mainfrom
feat/new_deployment_and_evaluation_pipeline

vividf commented Oct 8, 2025 •

edited

Loading

Uh oh!

KSeangTan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KSeangTan left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

vividf commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AWML Unified Deployment Framework

Key Contributions

1. Unified Deployment Pipeline

2. Cross-Backend Verification

3. Integrated Evaluation Framework

How to run

Uh oh!

KSeangTan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KSeangTan left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vividf commented Oct 8, 2025 •

edited

Loading