Feat: Beta namespace - #1919
Conversation
…pace as a CMake option, default OFF
|
Important Review skippedToo many files! This PR contains 166 files, which is 66 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (166)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…uirement for push event
There was a problem hiding this comment.
Pull request overview
This PR introduces an opt-in beta (experimental) namespace in depthai-core, wiring new beta datatypes/nodes/utilities through CMake, core serialization/parsing, and Python bindings, along with initial examples/tests.
Changes:
- Add
DEPTHAI_BUILD_BETAbuild option and propagateDEPTHAI_HAVE_BETAto conditionally compile/export beta APIs. - Introduce beta datatypes/nodes/properties/utilities and integrate them into
DatatypeEnum, stream parsing, and host replay behavior. - Add Catch2 + Python tests and a Python example for beta functionality; includes several formatting-only cleanups.
Reviewed changes
Copilot reviewed 163 out of 166 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/src/onhost_tests/beta_parser_properties_test.cpp | Adds on-host tests for beta parser property serialization/round-tripping. |
| tests/src/ondevice_tests/pipeline/node/image_manip_test.cpp | Formatting change in ImageManip test config setup. |
| tests/CMakeLists.txt | Conditionally adds beta on-host test when DEPTHAI_BUILD_BETA is enabled. |
| src/utility/ProtoSerialize.cpp | Marks additional beta datatypes as not supported for protobuf deserialization. |
| src/utility/ImageManipImpl.cpp | Minor comment formatting tweak. |
| src/pipeline/node/host/Replay.cpp | Extends replay datatype handling for new enum values (unsupported types). |
| src/pipeline/node/Camera.cpp | Style-only formatting change. |
| src/pipeline/datatype/StreamMessageParser.cpp | Adds parsing branches for beta datatypes under DEPTHAI_HAVE_BETA. |
| src/pipeline/datatype/DatatypeEnum.cpp | Extends datatype hierarchy to include new beta datatypes. |
| src/beta/utilities/RFDETR/RFDETRUtils.hpp | Adds RF-DETR decoding utility API (beta). |
| src/beta/utilities/PPText/PPTextUtils.hpp | Adds PP-Text detection decoding utility API (beta). |
| src/beta/utilities/MLSD/MLSDUtils.hpp | Adds MLSD line decoding utility API (beta). |
| src/beta/utilities/MediaPipe/MediaPipeUtils.hpp | Adds MediaPipe palm detection utility API (beta). |
| src/beta/utilities/LaneDetection/LaneDetectionUtils.hpp | Adds lane detection utility API (beta). |
| src/beta/utilities/Detection/MaskUtils.hpp | Adds mask processing helpers (beta). |
| src/beta/utilities/Detection/DetectionUtils.hpp | Adds detection utilities (e.g., NMS) (beta). |
| src/beta/utilities/Classification/ClassificationUtils.hpp | Adds classification tensor helpers (beta). |
| src/beta/properties/ImgDetectionsFilterProperties.cpp | Implements beta properties dtor for ImgDetectionsFilter. |
| src/beta/node/EmbeddingsParser.cpp | Adds EmbeddingsParser node implementation (beta). |
| src/beta/datatype/Predictions.cpp | Adds Predictions datatype implementation (beta). |
| src/beta/datatype/Lines.cpp | Adds Lines datatype implementation (beta). |
| src/beta/datatype/Keypoints.cpp | Adds Keypoints datatype implementation (beta). |
| src/beta/datatype/ImgDetectionsFilterConfig.cpp | Adds ImgDetectionsFilterConfig datatype implementation (beta). |
| src/beta/datatype/Classifications.cpp | Adds Classifications datatype implementation (beta). |
| include/depthai/pipeline/node/StereoDepth.hpp | Wraps warning comment for formatting. |
| include/depthai/pipeline/datatype/DatatypeEnum.hpp | Adds new datatype enum entries for beta datatypes. |
| include/depthai/depthai.hpp | Conditionally exports beta headers when DEPTHAI_HAVE_BETA is set. |
| include/depthai/beta/properties/YuNetParserProperties.hpp | Adds YuNetParser properties (beta). |
| include/depthai/beta/properties/XFeatStereoParserProperties.hpp | Adds XFeatStereoParser properties (beta). |
| include/depthai/beta/properties/XFeatMonoParserProperties.hpp | Adds XFeatMonoParser properties (beta). |
| include/depthai/beta/properties/SuperAnimalParserProperties.hpp | Adds SuperAnimalParser properties (beta). |
| include/depthai/beta/properties/SCRFDParserProperties.hpp | Adds SCRFDParser properties (beta). |
| include/depthai/beta/properties/RFDETRParserProperties.hpp | Adds RFDETRParser properties (beta). |
| include/depthai/beta/properties/RegressionParserProperties.hpp | Adds RegressionParser properties (beta). |
| include/depthai/beta/properties/PPTextDetectionParserProperties.hpp | Adds PPTextDetectionParser properties (beta). |
| include/depthai/beta/properties/MPPalmDetectionParserProperties.hpp | Adds MPPalmDetectionParser properties (beta). |
| include/depthai/beta/properties/MLSDParserProperties.hpp | Adds MLSDParser properties (beta). |
| include/depthai/beta/properties/MapOutputParserProperties.hpp | Adds MapOutputParser properties (beta). |
| include/depthai/beta/properties/LaneDetectionParserProperties.hpp | Adds LaneDetectionParser properties (beta). |
| include/depthai/beta/properties/KeypointParserProperties.hpp | Adds KeypointParser properties (beta). |
| include/depthai/beta/properties/ImgDetectionsFilterProperties.hpp | Adds ImgDetectionsFilter properties (beta). |
| include/depthai/beta/properties/ImageOutputParserProperties.hpp | Adds ImageOutputParser properties (beta). |
| include/depthai/beta/properties/HRNetParserProperties.hpp | Adds HRNetParser properties (beta). |
| include/depthai/beta/properties/FastSAMParserProperties.hpp | Adds FastSAMParser properties (beta). |
| include/depthai/beta/properties/EmbeddingsParserProperties.hpp | Adds EmbeddingsParser properties (beta). |
| include/depthai/beta/properties/ClassificationSequenceParserProperties.hpp | Adds ClassificationSequenceParser properties (beta). |
| include/depthai/beta/properties/ClassificationParserProperties.hpp | Adds ClassificationParser properties (beta). |
| include/depthai/beta/nodes.hpp | Adds umbrella include for beta nodes. |
| include/depthai/beta/node/RegressionParser.hpp | Adds RegressionParser node API (beta). |
| include/depthai/beta/node/MapOutputParser.hpp | Adds MapOutputParser node API (beta). |
| include/depthai/beta/node/ImgDetectionsFilter.hpp | Adds ImgDetectionsFilter node API (beta). |
| include/depthai/beta/node/ImageOutputParser.hpp | Adds ImageOutputParser node API (beta). |
| include/depthai/beta/node/HRNetParser.hpp | Adds HRNetParser node API (beta). |
| include/depthai/beta/node/EmbeddingsParser.hpp | Adds EmbeddingsParser node API (beta). |
| include/depthai/beta/node/ClassificationParser.hpp | Adds ClassificationParser node API (beta). |
| include/depthai/beta/datatypes.hpp | Adds umbrella include for beta datatypes. |
| include/depthai/beta/datatype/Predictions.hpp | Adds Predictions datatype API (beta). |
| include/depthai/beta/datatype/Map2D.hpp | Adds Map2D datatype API (beta). |
| include/depthai/beta/datatype/Lines.hpp | Adds Lines datatype API (beta). |
| include/depthai/beta/datatype/Keypoints.hpp | Adds Keypoints datatype API (beta). |
| include/depthai/beta/datatype/ImgDetectionsFilterConfig.hpp | Adds ImgDetectionsFilterConfig datatype API (beta). |
| include/depthai/beta/datatype/Clusters.hpp | Adds Clusters datatype API (beta). |
| include/depthai/beta/datatype/Classifications.hpp | Adds Classifications datatype API (beta). |
| examples/python/Beta/img_detections_filter.py | Adds beta Python example for ImgDetectionsFilter. |
| examples/cpp/ToF/tof_minimal.cpp | Formatting-only change. |
| examples/cpp/ToF/tof_all_queues.cpp | Formatting-only change. |
| examples/cpp/ToF/tof_align.cpp | Include ordering + formatting-only changes. |
| examples/cpp/IMU/imu_gyroscope_accelerometer.cpp | Whitespace cleanup. |
| CMakeLists.txt | Adds beta sources + compile definition DEPTHAI_HAVE_BETA when enabled. |
| cmake/depthaiOptions.cmake | Introduces DEPTHAI_BUILD_BETA option. |
| cmake/depthaiConfig.cmake.in | Exports DEPTHAI_BUILD_BETA in package config. |
| cmake/Depthai/DepthaiDeviceRVC4Config.cmake | Updates RVC4 version string. |
| bindings/python/tests/CMakeLists.txt | Adds beta Python test when beta build is enabled. |
| bindings/python/tests/beta_img_detections_filter_test.py | Adds Python tests for beta ImgDetectionsFilter availability/passthrough. |
| bindings/python/src/pipeline/PipelineBindings.cpp | Treats depthai.beta.node classes as “from bindings” for create-path selection. |
| bindings/python/src/pipeline/node/NodeBindings.cpp | Adds beta node bind callstack entries and creates depthai.beta.node submodule. |
| bindings/python/src/pipeline/node/Common.hpp | Adds helper to register beta nodes (addBetaNode / ADD_BETA_NODE_DERIVED). |
| bindings/python/src/pipeline/node/CameraBindings.cpp | Formatting-only change. |
| bindings/python/src/pipeline/datatype/TrackletsBindings.cpp | Formatting-only change. |
| bindings/python/src/pipeline/datatype/TrackedFeaturesBindings.cpp | Formatting-only change. |
| bindings/python/src/pipeline/datatype/SpatialLocationCalculatorDataBindings.cpp | Formatting-only change. |
| bindings/python/src/pipeline/datatype/MessageGroupBindings.cpp | Formatting-only change. |
| bindings/python/src/pipeline/datatype/AprilTagsBindings.cpp | Formatting-only change. |
| bindings/python/src/DatatypeBindings.cpp | Adds conditional exposure of beta datatype enum values and binding callstack entries. |
| bindings/python/src/beta/node/YuNetParserBindings.cpp | Adds Python bindings for YuNetParser (beta). |
| bindings/python/src/beta/node/XFeatStereoParserBindings.cpp | Adds Python bindings for XFeatStereoParser (beta). |
| bindings/python/src/beta/node/XFeatMonoParserBindings.cpp | Adds Python bindings for XFeatMonoParser (beta). |
| bindings/python/src/beta/node/SuperAnimalParserBindings.cpp | Adds Python bindings for SuperAnimalParser (beta). |
| bindings/python/src/beta/node/SCRFDParserBindings.cpp | Adds Python bindings for SCRFDParser (beta). |
| bindings/python/src/beta/node/RFDETRParserBindings.cpp | Adds Python bindings for RFDETRParser (beta). |
| bindings/python/src/beta/node/RegressionParserBindings.cpp | Adds Python bindings for RegressionParser (beta). |
| bindings/python/src/beta/node/PPTextDetectionParserBindings.cpp | Adds Python bindings for PPTextDetectionParser (beta). |
| bindings/python/src/beta/node/MPPalmDetectionParserBindings.cpp | Adds Python bindings for MPPalmDetectionParser (beta). |
| bindings/python/src/beta/node/MLSDParserBindings.cpp | Adds Python bindings for MLSDParser (beta). |
| bindings/python/src/beta/node/MapOutputParserBindings.cpp | Adds Python bindings for MapOutputParser (beta). |
| bindings/python/src/beta/node/LaneDetectionParserBindings.cpp | Adds Python bindings for LaneDetectionParser (beta). |
| bindings/python/src/beta/node/KeypointParserBindings.cpp | Adds Python bindings for KeypointParser (beta). |
| bindings/python/src/beta/node/ImgDetectionsFilterBindings.cpp | Adds Python bindings for ImgDetectionsFilter + config/properties (beta). |
| bindings/python/src/beta/node/ImageOutputParserBindings.cpp | Adds Python bindings for ImageOutputParser (beta). |
| bindings/python/src/beta/node/HRNetParserBindings.cpp | Adds Python bindings for HRNetParser (beta). |
| bindings/python/src/beta/node/FastSAMParserBindings.cpp | Adds Python bindings for FastSAMParser (beta). |
| bindings/python/src/beta/node/EmbeddingsParserBindings.cpp | Adds Python bindings for EmbeddingsParser (beta). |
| bindings/python/src/beta/node/ClassificationSequenceParserBindings.cpp | Adds Python bindings for ClassificationSequenceParser (beta). |
| bindings/python/src/beta/node/ClassificationParserBindings.cpp | Adds Python bindings for ClassificationParser (beta). |
| bindings/python/src/beta/datatype/PredictionsBindings.cpp | Adds Python bindings for Predictions datatype (beta). |
| bindings/python/src/beta/datatype/Map2DBindings.cpp | Adds Python bindings for Map2D datatype (beta). |
| bindings/python/src/beta/datatype/LinesBindings.cpp | Adds Python bindings for Lines datatype (beta). |
| bindings/python/src/beta/datatype/KeypointsBindings.cpp | Adds Python bindings for Keypoints datatype (beta). |
| bindings/python/src/beta/datatype/ClustersBindings.cpp | Adds Python bindings for Clusters datatype (beta). |
| bindings/python/src/beta/datatype/ClassificationsBindings.cpp | Adds Python bindings for Classifications datatype (beta). |
| bindings/python/setup.py | Allows enabling beta build via env var to CMake args. |
| bindings/python/CMakeLists.txt | Adds beta binding sources when DEPTHAI_BUILD_BETA is enabled. |
| .github/workflows/python-main.yml | Enables beta build in Python CI workflow and passes CMake flag. |
| .github/workflows/main.workflow.yml | Adds workflow input + env-based logic to build with/without beta; passes CMake flag. |
Suppressed comments (1)
tests/src/onhost_tests/beta_parser_properties_test.cpp:75
- The schema validation loop only asserts properties if the expected nodes are encountered; if a node is missing from schema.nodes, the test will still pass (false negative). Consider tracking which expected node names were seen and REQUIRE at the end that all expected nodes were found.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #include <catch2/catch_all.hpp> | ||
| #include <depthai/depthai.hpp> |
| case DatatypeEnum::ImgDetectionsFilterConfig: | ||
| #ifdef DEPTHAI_HAVE_BETA | ||
| return parseDatatype<beta::ImgDetectionsFilterConfig>(metadataStart, serializedObjectSize, data, fd); | ||
| #else | ||
| break; | ||
| #endif |
| using namespace dai; | ||
| using namespace dai::beta::node; | ||
|
|
||
| auto betaModule = m.def_submodule("beta", "Experimental APIs"); |
There was a problem hiding this comment.
bindings/python/src/beta/node/KeypointParserBindings.cpp doesn't have this (at least not explicit). How come the difference?
moratom
left a comment
There was a problem hiding this comment.
Neat, some experimental nodes :)
A few high level comments:
- I think basic tests would make sense still, even though the API is experimental
- Maybe makes sense to add a notice to README what's the purpuse of the beta submodule (unstable API, ...)
- RVC2 FW still needs to be bumped for safety, to get the enums updated
| cv::Mat colored; | ||
| logDepth.convertTo( | ||
| colored, CV_8U, 255.0 / (logMaxDepth - logMinDepth), -logMinDepth * 255.0 / (logMaxDepth - logMinDepth)); | ||
| logDepth.convertTo(colored, CV_8U, 255.0 / (logMaxDepth - logMinDepth), -logMinDepth * 255.0 / (logMaxDepth - logMinDepth)); |
|
From CI: this doesn't seem like a flaky test, OS 1.27.1 RVC4 |
Purpose
This PR adds a new beta namespace to depthai. The namespace is meant to include features and new nodes that are subject to large API and behavioural changes in any proceeding depthai version without any prior warning. Beta node run only on RVC4 devices, if you have an RVC2 device, you have to run the node on host.
Specification
In addition, the first supported beta nodes are all parsers from depthai-nodes and their respective message types. Their implementations have been compared to their results from depthai-nodes with no difference in output for the same given NN input. It is not expected that the API for this batch of parsers will stay stable.
Dependencies & Potential Impact
Possible issues with RVC2 devices on enumeration of messages. Other than that the beta nodes are isolated
Deployment Plan
Testing & Validation
comparison of outputs with depthai-nodes implementations.
AI Usage
Assisted-by: Claude Fable 5 Max & Codex 5.6 Sol xhigh
Submitted code was reviewed by a human: tested and validated. Since it is beta feature
The author is taking the responsibility for the contribution: YES