From ecd498c34afb8de90cb667585c6145aaa65018a2 Mon Sep 17 00:00:00 2001 From: Physic69 <162324988+Physic69@users.noreply.github.com> Date: Wed, 22 Jul 2026 10:45:19 +0530 Subject: [PATCH] Add CAD_Extruder --- projects/vintage_CAD_Extractor.md | 101 ++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 projects/vintage_CAD_Extractor.md diff --git a/projects/vintage_CAD_Extractor.md b/projects/vintage_CAD_Extractor.md new file mode 100644 index 0000000..cda46e2 --- /dev/null +++ b/projects/vintage_CAD_Extractor.md @@ -0,0 +1,101 @@ +# Interactive AI CAD Extractor — Stage 3 Launch (Final Demo) + +* **Participant:** Physic69 +* **Stage completed:** 3 +* **Stage 3 Repository:** https://github.com/Physic69/RDK-Challenge +* **Stage 1 & 2 History:** https://github.com/Physic69/RDK-X5-Challenge +* **Demo video:** + +> All images below are hosted on the participant's own repository and are embedded by URL — no image assets are added to the challenge repository. + +--- + +## Summary + +**Interactive AI CAD Extractor** is a real-time, multi-task edge computing pipeline built on the **D-Robotics RDK X5**. It transforms physical objects into 3D digital assets by simultaneously running monocular depth estimation and open-vocabulary object detection. + +Using a BPU-accelerated **YOLO-World** model, it detects specific target prompts (such as a mouse, bottle, or keyboard) via an offline JSON vocabulary. Concurrently, a BPU-accelerated **MiDaS Small** model generates a dense depth map. A Human-in-the-Loop (HITL) CPU node intercepts this ROS 2 data stream, halts the terminal to prompt the user, and uses a custom "cookie-cutter" algorithm to extrude the target's depth ROI into a perfectly formatted `.obj` CAD file directly to local storage. + +For Stage 3, the system is a complete prototype featuring dynamic open-vocabulary filtering, dual-model BPU scheduling, and a resilient ROS 2 data flow that safely handles multi-object NMS arrays without crashing. + +--- + +## Challenge 1 — Prototype Integration + +A single coherent system integrating on-board AI, ROS 2, digital actuation, and sensor fusion. + +| Requirement | How it is met | +| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| **AI model(s) on board** | YOLO-World (Open-Vocabulary) + MiDaS Small (Depth), both BPU-accelerated on the RDK X5 | +| **ROS 2 communication** | Nodes communicate over standard `sensor_msgs/Image` and serialized JSON `std_msgs/String` topics | +| **Digital actuation / limits** | Extracts spatial data to local disk as a `.obj` file; safely drops vocabulary targets that lack offline embeddings to prevent inference crashes | +| **Sensor fusion** | RGB vision fused with AI-inferred spatial depth mappings to create 3D vertices | +| **Quick-start README** | [README.md](https://github.com/Physic69/RDK-Challenge/blob/main/README.md) with clone, build, and launch instructions | +| **Documented launch file** | Launch pipeline consists of two distinct nodes: `vision_node` (Perception) and `cad_writer_node` (Interactive Extrusion) | + +--- + +## Challenge 2 — Real-Time AI Inference + +* **BPU acceleration:** The primary perception model, **YOLO-World** (`yolo_world.bin`), runs on the RDK X5 BPU via the modern `hbm_runtime`. The depth estimation model, **MiDaS Small** (`midas_small_256_compiled.bin`), runs concurrently on the BPU via the legacy `pyeasy_dnn` interface. +* **Two concurrent workloads:** (1) Continuous monocular depth mapping and dynamic bounding-box generation running on the BPU; (2) an interactive CPU-bound generation loop that awaits keyboard input to calculate Z-axis thickness and plot 3D vertices into a `.obj` file. + +### Interactive Workflow + +The terminal dynamically prompts the operator based on live BPU detections: + +```text +======================================== +TARGETS DETECTED IN WORKSPACE: +======================================== +[0] BOTTLE (Confidence: 0.87) +[1] MOUSE (Confidence: 0.62) +[2] CANCEL / RE-SCAN + +Select target ID to extract CAD [0-2]: 0 +[INFO] Extracting bottle... +[INFO] Calculating spatial vertices... +[SUCCESS] Saved 3D asset with 2450 vertices to /root/bottle_extracted.obj +``` + +Full benchmark data, including resolution, FPS/latency, model names, and tool versions, is available in `docs/architecture.md`. + +--- + +## Challenge 3 — Final Demo & Packaging + +| Deliverable | Link | +| -------------------------- | ----------------------------------------------------------------------------------------------------- | +| Demo video (YouTube) | TO ADD | +| GitHub repository (public) | https://github.com/Physic69/RDK-Challenge | +| Technical documentation | docs/architecture.md | +| Benchmark evidence | Terminal logs confirm dual `[BPU_PLAT] soc info(x5)` scheduling across `hbm_runtime` and `pyeasy_dnn` | + +### Pipeline Startup Order + +| Order | Node | Purpose | +| ----- | ------------------------------ | ---------------------------------------------------------------------------- | +| 1 | `rdk_vision_pkg vision_node` | BPU depth-map generation and open-vocabulary YOLO-World bounding boxes | +| 2 | `cad_extruder cad_writer_node` | Interactive HITL terminal, mask isolation, and `.obj` point-cloud generation | + +--- + +## Technical Highlights + +* **Models:** YOLO-World (BPU, Open-Vocabulary) + MiDaS Small 256×256 (BPU). +* **Perception → Generation:** RGB feed → dual inference → ROS 2 JSON serialization → grayscale ROI extraction → Z-axis scalar mapping → `.obj` file write. +* **Safety & Recovery:** Explicit handling of NumPy NMS array ambiguities (`len(keep) == 0`) and automatic pre-flight vocabulary checks to strip unavailable text prompts before they reach BPU memory allocation. + +--- + +## Links & Evidence + +* **Repository & README:** https://github.com/Physic69/RDK-Challenge +* **Demo video:** +* **Technical docs & benchmarks:** https://github.com/Physic69/RDK-Challenge/blob/main/docs/architecture.md + +--- + +## Declaration + +I agree that this showcase document may be used by the Robotics Dream Keeper Challenge organizers as described in the official README, including for promotion, judging, and archival purposes.