2D inverted pendulum balanced by 2 reaction wheels. The controller learns via Reinforcement Learning in simulation and runs the trained policy on-device. ROS2 and ZephyrOS running on Arduino UNO Q.
The hypothesis is that a machine-learning trained in simulation with randomized parameters will generalize and adapt itself to the real-world system despite differences in motors and mass distribution, whereas a PID controller would require manual retuning.
| Component | Part |
|---|---|
| MCU board | Arduino UNO Q (STM32U585 @ 160 MHz + QRB2210 Linux SBC) |
| IMU | MPU6050 |
| Motor | NEMA 17 stepper (17PM-K502-G6WM) |
| Driver | A4988 |
| Power | 12–24 V for motor, 3.3 V logic from board |
Steppers are bad for this but this is what I had on hand. Use BLCD with encoders if you can.
┌─────────────────────────────────────────────────────┐
│ Zephyr RTOS (STM32U585) │
│ │
│ thread_sensors ──► k_msgq ──► thread_control │
│ │ │ │
│ MPU6050 (I2C) STEP/DIR GPIO │
│ Complementary filter (A4988 microstepping) │
│ │ │
│ thread_comms ◄──── k_msgq ◄─────────┘ │
│ │ │
│ simple binary UART frames │
└───────────────┬─────────────────────────────────────┘
│ LPUART1 → /dev/ttyHS1
┌───────────────▼──────────────────────────────────────┐
│ QRB2210 Linux SBC (on-board) — ROS 2 Jazzy │
│ serial_bridge_node reads /dev/ttyHS1 │
│ ──► /pendulum/state (local DDS) │
│ webots_relay.py ──► TCP client → 127.0.0.1:9001 │
└───────────────┬──────────────────────────────────────┘
│ TCP over USB (adb reverse tcp:9001)
┌───────────────▼──────────────────────────────────────┐
│ Windows — Webots R2025a + Python │
│ TCP server :9001 ──► Digital Twin │
│ RL Training (Stable Baselines3) ──► policy.h │
└──────────────────────────────────────────────────────┘
Transport. The QRB2210 runs native ROS 2, so
webots_relay.pyruns on the board, subscribes to/pendulum/stateover local DDS, and pushes the twin packet to Webots over a TCP-over-USB tunnel (adb reverse). No network IP, no WSL2. The earlier WiFi + WSL2 relay path is deprecated — see docs/deprecated_wifi_wsl2.md. WSL2 + the FastDDS TCP profiles remain only as an optional RViz2/visualisation path.
ROS2 runs natively on the QRB2210. The STM32 sends lightweight binary frames over UART; a Python node on the QRB2210 parses them and publishes topics. Details: docs/serial_communication.md
| Thread | Priority | Stack | Role |
|---|---|---|---|
thread_sensors |
2 | 2048 B | Read MPU6050, apply complementary filter |
thread_control |
1 | 2048 B | PID / RL policy → compute step frequency |
thread_comms |
3 | 2048 B | Send state as binary UART frames to QRB2210 |
firmware/
└── zephyr/
├── blinky/ # hardware verification sample
└── reaction_wheel/ # main application
├── CMakeLists.txt
├── prj.conf
├── boards/
│ └── arduino_uno_q.conf
└── src/
├── app.h # shared types, message queue externs
├── main.c # queue definitions, main()
├── sensors.c/h # IMU read + filter
├── control.c/h # PID / policy inference
└── comms.c/h # binary UART frames to QRB2210
docs/
├── arduino_uno_q_zephyr.md # board setup, flashing, peripherals
├── qrb2210_ros2_setup.md # ROS2 install + startup procedure
├── serial_communication.md # arduino-router investigation, RTT solution
└── deprecated_wifi_wsl2.md # old WiFi + WSL2 relay architecture (reference)
ros2_ws/
├── serial_bridge_node.py # reads /dev/ttyHS1, publishes /pendulum/state
├── webots_relay.py # (on board) /pendulum/state → TCP-over-USB to Webots
├── wsl2_relay.py # DEPRECATED (WiFi + WSL2 + UDP)
├── fastdds_tcp_server.xml # DEPRECATED (FastDDS TCP, WiFi era)
└── fastdds_tcp_client.xml # DEPRECATED (FastDDS TCP, WiFi era)
scripts/
├── flash.ps1 # Ctrl+Shift+B flash via ADB + GDB
└── rtt_monitor.ps1 # RTT console (printk output over SWD)
simulation/
└── webots/
├── worlds/pendulum.wbt # 2-wheel free-standing pendulum on a table
└── controllers/pendulum_controller/ # SIMULATION (physics) + DIGITAL_TWIN (display)
training/
└── rl/ # Stable Baselines3 PPO training
| Tool | Version | Notes |
|---|---|---|
| nRF Connect for VS Code | v3.3.0 | includes Zephyr SDK, west |
| ADB Platform Tools | latest | C:\tools\platform-tools\ in PATH |
| WSL2 Ubuntu 24.04 | — | ROS2 Jazzy + micro_ros_agent |
| Webots | R2025a | Windows native |
| Python | 3.11 | stable-baselines3, gymnasium |
Flashing uses QRB2210 (Linux SoC on the board) as a debug bridge — no ST-LINK needed.
# Ctrl+Shift+B in VS Code, or manually:
.\scripts\flash.ps1See docs/arduino_uno_q_zephyr.md for full setup details.
Arduino UNO Q routes the STM32 UART through arduino-router, a Linux daemon that speaks MessagePack RPC — not raw serial. PuTTY, PowerShell SerialPort, and Arduino IDE Serial Monitor all show nothing because they expect raw bytes.
The correct solution is Segger RTT over the SWD debug connection (same path used for flashing):
# Runs in VS Code Terminal — "RTT Monitor" task (Ctrl+Shift+B → pick task)
.\scripts\rtt_monitor.ps1The script kills any stale openocd, starts arduino-debug on the board, configures RTT via OpenOCD telnet (port 4444), and streams printk output from port 9090.
Required prj.conf entries:
CONFIG_USE_SEGGER_RTT=y
CONFIG_RTT_CONSOLE=y
CONFIG_UART_CONSOLE=n # must disable UART, otherwise it takes priority and RTT gets 0 bytesSee docs/serial_communication.md for the full investigation.
Open firmware/zephyr/reaction_wheel in nRF Connect for VS Code, add build configuration for arduino_uno_q/stm32u585xx, then Build + Flash.