patrolingDrone

Repository: rbdlabhaifa/patrolingDrone

What is this project?

patrolingDrone is an RBD Lab, University of Haifa stack for 24/7 autonomous DJI Tello patrol: ORB-SLAM2 localization, waypoint navigation through saved checkpoints, automatic return-to-charger via ArUco visual landing, and Raspberry Pi Bluetooth power cycling so the drone can recharge unattended and resume loops.

It combines a forked ORB-SLAM2 core, a Drone/Charger C++ layer (ctello UDP), and the boxCharger/ Python ArUco landing scripts (same family as Smart-Charger).

What does it do?

Phase	Executable	Purpose
Map + mark waypoints	tello_mapping	Manual flight, SLAM map, Save Destination in viewer
Autonomous patrol	tello_navigation	Loop checkpoints, SLAM pose, charge when low
24/7 supervisor	tello_main	Restarts navigation after each lap
Pi power control	Drone/BluetoothServer	GPIO on/off Tello via Bluetooth
ArUco docking	boxCharger/	Precision land on charging pad

Typical workflow:

git clone https://github.com/rbdlabhaifa/patrolingDrone.git
cd patrolingDrone && ./build.sh
# 1) tello_mapping → save morning/evening maps + destinations
# 2) Pi runs tello_main → endless patrol + charge

See demos/README.md for step-by-step demos.

---

1. Project Overview

Problem solved: Tello battery (~13 min) limits continuous lab experiments. This system patrols a mapped route, returns to a smart charger, powers off/on through a Pi, and repeats without human intervention.

Primary users:

Audience	Use
RBD Lab drone team	Long-running indoor patrol experiments
Mapping session operator	tello_mapping + Save Destination checkpoints
Field deploy	Pi on drone runs tello_main

What it is not: Outdoor GPS navigation, multi-drone fleet manager (see multiple_drones branch), or production-ready cross-platform product — many paths are hardcoded to lab machines.

Notable branches: master, simulator, multiple_drones, Support_ubuntu22, make_config_files.

---

2. Architecture & Tech Stack

Layer	Technology
Language	C++17, Python 3
SLAM	ORB-SLAM2 fork (ORB_SLAM2 namespace)
Drone I/O	ctello (UDP Tello SDK wrapper)
Vision	OpenCV, Pangolin viewer
Charging	ArUco PID (boxCharger/), Bluetooth GPIO
Build	CMake, build.sh

flowchart TB
  MAIN[tello_main] --> NAV[tello_navigation]
  NAV --> DRONE[Drone class]
  DRONE --> SLAM[ORB-SLAM2 TrackMonocular]
  DRONE --> NAVTHREAD[navigation_thread]
  NAVTHREAD --> CHK[Checkpoints loop]
  CHK -->|low battery| CHG[Charger + boxCharger]
  CHG --> BT[Pi BluetoothServer GPIO]
  SLAM --> MAP[Slam_latest_Map_morning/evening.bin]
  SLAM --> DEST[drone_destinations_*.txt]

Loading

Key components:

Path	Role
Examples/Monocular/tello_mapping.cc	Interactive mapping + Save Destination
Examples/Monocular/tello_navigation.cc	SLAM + patrol + map/day-night switch
Examples/Monocular/tello_main.cc	Lap supervisor (fork + restart)
Drone/src/Drone/Drone.cpp	Navigation thread, pose updates
Drone/src/Charger/Charger.cpp	Low-battery → ArUco land → power off
boxCharger/	Python ArUco landing (KeyboardControl_Charge.py)
Drone/BluetoothServer/	Pi-side power relay

---

3. Prerequisites

Hardware

• DJI Tello + extra batteries
• Raspberry Pi on drone (Bluetooth to flight PC / power control)
• Smart charger with ArUco marker (see boxCharger/, Smart-Charger)
• Linux PC for mapping (Ubuntu 20.04 / 22.04 tested)

Software

• OpenCV 3.x+, Pangolin, Eigen3 (/usr/local/include/eigen3/ in CMake)
• Boost (serialization, system)
• BlueZ dev libs (BluetoothServer on Pi)
• Python deps for charger: pip install -r requirements.txt

Bundled in repo

• config/*.yaml — per-drone camera calibrations (tello_9F5EC2_640.yaml, etc.)
• drone_destinations_morning.txt / _evening.txt — sample checkpoints
• boxCharger/corners_dist_matrix_1.p — ArUco landing filter data
• Vocabulary/ORBvoc.txt.tar.gz
• aliases.txt — shell shortcuts

---

4. Environment Setup

Clone

git clone https://github.com/rbdlabhaifa/patrolingDrone.git
cd patrolingDrone

Configuration template

cp config/patrol_settings.example.yaml config/patrol_settings.yaml
# Use as reference — edit source files (paths not loaded at runtime yet)

Must edit in source:

File	Parameters
Examples/Monocular/tello_navigation.cc	tello_name, rpi_bluetooth_address, destination paths
Examples/Monocular/tello_main.cc	~/patrolingDrone path in system() call
Drone/src/Charger/Charger.cpp	KeyboardControl_Charge.py path
boxCharger/*.py	Windows/lab paths (/home/rbdlab2/...)

Camera YAML: Match your Tello (config/tello_<SSID>.yaml). Update aliases.txt accordingly.

Pi setup

cd Drone/BluetoothServer/build && cmake .. && make
echo '/path/to/BluetoothServer' >> ~/.bashrc

---

5. Build & Run Instructions

Build

chmod +x build.sh
./build.sh

Outputs: build/examples/tello_mapping, tello_navigation, tello_main, lib/libORB_SLAM2.so.

Mapping session (once per environment / lighting)

./build/examples/tello_mapping Vocabulary/ORBvoc.txt config/tello_9F5EC2_640.yaml

Save checkpoints → rename to Slam_latest_Map_morning.bin + drone_destinations_morning.txt (and evening variants).

Deploy 24/7

On Pi (battery power):

./build/examples/tello_main

Tests / CI

No automated CI. Smoke tests: socket_testing, calibrate_cam, manual tello_navigation single lap.

---

6. Repository Structure

patrolingDrone/
├── build.sh
├── CMakeLists.txt
├── aliases.txt
├── config/
│   ├── patrol_settings.example.yaml   # ★ Settings reference
│   └── tello_*.yaml                     # Per-drone calibration
├── Examples/Monocular/
│   ├── tello_mapping.cc                 # ★ Map + checkpoints
│   ├── tello_navigation.cc              # ★ Autonomous patrol
│   └── tello_main.cc                    # ★ 24/7 loop supervisor
├── Drone/
│   ├── src/Drone/Drone.cpp
│   ├── src/Charger/Charger.cpp
│   └── BluetoothServer/                 # ★ Pi power control
├── boxCharger/                          # ArUco landing (Python)
├── ctello/                              # Tello UDP library
├── include/ / src/                      # ORB-SLAM2 core
├── Thirdparty/                          # DBoW2, g2o
├── Vocabulary/ORBvoc.txt.tar.gz
├── drone_destinations_morning.txt       # Sample waypoints
├── demos/README.md
└── align_map_to_xy.py                   # Offline map alignment helper

---

7. Core Workflows & Data Flow

Workflow A — Mapping

tello_mapping → manual Tello flight → ORB-SLAM map
  → localization mode → Save Destination at each checkpoint
  → Slam_latest_Map.bin + drone_destinations.txt
  → rename morning/evening copies

Workflow B — Patrol loop

tello_navigation
  → turn_drone_on_and_connect (Bluetooth)
  → load map by hour (morning vs evening)
  → read_drone_destinations → align_to_xy_axis
  → navigation_thread: checkpoint 1 → 2 → … → N → 1 → …
  → SLAM thread: TrackMonocular → update_pose
  → low battery → Charger → boxCharger ArUco land → power off → charge
  → reconnect → repeat

Workflow C — Supervisor

tello_main → while(true) system(tello_navigation) → count laps

Read first: Drone/src/Drone/Drone.cpp, Examples/Monocular/tello_navigation.cc, Drone/src/Charger/Charger.cpp.

---

8. Deployment & CI/CD

Deployment: Lab room with fixed charger station, Tello Wi‑Fi, Pi on drone for power cycling. Mapping laptop separate from Pi flight brain optional.

CI/CD: None configured.

---

9. Known Quirks & Technical Debt

Issue	Notes
Hardcoded /home/rbdlab2/ paths	C++ and Python — search repo before deploy
Settings not externalized	patrol_settings.example.yaml is documentation only
tello_main path	Assumes ~/patrolingDrone
Morning/evening split	Hour 6–19 vs else in tello_navigation.cc
Checkpoint 1 near charger	Must see ArUco; orientation matters on return leg
Memory leak mitigation	tello_main restarts process each lap
Legacy ORB-SLAM2 API	ORB_SLAM2 namespace, Save Destination in Tracking
boxCharger duplicates Smart-Charger	Keep paths in sync when updating landing code
Eigen path	CMake expects /usr/local/include/eigen3/

---

10. Troubleshooting / FAQ

1. SLAM lost / empty pose

Run relocalization rotation (rotate_to_relocalize), ensure map file matches lighting (morning vs evening).

2. Drone won't power on

Check Pi BluetoothServer running, rpi_bluetooth_address MAC correct.

3. ArUco landing fails

Verify boxCharger/corners_dist_matrix_1.p present; fix Python paths in Charger.cpp; see Smart-Charger demos.

4. Wrong checkpoints

Re-run mapping; confirm drone_destinations_*.txt path in tello_navigation.cc matches repo root.

5. Build fails on Eigen

# Install Eigen3 to /usr/local/include/eigen3/ or edit CMakeLists.txt EIGEN3_INCLUDE_DIR

6. Vocabulary missing

cd Vocabulary && tar xf ORBvoc.txt.tar.gz

---

Related RBD Lab projects

Smart-Charger · LidarDrone · RBD-SLAM · ORB_SLAM3 · AI634Code

License

ORB-SLAM2-derived components follow upstream GPL/BSD licensing. Confirm with RBD Lab before external redistribution.