Authors: Ibrahim Jubran, Fares Fares, Yuval Alfassi, Firas Ayoub and Dan Feldman from the Robotics & Big Data Labs.
The Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model, by aligning pairs of 2D captured image points to their corresponding 3D points in the model. We suggest an algorithm that runs on weak IoT devices in real-time but still provides provable theoretical guarantees for both running time and correctness. Existing solvers provide only one of these requirements. Our main motivation was to turn the popular DJI's Tello Drone (<90gr, <$100) into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor, by simply attaching a Raspberry PI Zero (<9gr, <$25) to it. This tiny micro-processor takes as input a real-time video from a tiny RGB camera, and runs our PnP solver on-board. Extensive experimental results, open source code, and a demonstration video are included.
This work was accepted to The 2022 IEEE/RSJ International Conference on Intelligent Robotics and Systems (IROS'22) PDF.
If you use Newton-PnP in an academic work, please cite:
@article{jubran2022newton,
title={Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones},
author={Jubran, Ibrahim and Fares, Fares and Alfassi, Yuval and Ayoub, Firas and Feldman, Dan},
journal={arXiv preprint arXiv:2203.02686},
year={2022}
}
We have tested the library in Ubuntu 20.04, and Raspberry Pi OS but it should be easy to compile in other platforms.
Tested with C++17.
We use OpenCV to manipulate images and features. Dowload and install instructions can be found at: http://opencv.org. Tested with OpenCV 3.4.16.
Download and install instructions can be found at: http://eigen.tuxfamily.org. tested with Eigen 3.3.0.
Install Boost free peer-reviewed portable C++ source libraries: https://www.boost.org/.
sudo apt-get install libboost-all-dev
Clone the repository:
git clone https://github.com/rbdlabhaifa/Newton-PnP.git
Please make sure you have installed all required dependencies (see section 2). Execute:
mkdir build
cd build
cmake ..
make
cd build
./NPnP