Skip to content
scott edited this page Jan 2, 2025 · 32 revisions

Whoopnet

Whoopnet enables tiny FPV drones to operate autonomously by leveraging offboard neural networks. It facilitates the processing of raw visual and inertial data while providing the interface to deliver RC flight commands for classical and ai driven control.

While compatible with any unmodified FPV multirotor that supports ELRS, HDZero and Betaflight, the project focuses on targeting sub-25-gram vehicles and advancing capabilities via offboard compute.

This allows a 20g robot to have a 20kg brain.

Read the docs.


Goals:

  • Stable State Estimation with VINS (Visual-Inertial Navigation)
  • Reproduce Deep Drone Acrobatics with a tiny whoop and off-board compute.

whoopnet_hw whoopnet_hw
whoopnet_hw
io

Challenges:

  • Un-syncronized Camera and IMU feeds
  • Low Frequency IMU (~100hz)
  • Rolling Shutter Camera
  • ELRS Telemetry Bandwidth
  • Noise
  • Murphy

Strategy

  • Synchronize IMU and Camera via FC timestamp over OSD and telemetry
  • Betaflight Modifications:
    1. Disable existing telemetry. Send raw IMU CRSF telemetry packets @ ~100hz (with FC timestamp)
    2. Increase OSD refresh rate to 30hz and send FC timestamp (receive and extract via OCR)
  • ELRS v3 F1000HZ to support the telemetry bandwidth requirements
  • HDZero Video System (low and fixed latency)

Hardware

FPV Multirotors

  • Any multirotor with ELRS, Betaflight, HDZero and a VRX that has HDMI OUT
  • 65mm class (sub-25-gram) multirotors:
    • Mobula6 HDZero ECO 2024
    • Mobula6 Freestyle HD
  • 3.5" class (sub-250-gram) multirotors:
    • Crux35 HDZero

ELRS Transmitter(s)

  • Used for remote communication and control with "high-speed" telemetry.
  • Example: BETAFPV 2.4GHZ 1W Micro RF Module (Supports CRSF over USB)

VRX with HDMI Output

  • A video receiver capable of streaming real-time FPV video feed through an HDMI interface.
  • HDZero VRX (HDZero Monitor, VRX Module or Goggles) /w HDMI Output**

Video Capture Device

  • USB HDMI Capture device that supports 1280x720 @ 30HZ

RC Handset

  • For Manual Flight Control and Expert Pilot Training
  • Boxer Radio Transmitter
  • SquidStick ELRS RX To Gamepad (for wireless experience) OR use Boxer Radio USB Gamepad Mode

Clone this wiki locally