Hand Gesture Based FSDS + Arduino Control

This project uses hand tracking from a webcam to control a Formula Student Driverless Simulator (FSDS) vehicle and also send steering-based commands to an Arduino.

What this project does

The Python script:

• detects both hands using MediaPipe Hand Landmarker
• calibrates a neutral steering pose
• converts hand tilt into steering command
• uses hand opening gestures for longitudinal control
• sends vehicle control to FSDS
• sends simple serial commands to Arduino based on steering direction

The Arduino sketch:

• listens for serial commands from Python
• drives the connected motor left, right, or stop
• also prints analog A0 readings to Serial Monitor

Files

• hand_motion.py → main Python control script (IMPORTANT - ENSURE THE ARDUINO COM PORT IS SAME AS IN IDE)
• arduino_code.ino → Arduino motor control sketch (DO NOT TOUCH)
• hand_landmarker.task → MediaPipe model file required by the Python script

#NOTE - SET THE MAXIMUM VEHICLE SPEED IN TARGET SPEED WITHIN THE hand_motion.py FILE

Control logic

Python side

• steering is computed from the angle between both hands
• the system auto-calibrates when both hands are visible
• pressing c performs manual calibration
• Esc exits the program

Gesture actions used by the code

• left hand open → applies brake
• right hand open → activates PID speed hold to target speed
• no valid hands → zero input
• if hands are lost for 5 seconds, calibration resets and the simulator resets

Arduino serial commands

The Python script sends:

• L → one steering direction
• R → opposite steering direction
• C or S → stop / neutral

Your current Arduino sketch maps them like this:

• receives R → calls moveLeft()
• receives L → calls moveRight()
• receives C or S → stops motor

So yes, the naming is reversed between command letter and motor function name. That is not automatically wrong, but it is something to keep in mind when testing hardware.

Requirements

Python

Install these packages in your Python environment:

pip install opencv-python mediapipe pyserial numpy