MASLAB 2026 - Team 5: Autonomous Can-Sorting Robot

MIT Mobile Autonomous Systems Lab (6.141/16.405) - January 2026

Fully autonomous robot that detects, collects, and deposits colored cans into matching zones using computer vision, depth sensing, and closed-loop motor control.

---

Table of Contents

• Overview
• The Competition
• System Architecture
• Hardware
• Vision Pipeline
• Training Dataset
• Autonomous Strategies
• Project Structure
• Setup & Installation
• Usage
• Configuration
• Archive

---

Overview

This repository contains the complete software stack for Team 5's MASLAB 2026 competition robot. The system combines a custom-trained YOLOv8 object detection model with Intel RealSense depth sensing to identify and locate colored cans and zones in real-time. A closed-loop control system using IMU heading correction and encoder-balanced motor control enables precise autonomous navigation, can pickup via a servo-driven claw, and deposit into corresponding zones.

Key Capabilities

• 7-class YOLO detection: Red/green/yellow cans, red/green/yellow zones, and opponent robot
• Depth-aware targeting: RealSense D435i provides per-pixel depth for accurate distance measurement
• Visual servoing: Continuous approach with real-time steering correction based on object tracking
• Blue boundary safety: HSV-based detection of arena boundary tape prevents crossing into opponent territory
• Blind-finish approach: Encoder-based final approach when target enters camera blind spot (<35cm)
• IMU heading hold: BNO08X 9-DOF IMU maintains straight-line driving and precise turns

---

The Competition

MASLAB is MIT's annual autonomous robotics competition. Robots operate in a divided arena with:

• Colored cans (red, green, yellow) scattered across the field
• Colored zones (matching colored mats on the floor) where cans must be deposited
• Blue boundary tape dividing the arena between two competing robots
• 3-minute match time limit for fully autonomous operation

Arena Elements

Element	Description	Detection Method
Red/Green/Yellow Cans	Small colored cylinders to collect	YOLOv8 (trained model)
Red/Green/Yellow Zones	Floor mats for can deposit	YOLOv8 (trained model)
Blue Boundary	Tape dividing arena halves	HSV color detection
Opponent Robot	Other team's robot to avoid	YOLOv8 (trained model)

Sample arena view -- cans near a green zone with blue boundary tape visible:

Object Examples

Colored Cans (targets to collect):

Red Can	Green Can	Yellow Can

Deposit Zones (matching colored floor mats):

Green Zone	Red Zone

Blue Boundary Tape (arena divider, detected via HSV):

Multi-object scene (green cans near blue tape and zone):

---

System Architecture

                    ┌─────────────────────────────────────────────┐
                    │             Strategy Layer                  │
                    │  comp.py / tournament.py / strategies.py    │
                    │  (state machines, search, approach, deposit)│
                    └──────────┬──────────────┬───────────────────┘
                               │              │
                    ┌──────────▼──────┐  ┌────▼──────────────────┐
                    │  Vision Layer   │  │   Control Layer       │
                    │                 │  │                       │
                    │  yolo.py        │  │  motor.py             │
                    │  (7-class det.) │  │  (drive, turn, lift)  │
                    │                 │  │                       │
                    │  hsv.py         │  │  claw.py              │
                    │  (blue boundary)│  │  (servo gripper)      │
                    │                 │  │                       │
                    │  depth.py       │  │                       │
                    │  (RealSense)    │  │                       │
                    │                 │  │                       │
                    │  utils.py       │  │                       │
                    │  (scoring, etc.)│  │                       │
                    └────────┬────────┘  └───────┬───────────────┘
                             │                   │
                    ┌────────▼───────────────────▼───────────────┐
                    │            Hardware Layer                   │
                    │                                            │
                    │  RealSense D435i    Raven Motor Controller │
                    │  BNO08X IMU         Adafruit Stepper HAT  │
                    │  Servo (CH4)        Encoders (3200 CPR)   │
                    └────────────────────────────────────────────┘

Module Dependency Graph

config.py ← (loaded by all modules)
    │
    ├── yolo.py      ← ultralytics YOLO model
    ├── depth.py     ← pyrealsense2
    ├── hsv.py       ← OpenCV HSV processing
    ├── motor.py     ← raven, adafruit, IMU
    ├── claw.py      ← raven servo control
    └── utils.py     ← scoring, approach logic
          │
          └── strategies.py  ← orchestrates all modules
                │
                ├── comp.py        (competition: SimpleAutonomous)
                ├── tournament.py  (tournament: exploration + game)
                └── main.py        (first can sequence)

---

Hardware

Components

Component	Model	Interface	Purpose
Depth Camera	Intel RealSense D435i	USB 3.0	RGB + depth at 640x480 @ 30fps
IMU	BNO08X (9-DOF)	I2C (800kHz)	Quaternion-based heading for turns
Motor Controller	MIT Raven	I2C	Dual drive motors + encoder reading
Drive Motors	DC w/ 50:1 gearbox	Raven CH1, CH2	3200 CPR encoder, 3.875" wheels
Lift Motor	NEMA 17 Stepper	Adafruit HAT (I2C)	T8 lead screw, 0.8cm/rev lift
Claw Servo	Standard servo	Raven CH4	-27deg (open) to 80deg (closed)
Power	12V via Raven CH5	Direct	Powers Adafruit HAT for stepper

Motor Channel Mapping

Raven CH1 → Right drive motor
Raven CH2 → Left drive motor
Raven CH4 → Claw servo
Raven CH5 → 12V power supply → Adafruit MotorKit HAT → Stepper M1/M2

Physical Dimensions

• Wheel diameter: 3.875 inches
• Camera height: 51 cm from ground
• Camera tilt: 58.5 degrees from horizontal
• Camera-to-front offset: 1.3 inches
• Claw reach: 8 inches from robot front
• Lift range: 5.3 cm (down) to 12.3 cm (up) = 7.0 cm travel

---

Vision Pipeline

YOLO Object Detection (yolo.py)

Custom-trained YOLOv8s model (best.pt) detecting 7 classes:

Class ID	Class Name	Type
0	red_zone	Deposit zone
1	green_zone	Deposit zone
2	yellow_zone	Deposit zone
3	robot	Opponent
4	red_can	Collection target
5	green_can	Collection target
6	yellow_can	Collection target

Each detection includes bounding box, confidence score, and depth (via RealSense).

Depth Processing (depth.py)

The RealSenseCamera class provides:

• Aligned RGB + depth frames
• Per-pixel depth queries
• Robust median depth within bounding boxes (noise-resistant)
• Valid depth range: 0.2m - 3.0m

Distance Calculation (utils.py)

Raw depth is corrected for camera tilt angle:

horizontal_distance = depth * cos(58.5deg) = depth * 0.5225

This converts angled depth readings to true ground-plane distance.

Blue Boundary Detection (hsv.py)

HSV-based detection of blue arena tape:

• Morphological cleanup (open + close) for noise removal
• Elongation filtering (tape is thin and long)
• Nearest-point queries for safety distance checks
• Prevents robot from crossing into opponent's half

---

Training Dataset

The YOLO model was trained on a custom dataset of 238 annotated images collected from the robot's onboard RealSense camera.

Dataset Statistics

Split	Images
Training	215
Validation	8
Test	15
Total	238

Local Training Images

The data/images/ directory contains categorized training samples:

Category	Count	Description
scenes/	185	Full arena scenes with multiple objects
cans/red/	12	Red can close-ups at various distances
cans/green/	18	Green can close-ups at various distances
cans/gold/	6	Yellow/gold can close-ups
zones/red/	2+	Red deposit zone floor mats
zones/green/	3+	Green deposit zone floor mats
zones/gold/	varies	Yellow/gold deposit zone floor mats
blue_line/	6	Blue boundary tape samples

Full Dataset

The complete annotated dataset (238 images, 10.7 MB) is hosted on Ultralytics HUB:

View Dataset on Ultralytics HUB

Training Pipeline

See docs/ML_CAN_DETECTION_GUIDE.md for the full training workflow:

1. Collect images on-robot using tests/capture_images.py
2. Annotate with bounding boxes (Ultralytics HUB or Roboflow)
3. Train YOLOv8s model (100 epochs, Google Colab recommended)
4. Deploy best.pt weights to robot's src/ directory

---

Autonomous Strategies

SimpleAutonomous (comp.py)

The primary competition strategy. Repeating cycle:

SEARCH → APPROACH → GRAB → BACKUP → FIND ZONE → DEPOSIT → REPEAT

• Supports color priority (yellow/green/red first for point optimization)
• Visual servoing approach with blind-finish fallback
• Blue boundary safety checks throughout
• Handles motor/servo voltage recovery delays

Tournament Exploration (tournament.py)

Two-phase strategy for tournament matches:

Phase 1 - Exploration:

Origin → Drive 30" → Scan → Drive 30" → Scan → Drive 30" → Scan → Return

Maps all visible cans and zones in world coordinates.

Phase 2 - Game: Prioritized collection based on mapped positions.

Strategy Pattern (strategies.py)

Object-oriented strategy framework with composable phases:

• FindFirstCanStrategy - Search and grab the first can found
• FindMatchingCansStrategy - Find additional cans of the same color
• NavigateToZoneStrategy - Navigate to matching zone and deposit
• TwoCanStrategy / ThreeCanStrategy - Multi-can collection sequences
• FullCycleStrategy - Complete autonomous cycle (find, grab, deposit)
• BlueBoundaryGuard - Continuous blue tape monitoring

---

Project Structure

maslab_2026/
├── README.md                  # This file
├── .gitignore
├── requirements.txt           # Python dependencies
│
├── src/                       # Core source code
│   ├── config.py              # Central configuration (loads from log.txt)
│   ├── log.txt                # Tunable parameters (edit this!)
│   ├── best.pt                # Trained YOLOv8 model weights (18.4MB)
│   ├── yolo.py                # YOLO 7-class object detection
│   ├── depth.py               # RealSense D435i camera wrapper
│   ├── hsv.py                 # HSV blue boundary detection
│   ├── motor.py               # Motor control (drive, turn, lift)
│   ├── claw.py                # Servo claw control
│   ├── claw_height.py         # Claw height calibration utility
│   ├── utils.py               # Shared utilities (distance, scoring, approach)
│   ├── strategies.py          # Strategy pattern classes
│   ├── comp.py                # Competition: SimpleAutonomous strategy
│   ├── tournament.py          # Tournament: exploration + game strategy
│   ├── main.py                # First can sequence (approach → grab → zone)
│   ├── auto.py                # Automation framework
│   └── run_live.py            # Live detection viewer with keyboard controls
│
├── tests/                     # Test & calibration scripts
│   ├── camera_test.py         # Camera initialization & HSV testing
│   ├── claw_test.py           # Claw servo test
│   ├── motor_test.py          # Motor control test
│   ├── elevator_test.py       # Elevator + claw interactive test
│   ├── drift_test.py          # Drift calibration
│   ├── final_test.py          # Full integration test
│   └── capture_images.py      # Capture training images from camera
│
├── data/                      # Training data
│   └── images/
│       ├── annotations.ndjson # YOLO bounding box annotations
│       ├── verify_annotations.py
│       ├── blue_line/         # Blue boundary tape samples (6 images)
│       ├── cans/              # Can images by color
│       │   ├── red/           #   Red cans (12 images)
│       │   ├── green/         #   Green cans (18 images)
│       │   └── gold/          #   Yellow/gold cans (6 images)
│       ├── zones/             # Zone images by color
│       │   ├── red/
│       │   ├── green/
│       │   └── gold/
│       └── scenes/            # Full arena scenes (185 images)
│
├── docs/
│   └── ML_CAN_DETECTION_GUIDE.md  # ML training pipeline guide
│
└── archive/                   # Previous implementations (reference only)
    ├── xander/                # Xander's original HSV-based system
    ├── tournament_variants/   # Tournament strategy iterations
    │   ├── tournament1.py
    │   ├── tournament2.py
    │   ├── tournament2a.py
    │   └── tournamenta.py
    ├── test_variants/         # Test script iterations
    │   ├── run_tests.py - run_tests5.py
    │   ├── comp_test_2.py
    │   ├── final_test0.py
    │   └── F_test1.py
    └── root_scripts/          # Early standalone test scripts
        ├── camera_testing.py
        ├── get_motor.py
        ├── getmotor2.py
        ├── mt.py
        ├── rasp_yolo.py
        └── vibecoding.py

---

Setup & Installation

Prerequisites

• Raspberry Pi 4 (or compatible SBC) running Raspberry Pi OS
• Intel RealSense D435i connected via USB 3.0
• MIT Raven motor controller on I2C bus
• Adafruit Motor HAT on I2C bus (for stepper lift)
• BNO08X IMU on I2C bus
• Python 3.9+

Install Dependencies

# Clone the repository
git clone https://github.com/jeffelin/maslab_2026.git
cd maslab_2026

# Install Python dependencies
pip install -r requirements.txt

# Install Raven (MIT MASLAB package - provided separately)
# Follow MASLAB staff instructions for raven installation

Verify Hardware

# Check I2C devices are detected
i2cdetect -y 1

# Test camera
cd src
python3 run_live.py

# Test motors
cd ../tests
python3 motor_test.py

---

Usage

All main scripts run from the src/ directory:

cd src

Live Detection Viewer

python3 run_live.py

Interactive viewer with keyboard controls:

• q / ESC - Quit
• SPACE - Pause/resume
• s - Save screenshot
• d - Toggle depth overlay
• m - Move to target (motor)
• g - Grab (close claw)
• o - Open claw
• Click - Inspect pixel (RGB, HSV, depth)

Competition Mode

python3 comp.py

Controls:

• SPACE - Start simple mode (any can)
• Y - Start with yellow priority
• G - Start with green priority
• R - Start with red priority
• C - Claw calibration
• ESC - Abort

Tournament Mode

python3 tournament.py

Controls:

• E - Run exploration phase
• M - Show learned map data
• T - Test detection
• ESC - Abort

Strategy Framework

python3 strategies.py

Runs the FullCycleStrategy: find 3 cans, navigate to zone, deposit.

---

Configuration

All tunable parameters are in src/log.txt. Edit this file to adjust robot behavior without changing code.

Key Parameters

Parameter	Default	Description
MOVE_MAX_SPEED	80	Maximum drive speed (0-100)
TURN_MAX_SPEED	40	Maximum turn speed
MIN_CONFIDENCE	0.50	YOLO detection threshold
BLIND_SPOT_DEPTH_CM	35.0	Camera blind spot distance
CLAW_OPEN_POSITION	-27	Servo degrees when open
CLAW_CLOSE_POSITION	90	Servo degrees when closed
BLUE_SAFETY_DISTANCE_M	0.3	Min distance from blue tape
LOST_CAN_BLIND_DRIVE_INCHES	7.48	Blind drive distance
FAST_START_INCHES	30	Initial forward burst
MAX_CANS_TO_COLLECT	3	Cans before depositing

See src/config.py for the full list of ~60 tunable constants.

---

Archive

The archive/ directory preserves previous development iterations for reference:

• xander/ - Original HSV-based detection system with Flask detection server, movement primitives, IMU calibration tools, and diagnostic scripts
• tournament_variants/ - 4 iterations of the tournament strategy tested during competition prep
• test_variants/ - 9 test script iterations from the development process
• root_scripts/ - Early standalone motor/camera test scripts

---

License

MIT MASLAB 2026 Competition - Team 5

Built at MIT Lincoln Laboratory, January 2026.