🏎️ Behavioral Cloning

End-to-end self-driving — a CNN that maps raw camera pixels straight to steering, learned by cloning a human driver.

🧭 Overview

An end-to-end deep-learning solution to the Udacity CarND Behavioral Cloning project: an NVIDIA-style CNN that learns to drive by cloning recorded human driving. It maps a single front-camera frame directly to control signals (steering, throttle, brake, speed) — no hand-engineered lane detection.

📹 record ──▶ 🧠 train ──▶ 🚗 drive ──▶ 🎬 video

1. Record good driving in the Udacity simulator, which writes a driving_log.csv plus left/center/right camera frames.
2. Train the network on those frames (bc-train). Each record is augmented into six samples — the three cameras, each also horizontally flipped — to teach recovery and cancel the track's directional bias.
3. Drive autonomously (bc-drive): the simulator streams telemetry over Socket.IO, the model predicts controls, and they are streamed back. Saved frames can be stitched into a video (bc-video).

Each timestep provides three camera views — the side cameras get a steering correction so the model learns how to steer back toward the lane centre:

Left	Center	Right

🧠 Model

A grayscale 160 × 320 frame is cropped (sky + hood removed) and normalized, then fed through the NVIDIA convolutional backbone and a dense regression head.

cropping2d ─▶ lambda(normalize) ─▶ 5× Conv2D (24,36,48,64,64) + MaxPool/Dropout
           ─▶ flatten ─▶ Dense(1164, 100, 50, 10) ─▶ Dense(4)
                                                       (~1.34M params)

Full layer-by-layer summary

See writeup_report.md for the complete architecture table, data-augmentation details, and training notes.

🚀 Quick start

Dependencies are managed with uv. Python is pinned to 3.10 (TensorFlow 2.15 ships no wheels for 3.12+). uv sync also installs the behavioral_cloning package and its console scripts:

uv sync

Note

Simulator compatibility: the Udacity simulator speaks the Engine.IO v3 protocol, so the driving server (bc-drive) requires python-socketio < 5 / python-engineio < 4. These are pinned in pyproject.toml; do not bump them without re-testing the simulator connection.

Place each recorded dataset in a sub-folder of a data directory (default data/), each containing a driving_log.csv and its IMG/ frames.

# Train (writes <modelname>.h5 and loss.png)
uv run bc-train --datadir data -d track1 track1_recovery -m model

# Drive the car autonomously (start the simulator in autonomous mode first)
uv run bc-drive results/track1.h5

# Record the run, then build a video from the saved frames
uv run bc-drive results/track1.h5 run_images
uv run bc-video run_images --fps 60

Each command is also available as python -m behavioral_cloning <train|drive|video>.

🧪 Tests

uv run pytest -q

The unit tests cover the pure data/model logic (augmentation, target normalization, model shapes, the PI controller) and run on CPU without a GPU, simulator, or recorded data.

🎬 Results

A trained model and a recorded autonomous run are checked in under results/:

• 🧠 Model — results/track1.h5
• 🎥 Video — results/run1.mp4

The car drives full-speed around track one for multiple consecutive loops without leaving the road.

🗂️ Project layout

Path	Purpose
behavioral_cloning/data.py	Dataset parsing and three-camera augmentation
behavioral_cloning/network.py	NVIDIA / LeNet model architecture
behavioral_cloning/training.py	Training entry point (bc-train)
behavioral_cloning/driving.py	Autonomous-driving server (bc-drive)
behavioral_cloning/video.py	Frame-to-video helper (bc-video)
behavioral_cloning/gpu.py	Shared GPU configuration helper
tests/	Pytest unit tests (CPU-only)
results/	Trained model + demo recording
examples/	Sample camera frames
writeup_report.md	Project writeup (rubric report)

🛠️ Development

This repository uses pre-commit for hygiene, black, and ruff, plus Conventional Commits with a required scope (e.g. fix(model): ...):

uv run pre-commit install && uv run pre-commit install --hook-type commit-msg