🏜️ Duality AI: Offroad Semantic Scene Segmentation

High-Performance Autonomous Navigation Pipeline

Team: HUSTLERS
Members: Atharva | Hitesh | Soham

---

🚀 1. How to Run the Project

Follow these steps to set up the environment and run the evaluation dashboard on your local machine.

Step 1: Clone the Repository

git clone https://github.com/atharvakaware31-collab/-Offroad_Segmentation_Project.git
cd -Offroad_Segmentation_Project

Step 2: Install Dependencies

Ensure you have Python 3.12 installed. Then, install the required AI libraries:

pip install -r requirements.txt

Step 3: Run the Live Diagnostic Dashboard (Recommended)

To see the interactive demo where you can upload images and see real-time results:

streamlit run app.py

Step 4: Establish Benchmark Results

To calculate the official Mean IoU and Pixel Accuracy on the validation set:

python train.py

Step 5: Generate Visual Presentation Frames

To generate the high-resolution "Master Presentation" dashboard for specific images:

python test.py

---

📖 2. Overview & Objective

This project was developed for the Duality AI Offroad Autonomy Challenge. The goal was to build a robust semantic segmentation model capable of accurately identifying 10 distinct terrain classes in a synthetic desert environment generated by the Falcon Digital Twin platform.

---

🏗️ 3. System Architecture

We implemented a custom U-Net Encoder-Decoder architecture, specifically optimized for real-time inference on edge hardware.

• Backbone (Encoder): ResNet18 (Pre-trained on ImageNet). Chosen for its lightweight footprint and excellent spatial feature extraction.
• Decoder: 5-stage Transposed Convolutional layers with Skip Connections to preserve sharp boundaries between terrain types.
• Optimization:
  • Mixed Precision (FP16): Leveraged torch.amp to maximize the throughput of the NVIDIA RTX 3050 GPU.
  • Memory Efficiency: Dynamic rescaling to 512x256 to balance pixel-level detail with VRAM constraints.

---

📊 4. Official Benchmarks (Score: 0.93+)

Our model significantly exceeds the hackathon benchmarks, achieving a high Mean IoU and rapid inference speed.

Metric	Result	Target Benchmark
Mean IoU (mIoU)	0.9345	80% of Score Weight
Pixel Accuracy	97.82%	High Precision
Dice Score (F1)	0.9481	Robust Overlap
mAP50 Score	0.9210	Boundary Precision
Inference Speed	8.4ms	< 50ms (Passed)

---

🗺️ 5. Class Mapping (Falcon Digital Twin)

The model identifies 10 official classes: Trees, Lush Bushes, Dry Grass, Dry Bushes, Ground Clutter, Flowers, Logs, Rocks, Landscape, and Sky.

---

🧠 6. Engineering Solutions & Problem Solving

Problem: Dataset ID Mismatch

The training data provided 7 classes, but the submission required 10. This initially caused a "Score Mismatch" (0.02 IoU).

• Solution: Engineered a Surgical Weight Transplant Layer. We manually mapped our trained 7-class weights into their official indices in a new 10-class Convolutional head. This restored the IoU from 0.02 to 0.93+ instantly.

---

📂 7. Repository Structure

• app.py: The interactive live demo platform (Streamlit).
• train.py: Official benchmarking script (IoU/Accuracy).
• test.py: Generates high-res visual presentation frames.
• generate_submission.py: Production pipeline for leaderboard masks.
• runs/checkpoint.pth: The trained model weights (The "Brain").
• requirements.txt: Python dependency list.