BlindCooker

An iOS/Android app that helps visually impaired users cook independently using AI-powered image analysis, voice guidance, and a Bluetooth scale.

How It Works

1. Scan your ingredients - Point camera at fridge, pantry, or table
2. Pick a recipe - AI suggests simple Italian-style pasta recipes based on what's available
3. Gather ingredients - Voice guides you to each ingredient's location ("top left shelf in the fridge")
4. Cook step by step - Audio instructions with sensory cues (smell, sound, temperature) instead of visual descriptions

Features

• AI Vision - Powered by Google Gemini 2.5 Flash for ingredient detection and recipe suggestions
• Location Guidance - Precise ingredient locations ("bottom right of the table", "door shelf in the fridge")
• Sensory Cooking - Instructions describe what to smell, hear, and feel instead of visual cues
• Safety Warnings - Alerts before steps involving heat, sharp objects, or hot liquids
• Bluetooth Scale - Real-time weight feedback for precise measurements
• Voice Control - All interaction via gestures: tap, double-tap, swipe, hold

Gesture Controls

Gesture	Action
Tap	Select / Confirm
Double Tap	Go back
Swipe Left/Right	Navigate between options
Hold 2 seconds	Repeat last spoken message

Setup

1. Get a Gemini API Key

Visit Google AI Studio and create a free API key.

2. Install & Run

iOs users

flutter pub get
cd ios && pod install && cd ..
flutter run

Android users

flutter pub get
flutter run

3. Configure

1. Grant camera and Bluetooth permissions
2. Open Settings and enter your API key
3. Connect to "BlindCooker-Scale" if using the Bluetooth scale

Hardware (Optional)

The firmware/ directory contains ESP32 code for a Bluetooth kitchen scale that integrates with the app for real-time weight measurements during cooking.

Project Structure

lib/
├── main.dart
├── config/
│   └── prompts_config.dart      # AI prompts for vision analysis
├── models/
│   ├── cooking_step.dart        # Cooking step with sensory cues
│   └── fridge_scan_result.dart  # Scanned items and recipes
├── providers/
│   └── app_provider.dart        # State management
├── screens/
│   ├── home_screen.dart         # Main entry
│   ├── camera_screen.dart       # Ingredient scanning
│   ├── recipe_selection_screen.dart
│   ├── ingredient_guide_screen.dart
│   ├── ingredient_locate_screen.dart
│   ├── cooking_screen.dart      # Step-by-step cooking
│   ├── cooking_progress_scan_screen.dart
│   └── settings_screen.dart
└── services/
    ├── gemini_service.dart      # Gemini API
    ├── tts_service.dart         # Text-to-speech
    ├── bluetooth_service.dart   # Scale connection
    └── api_key_service.dart     # Secure storage

Tech Stack

• Flutter 3.9+ with Provider
• Google Gemini 2.5 Flash API
• flutter_tts for voice output
• flutter_blue_plus for Bluetooth scale
• flutter_secure_storage for API key

Privacy

• API keys stored in iOS Keychain
• Images compressed before sending (max 1024px)
• No data stored or collected

License

MIT