AI-Powered In-Store Assistant for Retail
SAM transforms the in-store shopping experience by giving customers real-time inventory intelligence, contextual guidance, and sustainable nudges all through a sleek mobile app.
Retail customers often struggle to:
- Find products or check live availability, causing frustration and up to 30% sales loss from stockouts.
- Rely on staff, which drives up costs and causes in-store crowding.
- Discover eco-friendly alternatives at checkout.
We built a generative AI–powered assistant that customers launch by scanning a QR code at the store entrance. That QR code triggers a Supabase–FastAPI pipeline, streaming that location’s live inventory into the app with millisecond latency.
A custom LLM-driven intent engine (via OpenRouter API) classifies queries—whether a product search, recipe idea, or sustainability tip—and returns precise, actionable guidance. Behind the scenes, recipes are scraped with BeautifulSoup4, parsed via Pandas, and mapped against live stock to show aisle locations and quantities.
Speech-to-text (SpeechRecognition) and text-to-speech (pyttsx3) deliver a hands-free experience—vital for accessibility. Sustainable suggestions (e.g., paper straws, green-coin rewards) nudge greener shopping behaviors and boost repeat purchases.
+------------+ +-------------------------------------+ +--------------+
| React/Expo | <--> | FastAPI Backend | <--> | PostgreSQL & |
| Frontend | | +-------------------------------+ | | Supabase |
+------------+ | | AI Engine | | +--------------+
| | ┌─────────────────────────────┐ |
| | │ • Intent Classification │ |
| | │ • LLM (OpenRouter API) │ |
| | │ • Recipe Scraping (BS4) │ |
| | │ • Data Parsing (Pandas) │ |
| | │ • Speech In/Out (SR & TTS) │ |
| | └─────────────────────────────┘ |
+-------------------------------------+
- Frontend: React Native (Expo Go)
- Backend: FastAPI, Python
- Database & Realtime: PostgreSQL, Supabase
- LLM Intents: OpenRouter API
- Scraping & Analysis: BeautifulSoup4, Pandas
- Accessibility: SpeechRecognition, pyttsx3
- Node.js & npm
- Python 3.8+ & pip3
- Expo Go (mobile)
- Git
git clone https://github.com/codeitnav/Sparkathon-SAM.git
cd Sparkathon-SAMcd frontend
npm install
npx expo startScan the QR code in Expo Go (on the same Wi-Fi).
cd backend/API
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txtcd ../chatbot
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cd ../SAM
uvicorn main:app --host 0.0.0.0 --port 8000 --reload- Launch both backend services (API on port 8000, Chatbot on 5000).
- Start the Expo app and scan the store-specific QR code.
- In the mobile UI:
- Search a product or speak a recipe request.
- See real-time shelf locations, quantities, and checkout nudges.
- Tap the mic icon for hands-free voice input; listen to responses aloud.
- QR → Store ID
The app sends the store ID to FastAPI, which queries Supabase for that inventory snapshot. - Intent Classification
User input → OpenRouter LLM → JSON intent (e.g.{ "type": "recipe", "dish": "fried rice" }). - Data Enrichment
- For recipes: Scrape external sites (BeautifulSoup4), parse ingredients (Pandas).
- Match ingredients to live stock → return aisle & count.
- Accessibility Layer
- Voice in: SpeechRecognition → text.
- Voice out: pyttsx3 reads the JSON response.
- Sustainable Nudge
Suggest eco-friendly swaps + track “green coins” to boost loyalty.
We welcome code improvements, UX enhancements, or fresh comic panels ! Please fork, branch, and submit PRs with clear issue references.
Navya Srivastava – Frontend development & integration of services.
Ekansh Dubey – Dataset creation & UI design; Frontend assistance.
Aayush Chauhan – Chatbot core; backend assistance.
Manish Sharma – Supabase, FastAPI & backend development.
This project is MIT-licensed. Let’s spark smarter, greener retail together.