An AI-powered voice study companion for DeepLearning.ai courses
Features • Demo • Installation • Usage • Development • Contributing
Talk to your AI study buddy using Google Gemini Live API. Ask questions about the course content, get explanations, and have natural conversations while watching video lessons.
Automatically extracts and syncs with video transcripts. The AI has full context of what you're learning, so you can ask "What did they just say about transformers?" and get accurate answers.
The AI knows exactly where you are in the video. Pause at 5:15 to ask a question? Study Buddy understands the context up to that point and responds based on what you've just watched. Seek forward or backward? The AI instantly adapts and explains concepts from your new position—no need to explain what you're referring to!
Ask for visual explanations and get beautiful Mermaid diagrams rendered inline:
- Flowcharts for processes
- Sequence diagrams for interactions
- State diagrams for ML pipelines
- And more!
Generate quizzes based on the lesson content to test your understanding:
- Multiple difficulty levels (Easy, Medium, Hard)
- Instant feedback with explanations
- Score tracking
Take notes that are linked to specific moments in the video:
- Click on any note to jump to that timestamp
- Visual markers on the video timeline
- Persistent storage across sessions
A sleek, modern sidebar that integrates seamlessly with the DeepLearning.ai interface:
- Collapsible and resizable
- Personalized greetings
- Real-time audio visualizer
- Dark mode design
- Node.js (v18 or higher)
- npm (comes with Node.js)
- A Google Gemini API Key (Get one here)
git clone https://github.com/yourusername/deeplearning-study-buddy.git
cd deeplearning-study-buddynpm installnpm run buildThis creates a dist/ folder containing the built extension.
- Open Chrome and navigate to
chrome://extensions/ - Enable Developer mode (toggle in top-right corner)
- Click "Load unpacked"
- Select the
dist/folder from this project
- Go to any DeepLearning.ai course lesson
- The Study Buddy sidebar will appear on the right
- Find the API Key section in the sidebar on the top section
- Enter your Gemini API key and save
- Navigate to any lesson on learn.deeplearning.ai
- The Study Buddy sidebar will appear on the right
- Click "Start Session" to connect
- Wait for the AI to analyze the transcript
- Start talking! 🎤
Just speak naturally! Here are some things you can ask:
- "Explain what the instructor just said"
- "Can you draw a diagram of this architecture?"
- "Give me a quiz on this topic"
- "What's the difference between CNN and RNN?"
- "Summarize this lesson"
- Click the + Note button at any point in the video
- Type your note in the modal
- Notes are automatically linked to the current timestamp
- Click any note to jump back to that moment
| Shortcut | Action |
|---|---|
| Mute button | Toggle microphone mute |
deeplearning-study-buddy/
├── src/
│ ├── background/
│ │ └── serviceWorker.ts # Chrome extension background worker
│ ├── content/
│ │ ├── contentScript.ts # Main content script
│ │ ├── transcriptExtractor.ts
│ │ ├── videoObserver.ts
│ │ └── courseContext.ts
│ ├── sidebar/
│ │ ├── sidebar.ts # Sidebar logic
│ │ ├── sidebar.html
│ │ └── styles.css
│ ├── services/
│ │ ├── geminiLiveService.ts # Gemini Live API integration
│ │ └── notesStorage.ts
│ ├── utils/
│ │ └── audio.ts
│ └── types.ts # TypeScript interfaces
├── icons/
├── manifest.json
├── package.json
├── tsconfig.json
└── vite.config.ts
For hot-reloading during development:
npm run devThis watches for file changes and rebuilds automatically.
npm run build- TypeScript - Type-safe JavaScript
- Vite - Fast build tool
- Google Gemini Live API - Real-time voice AI
- Mermaid - Diagram rendering
- D3.js - Data visualization
- Chrome Extension Manifest V3
Your API key is stored locally in Chrome's extension storage. It never leaves your browser except when making requests to the Gemini API.
The extension requires these permissions:
activeTab- To interact with the current tabstorage- To save notes and settingshost_permissionsforlearn.deeplearning.ai- To inject the sidebar
We welcome contributions! Here's how you can help:
- Check existing issues
- If not found, create a new issue with:
- Steps to reproduce
- Expected vs actual behavior
- Screenshots if applicable
Open an issue with the enhancement label describing:
- The problem you're trying to solve
- Your proposed solution
- Any alternatives you've considered
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Make your changes
- Run the build to ensure no errors (
npm run build) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
- Use TypeScript for all new code
- Follow existing patterns in the codebase
- Add meaningful comments for complex logic
This project is licensed under the MIT License - see the LICENSE file for details.
- DeepLearning.ai for creating amazing courses
- Google Gemini for the incredible AI APIs
- Mermaid for beautiful diagram rendering
Have questions? Reach out!
- GitHub Issues: Create an issue
Made with ❤️ for the AI learning community