Voice-Based Laptop Assistant

This project is a voice-driven desktop assistant prototype that turns spoken commands into simple system actions. It uses an LLM to interpret intent and then maps that intent to local actions such as opening apps, launching browser tabs, searching the web, opening folders, and taking screenshots.

Features

• Speech-to-text command capture
• LLM-based command parsing
• Multi-action command support
• Open and close common desktop apps
• Open browser tabs, including private mode
• Search the web
• Open common folders such as Downloads, Documents, and Desktop
• Capture screenshots

Requirements

• Windows
• Python 3.10+
• Gemini API
• Microphone access for speech input

Setup

1. Create and activate a virtual environment.

2. Install the Python dependencies:

   pip install -r requirements.txt

   The current prototype also imports google-genai and Pillow, so install them if they are not already available in your environment.

3. Set your API key:

   set GEMINI_API_KEY=your_api_key_here

Usage

The main prototype lives in main_LLM.py.

• listen_command() captures speech from the microphone
• call_llm() converts raw text into structured actions
• process_command() executes the parsed actions

Supported intents:

• OPEN_APP
• CLOSE_APP
• SEARCH
• NEW_TAB
• OPEN_FOLDER
• TAKE_SCREENSHOT

Example commands:

• Open Chrome
• Close Notepad
• Search for AI news
• Open downloads folder
• Open new private tab in Edge
• Open new tab in Firefox

Notes

• The assistant is currently a prototype and is heavily Windows-specific.
• Browser-specific features are supported for Chrome, Edge, Firefox, and Brave when the browser is installed locally.
• Some actions rely on app names available on the local machine.
• main_LLM.py contains experimental code paths, so treat it as a starting point for further development.

Project Structure

• main_LLM.py — speech handling, LLM parsing, and desktop actions
• requirements.txt — Python dependencies used by the project