Jarvis is a modular, high-performance, and privacy-centric AI assistant designed specifically for the Linux desktop (optimized for Fedora/GNOME). Unlike traditional assistants that rely on cloud APIs, Jarvis operates on an offline-first philosophy, performing speech recognition, intent parsing, and neural speech synthesis entirely on your local hardware.
- 🛡️ Privacy by Design: All voice processing (STT), reasoning (LLM), and speech (TTS) happen locally. No audio data ever leaves your machine.
- ⚡ Hybrid Intent Engine: A three-tier routing system that combines high-speed Regex/Semantic matching with a sophisticated LLM fallback (via Ollama).
- 🎤 Neural Voice Interface:
- STT: Powered by
faster-whisperfor near-instant, high-accuracy transcription. - TTS: High-quality, interruptible neural speech synthesis using
Piper.
- STT: Powered by
- 🛠️ Desktop Integration: Native GNOME extension and overlay for visual feedback and seamless system control.
- 🔒 Secure Auth: Voiceprint authentication for sensitive operations like
sudoor file deletion. - 🔄 Event-Driven Architecture: Fully asynchronous Core built on a central Event Bus, ensuring zero-latency response times.
Jarvis follows a strictly decoupled, Pub-Sub Event-Driven design.
- Wake: The
AudioPipelinefeeds 16kHz chunks to theWakeWordDetector(Vosk/openWakeWord). - Listen: Upon trigger, the system shifts to
LISTENINGmode. An interruptible "Yeah?" acknowledgement plays while the microphone buffers speech. - Transcribe: Silence detection (VAD) triggers
STTEngine(Whisper) to convert audio to text. - Think: The
IntentRouterevaluates the text:- Tier 1: Semantic Fast Match (Keyword verbs).
- Tier 2: Regex Path (System/Media controls).
- Tier 3: Contextual LLM (Ollama/Phi-3) for complex natural language.
- Act: The
ActionRegistryexecutes the resolved command (App launch, volume control, web search, etc.). - Speak:
TTSEngineprovides verbal confirmation via neural synthesis.
- Core: Python 3.10+,
asyncio - Audio:
PyAudio,Vosk/openWakeWord - STT:
faster-whisper(CTranslate2) - LLM:
Ollama(default:phi3:mini) - TTS:
Piper(ONNX Runtime) - UI: GNOME Shell Extension (JavaScript), Python Overlay (Gtk/Qt)
- Linux (Fedora/GNOME recommended)
- Python 3.10 or higher
- Ollama installed and running (
ollama run phi3:mini) piper-ttsandffmpeginstalled on your system.
-
Clone the repository:
git clone https://github.com/yourusername/jarvis.git cd jarvis -
Set up Virtual Environment:
python -m venv .venv source .venv/bin/activate pip install -r requirements.txt -
Install GNOME Extension (Optional):
cd ui/gnome-extension ./install_extension.sh -
Configure: Edit
jarvis.yamlto match your hardware paths (especially TTS voice paths and Mic indices).
Jarvis is highly customizable via the central configuration file:
assistant:
wake_word: "jarvis"
follow_up_timeout: 12 # Seconds to keep listening
stt:
model_size: "base.en" # tiny, base, small, medium
fallback: "google" # Cloud fallback if offline fails
llm:
model: "phi3:mini"
temperature: 0.2
security:
voice_auth_enabled: true
sensitive_actions: ["system.sudo", "file.delete"]├── core/ # Application lifecycle & Event Bus
├── audio/ # Wake word & Microphone processing
├── brain/ # Intent parsing & LLM integration
├── actions/ # Extensible command modules (Apps, Media, Sys)
├── voice/ # STT and Neural TTS engines
├── ui/ # GNOME Extension & Desktop Overlay
├── data/ # Local models (Vosk, Wake Word)
└── tests/ # Pytest suite
We welcome contributions! Whether it's adding new actions/, improving the brain/ logic, or enhancing the ui/.
- Fork the repo.
- Create your feature branch (
git checkout -b feature/AmazingFeature). - Commit your changes.
- Push to the branch.
- Open a Pull Request.
Distributed under the MIT License. See LICENSE for more information.
Built with ❤️ for the Linux Community.