π§ Next-Gen Browser-Level AI Copilot β Understands Web Semantics, Executes Complex Tasks, Auto-Decides & Acts
Demo Videos β’ Core Features β’ Quick Start β’ Architecture β’ δΈζ
Browser Pilot is an intelligent browser assistant plugin built on the openJiuwen Agent Framework. It not only deeply understands web content but also reasons, plans, and executes based on user intent β acting as a true AI copilot that automatically completes multi-step, cross-page complex tasks.
π― Tell it what you want to do, and it will think, execute, and even learn from failures.
| π Smart Q&A | πΌ Office Assistance | π Complex Tasks | π Self-Evolution |
|---|---|---|---|
| Direct Q&A on web content | Auto-reply to emails | Watch recipe video β Auto-add ingredients to cart | Auto-reflect on failures |
| Precise screenshot region recognition | Excel data processing | Cross-app multi-step workflows | Context-based learning optimization |
| Deep Search | Form filling & submission | Price comparison, booking, ordering | Gets smarter with use |
Let AI see everything you see
- Web Q&A: Open any webpage, directly ask AI about page content
- Screenshot Recognition: Select any screen region, AI answers based on image content
- Deep Search: Complex questions auto-decomposed, multi-round search, deep summarization
Let AI handle repetitive work
- Email Processing: Read email content, understand context, auto-compose professional replies
- Excel Processing: Data cleaning, formula calculation, chart generation, report export
- Form Filling: Smart field recognition, auto-fill and submit
Cross-app, multi-step, seamless execution
- Example Scenario: Watch recipe video/article β Extract ingredient list β Open shopping app β Auto-add to cart
- Workflow Orchestration: Auto-plan task steps, sequential execution, exception handling
- Cross-Platform Coordination: Seamless connection between browser and local apps
AI that gets smarter with use
- Failure Reflection: Auto-analyze reasons when tasks fail
- Context Learning: Optimize execution strategies based on interaction history
- Self-Correction: Adjust approach and retry until success
Browser Pilot consists of a Browser Plugin (Frontend) and an Agent Service (Backend).
- Python 3.11+
- uv package manager
- Node.js 18+
- Chrome browser
git clone https://github.com/xxxx/browser-pilot.git
cd browser-pilot# Install uv (if not already installed)
# Windows: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
# macOS/Linux: curl -LsSf https://astral.sh/uv/install.sh | sh
uv sync
# Activate virtual environment
.\.venv\Scripts\Activate.ps1
## Install openjiuwen and browser-use
uv pip install openjiuwen==0.1.2
uv pip install browser-use==0.11.2
Create .env file at src/super_agent:
# === LLM Provider Configuration (at least one required) ===
# Option 1: OpenRouter (Recommended, supports multiple models)
OPENROUTER_API_KEY=your_openrouter_api_key
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1 # Optional, has default
# Option 2: Anthropic Direct
# ANTHROPIC_API_KEY=your_anthropic_api_key
# ANTHROPIC_BASE_URL=https://api.anthropic.com # Optional, has default
# Option 3: OpenAI Direct
# OPENAI_API_KEY=your_openai_api_key
# OPENAI_BASE_URL=https://api.openai.com/v1 # Optional, has default
# === Model Configuration ===
MODEL_NAME = "anthropic/claude-sonnet-4.5" # Optional, default: anthropic/claude-sonnet-4.5
MODEL_PROVIDER = "openrouter" # Optional, default: openrouter
# === Search Features (configure as needed) ===
# SERPER_API_KEY=your_serper_api_key
# GOOGLE_API_KEY=your_google_api_key
# PERPLEXITY_API_KEY=your_perplexity_api_key
# GEMINI_API_KEY=your_gemini_api_key
# === Browser Configuration ===
# BROWSER_USE_CDP_URL=http://127.0.0.1:9222 # Set when connecting to remote Chrome
# BROWSER_USE_LLM_MODEL=google/gemini-2.5-pro # Optional, has default
β οΈ Before the first run, please editbrowser_start_client.ps1orbrowser_start_client.shand update the Chrome executable path to match the Chrome installation on your local machine.
Windows:
# Start
.\start_agent.ps1macOS:
./start_agent.sh
Windows:
# Start
.\browser_start_client.ps1macOS:
./browser_start_client.sh- Visit
chrome://extensions/ - Enable "Developer mode" in top right
- Click "Load unpacked"
- Select
frontend/distdirectory - Open the plugin, click settings and enter http://localhost:8000 in Backend_URL
- Ensure backend services (port 8930 + port 8000) are running
- Launch Chrome browser with agent control support via the browser_start_client script
- Open any webpage or 1-2 blank tabs
- Click Browser Pilot icon in browser toolbar
- Enter a question to test if AI responds normally
| Feature | Description |
|---|---|
| SuperReAct | Enhanced ReAct loop: Think β Act β Observe, supports multi-round iterative reasoning |
| Browser Use | Browser automation, cross-page, multi-step task execution |
| Reflective Evolution | Auto-reflect on failures, adjust strategy and retry, gets smarter with use |
| Multimodal Visual Understanding | Supports visual understanding and Q&A for web content and screenshot regions |
Recommended to use OpenRouter for unified access (supports 100+ models), while also compatible with direct API connections to various providers:
| Provider | Models |
|---|---|
| Anthropic | Claude Sonnet 4 (Recommended), Claude Opus 4 |
| OpenAI | GPT-4o, O3, O3-mini |
| Gemini 2.0 Pro/Flash | |
| Others | Qwen, DeepSeek, Llama, etc. |
Contributions are welcome!
- Fork this repository
- Create feature branch (
git checkout -b feature/new-feature) - Commit changes (
git commit -m 'Add new feature') - Push branch (
git push origin feature/new-feature) - Submit Pull Request
- openJiuwen Agent Framework - Underlying Agent framework
β If you find this helpful, please give us a Star!