Weave

A plugin-based architecture for building AI agents with Python. Weave spawns plugins as separate processes, routes IPC via Redis Streams, and exposes a FastAPI HTTP bridge.

Features

• Async-First Architecture: Optimized for I/O-heavy AI agent workloads
• Process-Based Isolation: Each plugin runs in its own process with IPC via Redis Streams
• Hot Restart: Automatic plugin restart on crashes with exponential backoff
• Pub/Sub Messaging: Distributed event system and queue-based communication
• HTTP API: FastAPI bridge for external clients to call plugin methods
• Multi-Instance Support: Run multiple instances of the same plugin type

Async-First Architecture

Weave plugins are async-only by design, providing significant benefits for AI agent development:

Why Async?

1. High Concurrency: A single async plugin instance can handle hundreds of concurrent API calls (OpenAI, databases, external services) without spawning additional processes.

2. Responsive Cancellation: Async allows plugins to respond to stop signals and cancellation requests while waiting on long-running operations—critical for user experience.

3. Resource Efficiency: Lower memory footprint and fewer processes needed. One async process can handle 100 concurrent requests, whereas sync plugins would need 100 processes.

The Escape Hatch

For synchronous libraries (pandas, PIL, requests), use asyncio.get_running_loop().run_in_executor() to run blocking code in thread pools without freezing the event loop:

async def process_data(self):
    import pandas as pd
    
    def blocking_work(path):
        return pd.read_csv(path)
    
    loop = asyncio.get_running_loop()
    df = await loop.run_in_executor(None, blocking_work, "data.csv")
    return df.to_dict()

See docs/plugin/async-patterns.md for comprehensive guidance.

Quick Start

Prerequisites

• Python 3.11+
• Docker (for Redis Stack)

Installation

cd weave-server

# Create virtual environment
python -m venv .venv

# Activate (Windows PowerShell)
.\.venv\Scripts\Activate.ps1

# Activate (macOS/Linux)
source .venv/bin/activate

# Install
pip install -e .

# Start the server
python main.py

Server starts at http://localhost:8000

Create a Plugin

# plugins/my_plugin/main.py
from sdk import WeavePlugin
import asyncio

class MyPlugin(WeavePlugin):
    async def on_start(self):
        self.log("Plugin started!")
    
    async def greet(self, name: str) -> str:
        return f"Hello, {name}!"
    
    async def run_async(self):
        while True:
            await asyncio.sleep(60)
            self.log("Heartbeat...")

# plugins/my_plugin/plugin.yaml
name: "My Plugin"
version: "1.0.0"
plugin_id: "com.example.my-plugin"
entry_point: "main.py"
class_name: "MyPlugin"

permissions:
  ipc:
    - "host"

Test Your Plugin

# Call via HTTP
curl -X POST http://localhost:8000/api/plugins/com.example.my-plugin/greet \
  -H "Content-Type: application/json" \
  -d '{"name": "World"}'

Documentation

Internal developer docs live in docs/ and are built with MkDocs Material.

• Plugin Documentation - Getting started, examples, and reference
• Async Patterns Guide - Essential async development guide
• Local preview: pip install -r docs/requirements.txt && mkdocs serve
• Strict build: mkdocs build --strict
• Deploy: pushes to main run .github/workflows/docs.yml to publish to GitHub Pages

Architecture

• Host Server (weave-server/main.py): FastAPI server + process manager
• Core Subsystems (weave-server/core/):
  • loader.py: Manifest scanning + dependency resolution
  • process_manager.py: Plugin process spawning
  • router.py: IPC routing + permission enforcement
  • server.py: Event loop + HTTP bridge
  • redis_runtime.py: Redis Streams integration
• Plugin SDK (weave-server/sdk/):
  • plugin.py: Base WeavePlugin class
  • proxy.py: Plugin-to-plugin RPC proxies
  • client.py: Redis Streams client
  • queue.py, events.py: Pub/sub and event systems

Example Plugins

• Login Plugin - Authentication with inter-plugin calls
• Secure Database - Data storage with pub/sub
• More Examples - HTTP, databases, blocking code patterns

Contributing

See docs/plugin/dev-workflow-checklist.md for development best practices.

License

[Your License Here]