Pure Python Multi-Agent System

Small, typed demo that coordinates three agents (research, analysis, summary) and a verification agent with a simple orchestrator. No orchestration frameworks are used—only the Python standard library plus Pydantic for verifier outputs.

Setup (Python 3.12 + uv)

env UV_CACHE_DIR=.uv-cache uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e .

Optional dev tools (pytest for tests):

uv pip install -e ".[dev]"

Run the demo

python main.py "What are the key benefits of geometric multi-agent systems?"

Outputs include the final summary, verification verdict, and an agent trace with timestamps.

Tests

python -m pytest

Architecture

• Agents: Each agent is a class with process(input: str) -> AgentMessage returning typed Pydantic models with metadata (timestamp, agent_id, confidence, status). Downstream agents accept serialized JSON from the prior stage to keep signatures string-based.
  • ResearchAgent → mock findings and sources from the query.
  • AnalysisAgent → prioritizes findings, infers stance, surfaces risks.
  • SummaryAgent → crafts a narrative summary and highlights.
  • VerificationAgent → cross-checks analysis vs. summary.
• Prompts: Lightweight PromptTemplate objects feed mock prompt strings into each agent to mirror LLM-style templating without dependencies.
• Orchestrator: Declarative plan (PlanStep list) walks research → analysis → summary → verification and returns a final dict with summary, verification result, and raw outputs for traceability. Both sync (run) and async (run_async) entrypoints are available.
• Configuration: Tunables live in config.py (confidence defaults, verification thresholds).

Design decisions

• String-only agent interfaces: Agents keep process(input: str) per requirement; JSON serialization/deserialization keeps typed payloads flowing between steps.
• Pydantic models everywhere: Structured, validated outputs replace dataclasses/TypedDict for clarity and metadata handling.
• Declarative orchestration plan: A PlanStep DAG drives sequencing and dependency checks, with verification running asynchronously.
• Mock prompt templates: PromptTemplate injects LLM-like prompts without pulling in template libraries.
• Verification fan-out: Verifiers run concurrently with weighting, variance-based disagreement, and retry/backoff for resilience.

Verification Logic

• Structure: Verifiers share a common async interface (BaseVerifier.verify(...)) returning a Pydantic VerifierResult (score, passed flag, reasoning, verifier_id). The orchestrator holds a list of verifiers and runs them concurrently via asyncio.gather.
• Current verifier: CoverageAlignmentVerifier checks coverage of analysis key points in the summary, alignment with stance, and whether risks are acknowledged.
• Combining results: The verification agent applies weights, computes a weighted score, and flags disagreement when score variance exceeds a threshold. Majority vote + weighted score decide pass/fail. Confidence drops as variance grows or verifiers fail.
• Error handling: Each verifier call is retried with backoff; persistent failures lower confidence and are surfaced in issues.

Files

• main.py – CLI entrypoint.
• orchestrator/orchestrator.py – orchestrates agent calls and assembles the final response.
• agents/ – individual agent implementations and base protocol.
• verification/base.py – verifier interface and Pydantic result model.
• verification/verifier.py – verification agent + default coverage/alignment verifier.
• models.py – shared Pydantic models.
• tests/test_workflow.py – unit tests for agents, verification, and the orchestrator.
• pyproject.toml – Python version and optional dev dependency metadata.