Level 2 Agentic System — Code Review Coordinator

A multi-agent code review pipeline where a coordinator LLM delegates specialized analysis tasks to sub-agent tools, each powered by its own LLM with a focused system prompt. The coordinator never analyzes code directly — it orchestrates.

---

What It Does

Given a PR diff, the system produces a structured JSON report covering three dimensions:

Field	Source
bugs	bug_detector sub-agent
style_violations	style_reviewer sub-agent
improvement_suggestions	improvement_advisor sub-agent
summary	coordinator (synthesized from sub-agent outputs)

---

Project Structure

1_level_2/
├── coordinator.py          # Coordinator agent — orchestrates tool calls
├── run.py                  # Entry point — loads a diff and runs the pipeline
├── agents/
│   ├── bug_detector.py     # Sub-agent: logic errors, null dereferences, race conditions
│   ├── style_reviewer.py   # Sub-agent: naming, formatting, dead code, readability
│   └── improvement_advisor.py  # Sub-agent: maintainability and performance suggestions
├── utils/
│   └── verbose_callback.py # Logs tool call/return events to stdout
├── diffs/
│   └── buggy.diff          # Sample PR diff with intentional bugs and style issues
└── requirements.txt

---

This project is Level 2 because:

1. The coordinator does not analyze code itself. The coordinator's system prompt instructs it to "run the available tools, then assemble their outputs." It acts purely as an orchestrator — it decides which tools to call and how to combine their results, not what the bugs or violations are.

2. Each sub-agent is itself an LLM with a specialized role. bug_detector, style_reviewer, and improvement_advisor are not deterministic functions or API calls — each one invokes a separate ChatOpenAI instance with a focused system prompt that constrains its reasoning domain. This is the key Level 2 distinction: the "tools" are themselves LLMs.

3. Delegation happens via tool calling. The coordinator is a LangChain agent (create_agent) that receives the three sub-agents as @tool-decorated callables. When the coordinator LLM decides to call bug_detector, LangChain executes the tool call protocol (serialize → invoke → return result), and the coordinator continues reasoning with the result in its context. This is the standard tool-calling loop that defines Level 2 coordination.

4. The coordinator synthesizes across sub-agent outputs. After all tools return, the coordinator assembles the results into a single unified JSON review and adds an overall quality summary — a task that requires reading and reasoning over multiple sub-agent outputs, not just passing them through.

Flow diagram

PR diff
   │
   ▼
┌─────────────────────────────────────┐
│         Coordinator LLM             │
│  (decides which tools to call)      │
└──────┬──────────┬───────────┬───────┘
       │ tool call │ tool call │ tool call
       ▼           ▼           ▼
 ┌──────────┐ ┌──────────┐ ┌──────────────────┐
 │  bug_    │ │  style_  │ │   improvement_   │
 │ detector │ │ reviewer │ │    advisor       │
 │  (LLM)   │ │  (LLM)   │ │    (LLM)         │
 └────┬─────┘ └────┬─────┘ └──────┬───────────┘
      │            │              │
      └────────────┴──────────────┘
                   │ JSON arrays
                   ▼
         ┌─────────────────┐
         │  Coordinator    │
         │  assembles      │
         │  final report   │
         └────────┬────────┘
                  │
                  ▼
           Structured JSON
    { bugs, style_violations,
      improvement_suggestions,
      summary }

---

Setup

pip install -r requirements.txt
cp .env.example .env
# Add your OPENAI_API_KEY to .env

Run

python run.py

The script runs the coordinator twice on diffs/buggy.diff and prints a live trace of each tool call and its completion via _VerboseCallback.

---

Sample Diff

diffs/buggy.diff contains a PR with several intentional problems across auth/login.py and auth/middleware.py, including:

• An off-by-one error in get_all_users (range(len(users) + 1))
• MD5 used for password hashing in reset_password
• Missing null check after get_user (both files)
• Missing null/missing-key check after validate_token
• Naming convention violations (Login instead of login)
• Comparison against True explicitly (== True)
• Unauthenticated handler still called even when token is missing in require_auth

The three sub-agents divide responsibility cleanly: bugs go to bug_detector, naming and style issues go to style_reviewer, and structural improvements go to improvement_advisor.