Research Paper Analyzer with Multi-Agent AI

Metis, a full-stack Django-React application leveraging LangGraph for multi-agent orchestration, Tavily API for intelligent web search, and OpenAI GPT-4-mini/o3 to analyze research papers and provide an AI-powered chat interface.

Motivation

As a student doing research, I found it difficult and time-consuming to understand a new study. The hardest parts for me were discerning what was important versus what could be left for later, which drained a lot of time. A researcher can upload a paper and chat with the agent, inquiring about specific sections, citations, proofs, outside sources, and so on. The RAG Agent will simplify the job of the researcher by finding external sources to keep its answers up relevant and beneficial to the user, helping them streamline the process of understanding a study.

Setup Instructions

Backend Setup

1. Install Python dependencies:

cd backend
pip install -r requirements.txt

1. Configure environment variables:

# .env file
SECRET_KEY=your-django-secret
DEBUG=True
OPENAI_API_KEY=sk-...
TAVILY_API_KEY=tvly-...
CORS_ALLOWED_ORIGINS=http://localhost:5173

1. Run migrations:

python manage.py makemigrations
python manage.py migrate

1. Start server:

python manage.py runserver

Frontend Setup

1. Install Node dependencies:

cd frontend
npm install

2. Configure API URL:

# .env.development
VITE_API_URL=http://127.0.0.1:8000/api

3. Start dev server:

npm run dev

Frontend runs at http://localhost:5173/

Backend Overview

The backend utilizes a Django framework with a relational database (SQLite in this case, although in production a different relational database would be preferred). The requirements said to use MongoDB. However, due to the relational nature of this application, I thought it would be more efficient to use a relational database and put indices on queried fields.

The backend split up into 3 parts:

1. The User and UserProfile table (backend/users). The former is predefined by Django and keeps track of user information, such as username, if the user is an admin, and so on. The UserProfile option is defined by me, and currently only has the theme_preference option to toggle and save light/dark mode.
   • This option is there for future scalability; if more user-specific feateres needed to be defined, UserProfile provides that flexibility.
2. The ResearchPaper and ChatMessage tables (backend/research).

ResearchPaper - User-uploaded papers

• user: ForeignKey to User (JWT authenticated)
• title: Extracted by agent
• file: PDF stored in /media/papers/
• summary: JSON-serialized paper_knowledge (concepts, findings, citations, sections), extracted by agent
• created_at: Timestamp

ChatMessage - Conversation history

• paper: ForeignKey to ResearchPaper
• role: "user" or "assistant"
• content: Message text (user query or agent response)
• citations: JSON list of external sources used
• timestamp: Conversation ordering

3. The use of AI Agents, discussed below.

There also exists logging, visible at backend/agent_logs.log.example, showing the logs for the recorded test run.

Agent Overview

Multi-Agent System (multi_agent_orchestrator.py)

The application uses a collaborative multi-agent architecture built with LangGraph, where specialized AI agents work together to understand research papers and answer queries.

The Agents

1. CoordinatorAgent (o3)

• Analyzes user queries to determine intent and strategy
• Routes tasks to appropriate specialized agents
• Decides when external search is needed vs paper-only answers
• Creates execution plans based on query complexity
• Manages agent communication and workflow

2. ExtractionAgent (GPT-4.1-mini)

• Extracts paper title from PDF text
• Parses document into structured sections (Abstract, Introduction, Methodology, Results, etc.)
• Handles PDF text cleaning and normalization
• Sends structured data to KnowledgeAgent

3. KnowledgeAgent (o3)

• Builds comprehensive paper understanding
• Extracts key concepts, findings, and citations
• Identifies the Research Paper's fields of expertise (CS, Physics, Economics, etc.)
• Creates structured knowledge for the paper and stores it for future reference.

4. SearchAgent (Tavily Search, Extract)

• Executes web searches via Tavily Search on relevant terms found
• Searches for cited papers mentioned in bibliography
• Extracts text from cited work pulled from search using Tavily Extract
• Filters results by relevance score (>0.6 threshold)
• Returns high-quality external sources to supplement paper analysis for synthesis

5. SynthesisAgent (o3)

• Combines information from all sources (paper + web + citations)
• Generates coherent, concise answers
• Adapts response length based on query type
• Provides in-text citations when requested
• Avoids hallucination by admitting shortcomings in knowledge.

---

LangGraph Workflow

LangGraph orchestrates agent collaboration through a state-based graph with conditional routing:

START → CoordinatorAgent (analyze intent)
         ↓
    [Paper already uploaded?]
         ↓                    ↓
        NO                   YES
         ↓                    ↓
   ExtractionAgent     CoordinatorAgent
         ↓              (search decision)
   KnowledgeAgent            ↓
         ↓            [External search needed?]
         └─────→            ↓              ↓
                          YES             NO
                           ↓              ↓
                      SearchAgent         ↓
                           ↓              ↓
                           └──────────────┘
                                  ↓
                            SynthesisAgent
                                  ↓
                                 END

State Management:

• Shared ResearchState dictionary: PDF content, extracted knowledge, search results, query intent, agent messages
• Conditional edges route based on state, for example to skip extraction or searching nodes.

Agent Communication:

• Agents send typed messages (AgentMessage) to each other
• Messages include: sender, receiver, type (request/response/notification), payload
• All messages logged

---

Tavily Integration

Tavily complements the SearchAgent with up-to-date results.

1. Field-Specific Domain Filtering

FIELD_DOMAINS = {
    "cs": ["arxiv.org", "acm.org", "ieee.org", "github.com"],
    "medicine": ["pubmed.ncbi.nlm.nih.gov", "nature.com", "nejm.org"],
    "biology": ["pubmed.ncbi.nlm.nih.gov", "biorxiv.org", "nature.com"],
    ...
}

• SearchAgent detects research field from paper (CS, biology, etc.)
• Tavily searches only relevant academic domains
• Combines universal sources (Wikipedia, Google Scholar) with field-specific ones

2. Advanced Search Parameters

results = tavily_client.search(
    query=optimized_query,
    search_depth="advanced",          # Deep web crawling
    max_results=3,
    include_domains=field_domains,    # Field-specific filtering
    time_range="year"                 # Recent research only
)

3. Content Extraction for Citations

• When CoordinatorAgent identifies relevant cited papers in bibliography
• SearchAgent uses Tavily to find those papers online
• Extracts full abstracts/summaries using tavily_client.extract(urls=[...])
• Provides SynthesisAgent with cited paper context for richer answers

4. Filtering

• Only results with Tavily score > 0.6 are used
• Ensures high-quality, relevant sources
• Reduces noise from low-confidence matches

---

Frontend (frontend/)

Technology: React 19 + TypeScript + Vite

Key Components:

• PDFViewer.tsx: Renders PDF with react-pdf, shows current page
• Chatbox.tsx: Chat interface with citation display
• NavBar.tsx: Paper list navigation
• Home.tsx: Split view (PDF + Chat)

API Integration:

• Axios with 5-minute timeout for long PDF processing
• JWT authentication (access + refresh tokens)
• Real-time chat with citation rendering

---

Future Improvements

The structure of the Agents allows new changes to be easily integrated.

• Currently, the SynthesisAgent returns unformatted text, so adding a way to distinguish bullet points or lists would make it more appealing.
• The Agents log a "score" for each measure which os hard-coded for integration purposes. This could be changed to have the LLM give its confidence on a process. If the score is too low at the end of the routing, another agent could orchestrate corrections to fix the work of other agents.
• We can find better ways to cache repeated text to make querying cheaper.
• The current version does not analyze images, so this could added alongside text analysis
• Prompts to LLMs can always be improved.
• And more!