AI-Powered Scientific Literature Lifecycle Management

中文 · Documentation · Quick Start · Report Bug

---

Omelette is a full-stack application for managing the complete scientific literature lifecycle. Search across academic databases, deduplicate results, download and OCR papers, build RAG-powered knowledge bases, and interact with your literature through a ChatGPT-style conversational interface.

Om (Omni-) + Lit (Literature) = Omlit ≈ Omelette 🍳

Features

Literature Pipeline

• Keyword Management — Three-level hierarchy with LLM-powered term expansion and search formula generation for WOS, Scopus, PubMed
• Multi-Source Search — Federated search across Semantic Scholar, OpenAlex, arXiv, and Crossref
• Smart Deduplication — Three-stage pipeline: DOI exact match → title similarity → LLM verification
• PDF Crawler — Multi-channel download via Unpaywall, arXiv, and direct URL fallback
• OCR Processing — MinerU (auto-managed) + pdfplumber + PaddleOCR for scanned PDFs
• Incremental Subscription — RSS and API-based scheduled updates

AI & Knowledge Management

• RAG Knowledge Base — LlamaIndex + ChromaDB with GPU-aware embeddings, hybrid retrieval, and cited answers
• Chat Playground — ChatGPT-style conversational interface for literature Q&A
• Multi-Provider LLM — LangChain integration for OpenAI, Anthropic, Aliyun, Volcengine, and Ollama
• LangGraph Pipeline — StateGraph-based orchestration with HITL interrupt/resume

Research Tools

• Audio Overviews — LLM-generated dialogue audio for paper collections
• Citation Tools — APA, MLA, Chicago, IEEE, GB/T 7714 styles with bibliography builder
• Author Network — d3-force directed graph of co-authorship relationships
• Trend Analysis — Year-binned topic trends with emerging/declining detection
• Gap Analysis — LLM-powered research gap identification with novelty scoring
• Version Tracking — Paper version history with Semantic Scholar polling
• Paper Comparison — Side-by-side comparison with diff highlighting

Collaboration

• Team Members — Project team management with RBAC (read/write/admin)
• API Keys — SHA-256 hashed keys with scope-based access control
• Collection Management — Custom paper collections with AI-suggested tags
• MCP Integration — Model Context Protocol server for AI IDE clients

User Experience

• i18n — Full Chinese/English bilingual support
• PWA — Installable with offline caching
• Responsive — Mobile-optimized with horizontal scroll navigation
• Dark Mode — System-aware theme

Architecture

┌─────────────────────────────────────────────────────────┐
│                      Frontend (React 18)                 │
│  TypeScript · Vite · TailwindCSS · shadcn/ui · TanStack │
├─────────────────────────────────────────────────────────┤
│                    Backend (FastAPI)                     │
│  Python 3.12 · SQLAlchemy 2 (async) · Pydantic v2       │
├──────────┬──────────┬──────────┬──────────┬────────────┤
│  Search  │  Dedup   │  Crawler │   OCR    │    RAG     │
│ LangChain│ Semantic │ Unpaywall│  MinerU  │ LlamaIndex │
│  OpenAlex│ Scholar  │  arXiv   │PaddleOCR │  ChromaDB  │
├──────────┴──────────┴──────────┴──────────┴────────────┤
│                   Storage: SQLite + ChromaDB             │
└─────────────────────────────────────────────────────────┘

Layer	Technology
Frontend	React 18, TypeScript, Vite, TailwindCSS v4, shadcn/ui, TanStack Query
Backend	FastAPI, SQLAlchemy 2 (async), Pydantic v2, Python 3.12
Database	SQLite + aiosqlite, Alembic migrations
Vector Store	ChromaDB
RAG	LlamaIndex with GPU-aware BAAI/bge-m3 embeddings
LLM	LangChain (OpenAI, Anthropic, Aliyun Bailian, Volcengine, Ollama)
Orchestration	LangGraph with HITL interrupt/resume
OCR	MinerU (auto-managed) + pdfplumber (native) + PaddleOCR (scanned)
Docs	VitePress (bilingual EN/ZH)

Quick Start

Prerequisites

• Conda or Miniconda
• Node.js 22+
• (Optional) CUDA for GPU-accelerated OCR and embeddings

Setup

git clone git@github.com:sylvanding/omelette.git
cd omelette

# Backend
conda env create -f environment.yml
conda activate omelette
cp .env.example .env  # Edit with your API keys
cd backend && alembic upgrade head
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Frontend (new terminal)
cd frontend && npm install
npm run dev -- --port 3000

Open http://localhost:3000.

Key Environment Variables

Variable	Description
DATABASE_URL	SQLite path (default: sqlite:///./data/omelette.db)
DATA_DIR	Base path for PDFs, OCR output, ChromaDB
LLM_PROVIDER	openai, anthropic, aliyun, volcengine, ollama, or mock
OPENAI_API_KEY	OpenAI API key
ANTHROPIC_API_KEY	Anthropic API key
PDF_PARSER	auto, mineru, or pdfplumber
GPU_MODE	conservative, balanced (default), aggressive

See .env.example for the full list.

Development

# Frontend
cd frontend
npm test                 # 273 tests (Vitest + Testing Library)
npx tsc --noEmit         # TypeScript check
npx eslint src/          # Lint

# Backend
cd backend
pytest tests/ -v         # 861 tests (pytest-asyncio)
ruff check app/          # Lint

# E2E
npx playwright test      # 39 tests (requires frontend dev server)

API Overview

Base URL: /api/v1

Endpoint	Description
GET/POST /projects	Project CRUD
GET/POST /projects/{id}/papers	Paper management with search, filter, sort
GET/POST /projects/{id}/keywords	Keyword hierarchy
POST /projects/{id}/search/execute	Federated academic search
POST /projects/{id}/dedup/run	Run deduplication
POST /projects/{id}/crawl/start	Start PDF download
POST /projects/{id}/ocr/process	Run OCR on papers
POST /projects/{id}/rag/index	Build vector index
POST /projects/{id}/rag/query	RAG retrieval
POST /chat	Chat messages (playground)
GET/POST /projects/{id}/collections	Paper collections
GET/POST /subscriptions	Subscription management
GET /projects/{id}/analytics	Reading analytics
POST /projects/{id}/concepts/extract	Concept extraction
POST /projects/{id}/analysis/contradictions	Contradiction detection

Full documentation: API Reference

Project Structure

omelette/
├── backend/               # FastAPI application
│   ├── app/
│   │   ├── api/v1/        # REST endpoints (33 modules)
│   │   ├── models/        # SQLAlchemy ORM models
│   │   ├── schemas/       # Pydantic request/response schemas
│   │   ├── services/      # Business logic + LLM services
│   │   └── pipelines/     # LangGraph pipeline definitions
│   ├── alembic/           # Database migrations
│   ├── mcp_server.py      # MCP (Model Context Protocol) server
│   └── tests/             # pytest-asyncio tests (861 tests)
├── frontend/              # React SPA
│   └── src/
│       ├── pages/         # Route components (35 project pages)
│       ├── components/    # Reusable UI + layout
│       ├── services/      # Typed API client
│       ├── hooks/         # Custom hooks
│       ├── i18n/          # zh/en translations
│       └── test/          # Vitest setup, MSW mocks
├── e2e/                   # Playwright E2E tests (39 tests)
├── docs/                  # VitePress documentation (EN/ZH)
├── scripts/ralph/         # Ralph autonomous agent workflow
└── .github/workflows/     # CI (ruff, pytest, vitest, tsc)

Contributing

See CONTRIBUTING.md for guidelines.

License

MIT License — Copyright © 2026 Sylvan Ding