TLDR

TLDR is a web app for browsing, searching, reading, and annotating academic PDFs.

This repository contains the running backend service (Fastify + SQLite) and the web UI served by that backend.

What It Does

• Split-view reading UI: paper list on the left, PDF reader on the right.
• Search papers by title, author, or keyword.
• Restore reading position (page + normalized scroll).
• Stream PDFs with HTTP range support.
• Create highlights and notes.
• Ask AI questions about a highlight and keep the conversation history.
• Delete highlights and delete papers (including associated notes/highlights and on-disk PDF file).
• Import PDFs from URLs.

Metadata Pipeline

When a paper is added/imported, TLDR extracts metadata using a layered approach:

1. GROBID header parsing (if GROBID_URL is configured).
2. Embedded PDF metadata (dc:title, dc:creator, etc.).
3. First-page text/layout heuristics.
4. Filename fallback.

This improves title/author quality across mixed PDF sources.

Gemini Files Integration

TLDR can upload PDFs to Gemini Files and store gemini_file_id per paper.

• POST /api/highlights/:highlightId/ask-ai uses the stored Gemini file ID plus highlight context.
• Background backfill periodically uploads missing/stale papers when GEMINI_API_KEY is present.
• Use npm run gemini:upload to backfill existing records manually.

Requirements

• Node.js (LTS recommended)
• SQLite (embedded via better-sqlite3)
• Optional: GROBID service (for higher-quality metadata extraction)
• Optional: Gemini API key (for AI and Gemini Files)

Environment Variables

• TLDR_SERVICE_ROOT: service root; also used to find .env.
• TLDR_DB_PATH: SQLite DB file path (example: ./data/tldr.sqlite).
• TLDR_DOCUMENT_ROOT: root folder used to resolve stored pdf_path values.
• TLDR_PORT: API server port (default: 3000).
• TLDR_HOST: bind address (default: 0.0.0.0).
• TLDR_DEFAULT_USER_NAME: display name in notes UI.
• GROBID_URL: GROBID endpoint (example: http://localhost:8070).
• GEMINI_API_KEY: Gemini API key.
• GEMINI_MODEL: Gemini model name (default: gemini-2.5-flash).
• GEMINI_BACKFILL_INTERVAL_MINUTES: background upload interval.

Use .env.example as the baseline local config:

TLDR_SERVICE_ROOT=.
TLDR_DB_PATH=./data/tldr.sqlite
TLDR_DOCUMENT_ROOT=./data/papers

Local Development

npm install
npm run dev

Build

npm run build

Deploy

export TLDR_SERVICE_ROOT=/path/to/tldr-service
./deploy.sh

On first deploy (or when dependencies change), deploy.sh runs npm ci --omit=dev in the service root.

Run (Deployed)

cd "$TLDR_SERVICE_ROOT"
node dist/server.js

The server loads .env from:

1. $TLDR_SERVICE_ROOT/.env (if TLDR_SERVICE_ROOT is set)
2. ./.env (current working directory)
3. module root .env

Scripts

• npm run dev: run server in watch mode.
• npm run build: compile TypeScript to dist/.
• npm run start: run compiled server.
• npm run reindex:metadata: re-extract and refresh metadata.
• npm run gemini:upload: upload PDFs and populate gemini_file_id.
• npm run notes:clear: clear stored notes/highlights data.
• npm run migrate:gemini-id: run Gemini ID migration script.

API Endpoints

• GET /api/papers?q=...&sort=last_viewed: list/search papers.
• GET /api/papers/:paperId: paper details + reading position.
• PUT /api/papers/:paperId/reading-position: update reading position.
• POST /api/papers: add a paper from an existing relative pdfPath.
• POST /api/papers/import: download/import a paper from URL.
• DELETE /api/papers/:paperId: delete paper + related data + PDF file.
• GET /api/papers/:paperId/file: stream PDF.
• GET /api/papers/:paperId/highlights: list highlights for a paper.
• POST /api/papers/:paperId/highlights: create a highlight.
• GET /api/highlights/:highlightId: get one highlight.
• DELETE /api/highlights/:highlightId: delete one highlight.
• POST /api/highlights/:highlightId/notes: add note to highlight.
• POST /api/highlights/:highlightId/ask-ai: ask AI using highlight + Gemini file.

Notes

• pdf_path values are resolved relative to TLDR_DOCUMENT_ROOT.
• last_scroll_y is normalized to 0..1 within the last viewed page.