CyberMD

CyberMD is a flexible tool for converting various document formats into clean Markdown, supporting both OCR-based extraction (for PDFs and images) and direct conversion (for DOCX, PPTX, XLSX). The project includes a Streamlit web interface for interactive use, a CLI script for batch processing, and a REST API for programmatic access.

Features

• Multi-format Support: Convert PDF, JPG, PNG, DOCX, PPTX, XLS, XLSX to Markdown
• OCR Processing: Extract text and structure from scanned documents and images using Rednote DotsOCR
• Direct Conversion: Use MarkItDown for accurate conversion of Office files without OCR overhead
• Web Interface: Interactive UI with file upload, progress tracking, and download capability
• REST API: Programmatic access via HTTP API with dynamic API key authentication
• CLI Mode: Batch process files from an input directory using environment configuration
• Cache Management: Temporary file management with automatic cleanup functionality

Supported File Types

Type	Extensions	Processor
OCR Input	.pdf, .jpg, .jpeg, .png	OCRProcessor
Office	.docx, .pptx, .xls, .xlsx	MarkItDownProcessor

Quick Start

1. Installation

Using uv (recommended):

uv sync --python 3.12

Project structure:

project-root/
├── main.py              # Streamlit web interface
├── cli.py               # CLI batch processor
├── endpoint-api.py      # FastAPI REST server
├── tools/
│   ├── ocr_processor.py
│   ├── markitdown_process.py
│   └── dots_ocr/
├── input/
├── output/
├── api_upload/
├── api_output/
├── api_cache/
└── .env (optional)

2. Run the Web App

Launch the Streamlit interface:

uv run streamlit run main.py

Upload a document, click "Start Processing", and download the resulting Markdown.

3. Use CLI for Batch Processing

Place files in the input/ directory and run:

uv run cli.py

Processed Markdown files will appear in the output/ directory.

Note: You can customize input/output paths via .env file (see Configuration).

4. Start the REST API Server

Launch the FastAPI server:

uv run uvicorn endpoint-api:app --host 0.0.0.0 --port 8000

On startup, the API key will be printed and saved to .api_key file.

API Usage Example

# Get API key
curl http://localhost:8000/api-key

# Convert a document via upload
curl -X POST "http://localhost:8000/convert" \
  -H "X-API-Key: your_api_key_here" \
  -F "file=@document.pdf"

# Convert from server file path
curl -X POST "http://localhost:8000/convert/file-path" \
  -H "X-API-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{"file_path": "./input/document.pdf", "processor_type": "auto"}'

# Health check
curl -H "X-API-Key: your_api_key_here" http://localhost:8000/health

5. Use Obsidian Plugin

Install the plugin in your Obsidian vault:

1. Copy obsidian-plugin/ folder to your-vault/.obsidian/plugins/cybermd-converter/
2. Enable the plugin in Obsidian settings
3. Configure API URL and API Key in plugin settings
4. Click the ribbon icon to open the converter sidebar

See obsidian-plugin/README.md for detailed instructions.

Configuration

Create a .env file to override default settings:

INPUT_DIR=./input
OUTPUT_DIR=./output
API_PORT=8000

In main.py, you can also adjust these global settings:

NO_IMAGE = True      # Skip image output in OCR
NO_TABLE = False     # Enable table extraction in OCR

Processors

OCRProcessor

Handles scanned documents and images using OCR. Outputs structured Markdown with layout preservation.

• Input: PDF, JPG, JPEG, PNG
• Engine: Rednote DotsOCR
• Configuration: no_image, no_table, delete_cache

MarkItDownProcessor

Converts Office files directly to Markdown, preserving headings, lists, tables, and formatting.

• Input: DOCX, PPTX, XLS, XLSX
• Engine: Microsoft MarkItDown

Directory Structure

• input/: Upload or place files here for CLI batch processing
• output/: Final Markdown results are stored here (CLI mode)
• api_upload/: Temporary upload directory for API server
• api_output/: Output directory for API server
• api_cache/: Cache directory for API server OCR processing

Cleaning Up

Use the "Clear Cache and Output" button in the Streamlit sidebar to remove all temporary and output files.

Alternatively, manually delete the cache and output directories:

rm -rf input/* output/* api_upload/* api_output/* api_cache/*

Customization

To modify behavior:

• Edit get_processors() in main.py to change OCR or MarkItDown settings
• Update file extension lists (OCR_EXTENSIONS, MID_EXTENSIONS) to support more types
• Extend cli.py to handle additional formats or logging
• Modify endpoint-api.py to add new API endpoints

API Reference

Endpoints

Endpoint	Method	Description
/health	GET	Health check
/api-key	GET	Get current API key
/convert	POST	Upload and convert a document
/convert/file-path	POST	Convert a server-side file
/download/{filename}	GET	Download a converted file
/info	GET	Get server information

Authentication

All authenticated endpoints require the X-API-Key header:

curl -H "X-API-Key: cybermd_xxxxxxxx" http://localhost:8000/health

Development

Dependencies

Key dependencies (see pyproject.toml):

• fastapi + uvicorn - API server
• streamlit - Web UI
• markitdown - Office file conversion
• openai - OCR API client
• python-multipart - File uploads

Running Tests

# Place test files in input/ directory
uv run cli.py

License

This project is open-source. See the LICENSE file for details.