markitdown-local

A collection of locally-running tools that extend MarkItDown with offline OCR (via a local VLM) and offline audio transcription (via whisperfile), keeping all data on your machine with no cloud API calls.

---

Tools

Script	Launcher	Purpose
markitdown-ocr.py	markitdown-ocr.bat	Convert PDFs to Markdown using a local Vision-Language Model
markitdown-audio.py	markitdown-audio.bat	Transcribe audio files to Markdown using local Whisper
llamafiles.bat	—	Start the local llama.cpp VLM server (MiniCPM-V)

---

Requirements

• Python ≥ 3.10
• uv — used by the .bat launchers to run scripts with inline dependencies (no manual pip install needed)
• llamafile — for OCR / VLM inference (llamafile.exe on PATH)
• whisperfile — for audio transcription (whisperfile.exe next to the script or on PATH)

Model Files

Place model files in a models/ subfolder next to the scripts:

markitdown/
├── models/
│   ├── MiniCPM-V-4_5-Q4_K_M.gguf   ← VLM weights (for OCR)
│   ├── mmproj-model-f16.gguf         ← VLM multimodal projector
│   └── ggml-base.en.bin              ← Whisper model (auto-downloaded if missing)
├── llamafiles.bat
├── markitdown-audio.bat
├── markitdown-audio.py
├── markitdown-ocr.bat
└── markitdown-ocr.py

The Whisper base model (ggml-base.en.bin, ~142 MB) is downloaded automatically from Hugging Face the first time you run markitdown-audio if it is not already present.

---

Usage

OCR — PDF to Markdown

Start the VLM server first:

llamafiles.bat

This starts a local llamafile server on port 8080 using MiniCPM-V.

Then run OCR in a new terminal:

markitdown-ocr.bat input.pdf
markitdown-ocr.bat input.pdf -o result.md
markitdown-ocr.bat input.pdf -o result.md --port 8080

Argument	Default	Description
input	(required)	Path to the input PDF
-o, --output	output.md	Output Markdown filename
--port	8080	Port the llama.cpp server is listening on

---

Audio Transcription — Audio to Markdown

markitdown-audio.bat recording.wav
markitdown-audio.bat podcast.mp3 -o transcript.md
markitdown-audio.bat interview.flac -o notes.md -t 8 -l auto

Supported formats: .wav, .mp3, .ogg, .flac

Argument	Default	Description
input	(required)	Path to the input audio file
-o, --output	output.md	Output Markdown filename
-m, --model	models/ggml-base.en.bin	Path to a custom Whisper model
-t, --threads	4	Number of CPU threads to use
-ng, --no-gpu	off	Disable GPU acceleration
-l, --language	en	Language code, or auto for auto-detection

---

How It Works

OCR Pipeline

PDF → MarkItDown (markitdown-ocr plugin) → local llamafile VLM → Markdown

markitdown-ocr.py initialises MarkItDown with enable_plugins=True and points its OpenAI-compatible client at http://localhost:{port}/v1. The markitdown-ocr plugin renders each PDF page as an image and sends it to the VLM for extraction.

Audio Pipeline

Audio file → whisperfile.exe → transcript text → MarkItDown wrapper → Markdown

markitdown-audio.py registers a custom LocalAudioConverter with MarkItDown. When an audio file is passed, the converter writes it to a temporary file, calls whisperfile.exe via subprocess, and wraps the transcript in a simple Markdown document.

---

Python Dependencies

Dependencies are declared inline (PEP 723) and installed automatically by uv:

Script	Dependencies
markitdown-audio.py	markitdown
markitdown-ocr.py	markitdown, markitdown-ocr, openai, Pillow

---

Tips

• Run llamafiles.bat and keep that window open before using the OCR tool.
• Use -t to increase thread count on machines with many CPU cores for faster transcription.
• Use -l auto for non-English audio; use an explicit language code (e.g., -l fr) for best accuracy on known languages.
• Both tools output plain UTF-8 Markdown that can be opened in any editor or piped into further processing.