📄 PDF → JSON Extractor — ML Pipeline

Fully offline, LLM-free pipeline that accepts a PDF document (invoice, bank statement, or purchase order — including borderless tables) and extracts all content into structured JSON.

---

📋 Table of Contents

1. Overview
2. Pipeline Architecture
3. Approach
4. ML vs Rule-Based
5. Sample Files
6. Sample Output
7. Project Structure
8. Quick Start
9. GUI Usage
10. Challenges
11. Possible Improvements
12. Requirements

---

1. Overview

This project builds a fully offline, no-LLM pipeline that:

• Accepts a PDF (invoice, bank statement, or purchase order)
• Handles documents with borderless tables — tables with no visible horizontal or vertical lines
• Extracts all content into structured JSON output
• Provides a Tkinter desktop GUI (app.py) for non-technical users

All processing happens locally — no API calls, no internet connection required.

---

2. Pipeline Architecture

The pipeline (pipeline_ml.py) runs five sequential ML stages:

PDF Input
   │
   ▼
┌─────────────────────────────────────────────────────────┐
│  Stage 1 — PDF Type Detection                           │
│  Magika (Google ONNX) + text-density heuristic          │
│  → "digital" (pdfplumber) or "scanned" (Tesseract)      │
└──────────────────────────┬──────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────┐
│  Stage 2 — Word Extraction                              │
│  Digital  → pdfplumber  (exact bounding boxes)          │
│  Scanned  → Tesseract LSTM + Otsu thresholding          │
└──────────────────────────┬──────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────┐
│  Stage 3 — Layout Analysis  [DBSCAN]                    │
│  Clusters words into rows by y-coordinate proximity     │
│  eps = 55% of median character height (fully adaptive)  │
└──────────────────────────┬──────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────┐
│  Stage 4 — Region Classification  [GradientBoosting]    │
│  22-feature vector per row → region label:              │
│  HEADER / KEY_VALUE / TABLE_HEADER / TABLE_ROW /        │
│  SUMMARY / FOOTER                                       │
└──────────────────────────┬──────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────┐
│  Stage 5 — Semantic Extraction                          │
│  KEY_VALUE  → colon-anchor + token classifier           │
│  TABLE      → KDTree column assignment                  │
│  SUMMARY    → positional extraction (last token)        │
└──────────────────────────┬──────────────────────────────┘
                           │
                           ▼
                      JSON Output

---

3. Approach

Why DBSCAN for layout analysis?

Most PDF parsers assume fixed line spacing or known column counts. DBSCAN makes no such assumptions. By clustering word y-coordinates with an eps derived from the median character height at runtime, it discovers rows naturally in any document regardless of font size, line spacing, or column count. This is what allows it to handle borderless tables — without visible grid lines, the only spatial signal is word position.

Why GradientBoosting for classification?

GradientBoosting was chosen over a neural network because the feature space (22 normalised geometric + lexical features per row) is small, well-defined, and fully interpretable. It:

• Trains in milliseconds
• Requires no GPU
• Generalises well across document types when trained on real invoice/statement/PO examples
• All 22 features are document-type agnostic — nothing is hardcoded for invoices specifically

Why KDTree for table columns?

Once TABLE_HEADER rows are identified, their word x-centers define column positions. A KDTree provides O(log n) nearest-neighbour lookup to assign each TABLE_ROW word to its correct column — correctly handling variable-width columns and multi-word cells with no column-count assumptions.

---

4. Benefits of ML Approach over Rule-Based

A traditional rule-based pipeline uses hardcoded regex patterns, fixed column offsets, or known field names. This breaks the moment a document deviates from the expected template.

Capability	Rule-Based ❌	This ML Pipeline ✅
Borderless table detection	Relies on visible line coordinates in PDF	DBSCAN finds rows purely from word y-positions
New document types	Requires new rules per document template	GradientBoosting generalises across invoice, statement, PO with no code changes
Font size / spacing changes	Fixed pixel thresholds break on layout changes	eps adapts to median character height at runtime
Column count	Hardcoded per template (e.g. always 5 columns)	KDTree discovers columns from header word positions dynamically
Mixed content rows	Requires separate parsers wired together manually	Region classifier separates KEY_VALUE / TABLE_ROW / SUMMARY automatically
Multilingual / OCR content	Regex patterns are language-specific	Feature vector uses geometry + punctuation, not language-specific tokens
Maintenance	Every new template = new code + regression risk	Retrain on new examples — no code changes needed
Confidence scoring	Binary: matched or not	predict_proba() gives per-field confidence (extensible)

In plain terms

A rule-based system is a list of "If the text at position X matches pattern Y, call it field Z." Every new invoice layout needs a new rule. Every changed font size breaks column offsets. Every document without grid lines fails table detection entirely.

This pipeline instead asks: "What does a TABLE_ROW look like geometrically?" — it tends to have many words, a wide x-spread, digit-heavy content, and consistent word counts across adjacent rows. These signals hold true whether the document is an invoice, a bank statement, or a purchase order, and whether the table has borders or not.

Practical result: a single codebase that correctly extracted all 9 fields and all 4 table rows from the sample invoice without any document-specific configuration.

---

5. Sample Files Included

The repository includes a real input PDF and its corresponding JSON output so you can verify the pipeline before running it on your own documents.

File	Type	Description
sample_invoice_borderless.pdf	📥 Input	A one-page invoice with a borderless table (no visible grid lines). Tests the core challenge of the pipeline.
sample_invoice_borderless_20260217_205633.json	📤 Output	The exact JSON produced by pipeline_ml.py on the input PDF — 9 extracted fields, 1 table with 4 rows, raw text by page.

To reproduce the sample output yourself:

# CLI
python pipeline_ml.py sample_invoice_borderless.pdf

# GUI
python app.py   →  Browse PDF  →  Extract to JSON

The pipeline will print each ML stage's decisions to the console / log window.

---

6. Sample Output

Running the pipeline on sample_invoice_borderless.pdf produced:

Extracted Fields

Field	Value
invoice_number	INV-2024-12345
invoice_date	02/15/2024
due_date	03/15/2024
bill_to	Acme Corporation
subtotal	$5,610.00
tax_10	$561.00
total	$6,171.00
payment_terms	Net 30 days
contact	accounts@example.com | +1-555-0123

Extracted Table (borderless — no visible grid lines)

description	quantity	unit	price	amount
Web Development Services	40 hrs		$75.00	$3,000.00
Database Setup	10 hrs		$85.00	$850.00
API Integration	15 hrs		$80.00	$1,200.00
Testing & QA	8 hrs		$70.00	$560.00

How the borderless table was detected:

• DBSCAN separated the 5 table rows from surrounding key-value rows by y-coordinate gap
• GradientBoosting classified them as TABLE_HEADER and TABLE_ROW (not KEY_VALUE)
• KDTree assigned each word to description / quantity / unit / price / amount column
• Zero regex, zero hardcoded column names, zero knowledge of document type

Full JSON Structure

{
  "source_file": "sample_invoice_borderless.pdf",
  "pdf_type": "digital",
  "pipeline": "ML — DBSCAN layout + GradientBoosting classifier + KDTree columns",
  "total_pages": 1,
  "extracted_fields": {
    "invoice_number": "INV-2024-12345",
    "invoice_date":   "02/15/2024",
    "due_date":       "03/15/2024",
    "bill_to":        "Acme Corporation",
    "subtotal":       "$5,610.00",
    "tax_10":         "$561.00",
    "total":          "$6,171.00",
    "payment_terms":  "Net 30 days",
    "contact":        "accounts@example.com | +1-555-0123"
  },
  "tables": [
    {
      "headers":   ["description", "quantity", "unit", "price", "amount"],
      "rows": [
        { "description": "Web Development Services", "quantity": "40 hrs", "unit": "", "price": "$75.00",  "amount": "$3,000.00" },
        { "description": "Database Setup",           "quantity": "10 hrs", "unit": "", "price": "$85.00",  "amount": "$850.00"   },
        { "description": "API Integration",          "quantity": "15 hrs", "unit": "", "price": "$80.00",  "amount": "$1,200.00" },
        { "description": "Testing & QA",             "quantity": "8 hrs",  "unit": "", "price": "$70.00",  "amount": "$560.00"   }
      ],
      "row_count": 4
    }
  ],
  "raw_text_by_page": [
    {
      "page": 1,
      "text": "INVOICE Invoice Number: INV-2024-12345 Invoice Date: 02/15/2024 ..."
    }
  ]
}

---

7. Project Structure

your_project/
├── app.py                                        ← Tkinter GUI entry point
├── pipeline_ml.py                                ← ML pipeline (all 5 stages)
├── requirements.txt                              ← Python dependencies
├── README.md                                     ← This file
├── sample_invoice_borderless.pdf                 ← Sample input PDF
└── sample_invoice_borderless_20260217_205633.json ← Sample output JSON

---

8. Quick Start

1. Clone the repo

git clone https://github.com/your-username/pdf-json-extractor.git
cd pdf-json-extractor

2. Create a virtual environment (recommended)

python -m venv venv

# Windows
venv\Scripts\activate

# macOS / Linux
source venv/bin/activate

3. Install Python dependencies

pip install -r requirements.txt

4. Install Tesseract OCR (for scanned PDFs only)

OS	Command
Windows	Download from UB-Mannheim, add C:\Program Files\Tesseract-OCR to PATH
macOS	brew install tesseract
Linux	sudo apt install tesseract-ocr

Skip this step if you only process digital PDFs (text is selectable).

5. Install Poppler (required by pdf2image for scanned PDFs)

OS	Command
Windows	Download from oschwartz10612/poppler-windows, add bin/ to PATH
macOS	brew install poppler
Linux	sudo apt install poppler-utils

---

9. GUI Usage

python app.py

1. Click Browse PDF → select your invoice / statement / form PDF
2. The app auto-detects whether it's digital or scanned and shows a badge
3. Click Browse Folder → choose where to save the JSON
4. Click ▶ Extract to JSON
5. Watch the Pipeline Log — each ML stage prints its classification decisions
6. Click 🔍 View JSON to inspect the result in-app
7. Click 📂 Open Output Folder to find the saved file

💡 To add a GUI screenshot: take a screenshot of the running app, save it as screenshot.png in the project root, and it will appear here automatically on GitHub.

---

10. Challenges

Challenge	How it was addressed
Borderless table detection	DBSCAN spatial clustering + GradientBoosting region classifier — no dependency on visible grid lines at any stage
Font-size independence	All DBSCAN eps values derived from median character height at runtime; adapts to any font
"Unit" column splitting	The header Unit Price was tokenised into two columns (unit, price); the unit column correctly shows empty values since the invoice merges Unit Price into a single field
Key-value extraction	Colon-anchor strategy: if any token ends with :, everything before is the key and after is the value. Token classifier is the fallback
Mixed content per row	22-feature vector captures both positional (x_spread, rel_pos) and lexical (digit_ratio, n_currency, n_date) signals simultaneously
Scanned PDF support	Otsu thresholding (optimal binarisation threshold via inter-class variance maximisation) preprocesses images before Tesseract LSTM, filtered at confidence ≥ 30

---

11. Possible Improvements

• Train on a larger labelled dataset across more document types (receipts, W-2 forms, medical records) to improve GradientBoosting generalisation
• Replace the handcrafted 22-feature vector with a pre-trained layout model such as LayoutLMv3 or DiT for richer spatial-semantic understanding
• Add multi-page support — track running totals and key-value context across page boundaries
• Export to additional formats: CSV for tables, Excel for multi-sheet documents
• Add a confidence score per extracted field based on classifier predict_proba(), so downstream consumers know which fields to verify
• Integrate Poppler-based table detection as a secondary signal to reinforce DBSCAN row clusters on PDFs with partial visible lines
• Add a document-type pre-classifier to select specialised sub-models for invoice vs. statement vs. form

---

12. Requirements

pdfplumber>=0.10.0       # Digital PDF word bounding-box extraction
pdf2image>=1.16.0        # Converts scanned PDF pages to PIL images
pytesseract>=0.3.10      # Tesseract LSTM OCR wrapper
opencv-python>=4.8.0     # Otsu thresholding + image sharpening
Pillow>=10.0.0           # PIL image handling
scikit-learn>=1.3.0      # DBSCAN, GradientBoosting, StandardScaler
scipy>=1.11.0            # KDTree for O(log n) column assignment
numpy>=1.24.0            # Array ops, median, statistical features
magika>=0.5.0            # Google ONNX file-type classifier (optional)

Install all at once:

pip install -r requirements.txt

---

🔧 Troubleshooting

Error	Fix
ModuleNotFoundError: pipeline_ml	Make sure app.py and pipeline_ml.py are in the same folder
TesseractNotFoundError	Install Tesseract and add to PATH, or set pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe' in pipeline_ml.py
PDFPageCountError from pdf2image	Install Poppler and add its bin/ folder to PATH
magika warning on startup	Optional dependency — pipeline works without it
App freezes during extraction	Normal — ML training + OCR runs in a background thread; watch the log window
Failed loading language 'eng'	Re-run Tesseract installer, ensure English language data is checked

---

Built with pdfplumber · pytesseract · scikit-learn · scipy · OpenCV · Tkinter