PRD: NihongOCR MVP

[emcifuentesc]

NihongOCR MVP

Problem Statement

Japanese learners and readers frequently encounter text in images, screenshots, and scanned materials that they cannot read. Existing solutions either require uploading images to cloud-based OCR services (privacy risk, network dependency) or produce raw Japanese text without phonetic guidance. There is no lightweight, fully-offline macOS utility that captures a region of the screen and immediately presents both the original Japanese text and its Romaji transliteration in a readable, copyable format — including support for both modern horizontal text and traditional vertical text.

Solution

NihongOCR is a macOS menu bar app that captures images of Japanese text (via screen region selection or drag-and-drop) and produces a structured Reading — the original text with Kana readings (as furigana) and Hepburn Romaji transliteration — displayed in a small, always-on-top window. The entire pipeline runs offline: PaddleOCR extracts text, Sudachi tokenizes and extracts Kana readings, and pykakasi converts to Romaji. A bundled Python sidecar communicates with the Tauri-based frontend via stdin/stdout NDJSON.

User Stories

1. As a Japanese learner, I want to capture a region of my screen containing Japanese text, so that I can see how to read it without retyping.
2. As a Japanese learner, I want to see both the original Kanji and its Romaji transliteration, so that I can learn pronunciation while reading.
3. As a manga reader, I want to capture vertical Japanese text and see it rendered correctly, so that traditional layouts are supported.
4. As a user, I want the app to live in my menu bar and not clutter my dock, so that it feels like a native macOS utility.
5. As a user, I want to drag and drop an image onto the app, so that I can process saved images without taking a new screenshot.
6. As a user, I want to copy just the Japanese text or just the Romaji, so that I can paste into flashcards, dictionaries, or chat apps.
7. As a privacy-conscious user, I want all processing to happen locally on my machine, so that my screenshots never leave my computer.
8. As a user, I want the app to load quickly and show me when it is ready, so that I know when I can start capturing.
9. As a user, I want low-confidence OCR results to be visually indicated, so that I know which words might be wrong.
10. As a user, I want to toggle between horizontal and vertical orientation if the app guesses wrong, so that I can correct the layout.
11. As a user, I want the Reading window to stay on top of other windows, so that I can reference it while typing in another app.
12. As a user, I want the Reading window to appear centered on screen after a capture, so that my attention is already where the result is.
13. As a user, I want the original line structure from the image to be preserved, so that multi-line text remains readable.
14. As a user, I want consecutive lines to have alternating background colors, so that I can visually track which Romaji belongs to which Japanese line.
15. As a user, I want all text in the Reading window to be selectable, so that I can highlight and inspect it even if I do not use the copy buttons.
16. As a user, I want temp image files to be deleted immediately after processing, so that screenshots do not accumulate on disk.
17. As a user, I want the app to handle captures with no text gracefully, so that I know the capture worked but contained nothing.
18. As a user, I want to close the Reading window and return to a clean menu-bar-only state, so that the app does not clutter my screen.
19. As a user, I want a new capture to replace the current Reading, so that I always see the most recent result.
20. As a user with limited Japanese knowledge, I want to see Kana readings above Kanji (or to the right in vertical mode), so that I can sound out words I do not know.

Implementation Decisions

Modules

The MVP consists of four deep modules:

1. OCR Worker (Python sidecar)

   • Encapsulates the entire NLP pipeline: image preprocessing, PaddleOCR text detection, Sudachi tokenization and reading extraction, pykakasi Romaji conversion, and JSON serialization.
   • Interface: receives an image file path via stdin (NDJSON request), writes a JSON Reading to stdout (NDJSON response).
   • Loads PaddleOCR models and Sudachi dictionary once at startup and keeps them in memory.
   • Returns a Reading object with orientation (default horizontal), lines, and tokens.

2. Sidecar Manager (Rust / Tauri backend)

   • Encapsulates Python process lifecycle: spawn, health monitoring, crash recovery, and shutdown.
   • Interface: start_warmup() (triggers background model loading), process_capture(path: &Path) -> Result<Reading, SidecarError>.
   • Handles NDJSON framing (newline-delimited request/response pairs) over stdin/stdout.
   • On crash, attempts one automatic restart and re-shows the splash screen.

3. Capture Service (Rust / Tauri backend)

   • Encapsulates all input methods: menu-bar-triggered region capture (via screencapture -i), drag-and-drop file handling, and temp file lifecycle.
   • Interface: capture_region() -> Result<PathBuf, CaptureError> and receive_drop(path: PathBuf) -> Result<PathBuf, CaptureError>.
   • Writes captured images to a temp directory and returns the path.
   • Triggers cleanup of the temp file after the Sidecar Manager confirms processing is complete.

4. Reading Engine (TypeScript / frontend)

   • Encapsulates all rendering logic: horizontal vs vertical layout, furigana (<ruby>) placement, Romaji row positioning, alternating line colors, confidence underlines, and copy-button behavior.
   • Interface: render_reading(reading: Reading) -> void, copy_japanese(), copy_romaji().
   • Renders vertical text with CSS writing-mode: vertical-rl, furigana to the right, and Romaji horizontally beneath each column.
   • Provides an orientation toggle that re-renders the current Reading with flipped layout (does not re-run OCR).

Technical Clarifications

• The Python worker outputs a Reading schema where each Line preserves OCR-detected boundaries. Each Token carries surface, kana, and romaji. Punctuation is included as tokens with kana and romaji mirroring surface.
• The app bundles the entire Python runtime, OCR models, and Sudachi dictionary inside the .app bundle via a Python freezing tool (e.g., PyInstaller). No network is required at runtime.
• Confidence scores from PaddleOCR are passed through to the frontend. Any token with confidence below 0.7 receives a subtle orange underline.
• The Reading window is a fixed-size, always-on-top Tauri webview window. It opens centered and closes to menu-bar-only state.

API Contracts

• Python stdin protocol: Single-line JSON object {"type": "ocr", "image_path": "/tmp/..."} followed by newline.
• Python stdout protocol: Single-line JSON Reading object followed by newline.
• Rust → TypeScript: Tauri commands process_capture(path) and get_sidecar_status().

Testing Decisions

• OCR Worker: Integration tests using a small suite of sample images (horizontal Japanese, vertical Japanese, mixed text, empty image, low-confidence image). Tests verify the shape of the output JSON and approximate correctness of tokenization — exact token boundaries may vary by Sudachi version, so assertions should be on presence of key tokens rather than exact arrays.
• Sidecar Manager: Unit tests with a mock Python process (a shell script that echoes canned responses). Tests verify NDJSON framing, request/response pairing, and crash-restart behavior.
• Capture Service: Manual testing only. Automated testing of screencapture and drag-and-drop is prohibitively platform-specific and brittle.
• Reading Engine: Component tests using mock Reading data. Tests verify DOM structure for horizontal and vertical layouts, correct furigana placement, and that copy buttons extract the expected plain text.
• Good tests exercise external behavior (given this input, the user sees that output) and avoid asserting on internal DOM class names or implementation-specific structures.

Out of Scope

• History or persistence of past Readings.
• Settings panel or user preferences.
• Global keyboard shortcuts.
• OCR bounding box overlay on the source image.
• Part-of-speech tagging display.
• Auto-detection of text orientation as the sole source of truth (user toggle is MVP; auto-detection may be enhanced later).
• Non-macOS platforms.
• Cloud-based OCR or LLM-based correction.
• Export to file formats (PDF, TXT, etc.).

Further Notes

• The app name is NihongOCR.
• The domain glossary and architectural decisions are recorded in CONTEXT.md and docs/adr/.
• Temp files are cleaned up immediately after OCR to protect user privacy.
• The vertical text toggle is a display-only feature; it does not re-run OCR or re-tokenize.
• Future enhancements may include: history with SQLite storage, global hotkeys, confidence score display as numbers, and auto-orientation detection.