Add offline Chinese->English translation support and improved inplace layout#241
Add offline Chinese->English translation support and improved inplace layout#241itrejomx wants to merge 7 commits into
Conversation
Greptile SummaryThis PR adds Chinese→English offline translation (RapidOCR + OPUS-MT zh-en) alongside the existing Japanese pipeline, introduces a language selector in the GUI, and refactors the inplace overlay to stack overlapping labels for vertical reading flow.
Confidence Score: 3/5Not safe to merge as-is: the inplace overlay label-position bug will visibly swap translated text across regions whenever more than one caption is on screen. One confirmed P1 (label/position index mismatch after internal sort) pulls the score below the P1 ceiling of 4. The P2 findings (CUDA leak, warnings scope) are non-blocking but worth fixing before shipping. src/interpreter/overlay/base.py (P1 label mismatch), src/interpreter/translate.py (P2 CUDA/warnings scope) Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Screen Capture] --> B{source_language}
B -->|JAPANESE| C[JapaneseOCR / MeikiOCR]
B -->|CHINESE| D[ChineseOCR / RapidOCR]
C --> E[OCR regions list]
D --> E
E --> F{contains_japanese / contains_han filter}
F -->|pass| G{Translator wrapper}
F -->|skip| H[emit empty]
G -->|JAPANESE| I[JapaneseTranslator / Sugoi V4]
G -->|CHINESE| J[ChineseTranslator / OPUS-MT zh-en]
I --> K[translated regions]
J --> K
K --> L{overlay_mode}
L -->|BANNER| M[BannerOverlay]
L -->|INPLACE| N[arrange_overlay_regions sort by -x,y]
N --> O[zip(label_entries arranged) order mismatch]
O --> P[InplaceOverlay QLabels]
Reviews (2): Last reviewed commit: "chore: quiet Chinese model startup warni..." | Re-trigger Greptile |
| failed = self._process_worker.get_failed_models() | ||
| if "ocr" in failed: | ||
| if "ocr" in failed and self._source_language == SourceLanguage.JAPANESE: | ||
| delete_model_cache("rtr46/meiki.text.detect.v0") | ||
| delete_model_cache("rtr46/meiki.txt.recognition.v0") | ||
| self._fixing_ocr = True |
There was a problem hiding this comment.
_fixing_ocr never set for Chinese OCR failures
When OCR fails in Chinese mode and the user clicks "Fix Models", _fixing_ocr remains False because there is no branch for SourceLanguage.CHINESE. The worker restarts and the ocr_status signal fires "loading", but the status label path that shows "Downloading..." (guarded by _fixing_ocr) is skipped, so the label jumps directly from its error state to "Loading...". Setting self._fixing_ocr = True when Chinese OCR is in the failed set would keep the labels consistent.
| from pathlib import Path | ||
|
|
||
| ROOT = Path(__file__).resolve().parents[1] | ||
| SRC = ROOT / "src" |
| content_offset=content_offset, | ||
| ) | ||
|
|
||
| for (label, _, _), layout in zip(label_entries, arranged, strict=False): |
There was a problem hiding this comment.
Label-to-position mismatch due to internal sort in
arrange_overlay_regions
arrange_overlay_regions sorts the input regions list internally by (-x, y) before computing positions, so the returned arranged list is in a different order than label_entries. The zip(label_entries, arranged) on line 411 pairs by index, meaning label for region A ends up receiving the position computed for region B.
Concrete example: two regions at x=100 and x=200 — label_entries keeps the original order [A(x=100), B(x=200)], but arranged is [B_layout, A_layout] (sorted by -x). The zip assigns label_A.move(B_layout) and label_B.move(A_layout), swapping both overlays on screen.
Fix: match by text field rather than relying on positional alignment:
layout_by_text = {item["text"]: item for item in arranged}
for label, text, _ in label_entries:
if text in layout_by_text:
layout = layout_by_text[text]
label.move(layout["x"], layout["y"])
label.show()
self._labels.append(label)
Summary
Testing