Skip to content

feat: store msgctxt from Gettext (.po) files - #3459

Closed
Anty0 wants to merge 1 commit into
mainfrom
jirikuchynka/po-msgctxt
Closed

feat: store msgctxt from Gettext (.po) files#3459
Anty0 wants to merge 1 commit into
mainfrom
jirikuchynka/po-msgctxt

Conversation

@Anty0

@Anty0 Anty0 commented Feb 6, 2026

Copy link
Copy Markdown
Member

Summary

  • Parse and store msgctxt from PO files instead of ignoring it with a warning
  • Combine msgctxt + \u0004 (EOT) separator + msgid as the key name, following the GNU gettext standard
  • Store raw msgctxt in key custom metadata (_poFileMsgCtxt)
  • Export msgctxt line before msgid when the custom value is present
  • Display msgctxt visually in the translations UI with a dimmed prefix and | separator

Closes #3053

Summary by CodeRabbit

New Features

  • Added full support for PO file message contexts (msgctxt). Translations with context information can now be properly imported, stored, and exported while maintaining all context metadata
  • Translation keys with message context are now visually distinguished in the interface, making it easier to identify and manage contextual translations

Import msgctxt as part of the key name using the GNU gettext standard
EOT separator (\u0004), store the raw value in custom key metadata for
reliable round-trip export. Display context visually in the translations
UI with a pipe separator.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Feb 6, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR implements support for storing and handling the msgctxt (message context) field from Gettext (.po) files. Changes span the entire PO format lifecycle: parser now captures msgctxt instead of treating it as unsupported, the processor combines it with msgid as the key and stores context as metadata, the exporter reconstructs msgctxt on output, and the frontend displays context distinctly.

Changes

Cohort / File(s) Summary
PO Format Constants
backend/data/src/main/kotlin/io/tolgee/formats/po/contsants.kt
Added two constants: PO_FILE_MSG_CTXT_CUSTOM_KEY to identify msgctxt metadata and PO_MSGCTXT_SEPARATOR (Unicode U+0004) to delimit msgctxt from msgid in combined keys.
PO Parsing
backend/data/src/main/kotlin/io/tolgee/formats/po/in/PoParser.kt
Replaced error emission for msgctxt with flag-based handling; now collects msgctxt content via expectMsgCtxt state flag instead of rejecting it as unsupported.
Translation Data Model
backend/data/src/main/kotlin/io/tolgee/formats/po/in/data/PoParsedTranslation.kt
Added msgctxt StringBuilder property to store message context during parsing.
PO Processing
backend/data/src/main/kotlin/io/tolgee/formats/po/in/PoFileProcessor.kt
Introduced buildKeyName() helper to combine msgctxt and msgid with separator, and storeMsgCtxt() to persist context as custom metadata; applied to both singular and plural translation paths.
PO Exporting
backend/data/src/main/kotlin/io/tolgee/formats/po/out/PoFileExporter.kt
Added logic to extract msgctxt from combined key, write msgctxt field to file, and use actual msgid (with separator stripped) for msgid and plural forms; introduced writeMsgCtxt() helper and updated writeMsgIdPlural() signature.
Frontend Display
webapp/src/views/projects/translations/CellKey.tsx
Replaced direct keyName rendering with KeyNameWithContext component that splits on separator and displays msgctxt and msgid as separate styled elements.
Parser Tests
backend/data/src/test/kotlin/io/tolgee/unit/formats/po/in/PoParserTest.kt
Added test verifying correct parsing of msgctxt, msgid, msgstr, and plural forms across contextual and non-contextual entries.
Processor Tests
backend/data/src/test/kotlin/io/tolgee/unit/formats/po/in/PoFileProcessorTest.kt
Added tests validating msgctxt handling: distinct key generation per context, correct metadata storage, and preservation of non-contexted keys.
Exporter Tests
backend/data/src/test/kotlin/io/tolgee/unit/formats/po/out/PoFileExporterTest.kt
Added tests for msgctxt export with simple and plural forms, using helper exporters and constants to validate output structure.
Test Resource
backend/data/src/test/resources/import/po/example_msgctxt.po
Added German PO translation file with msgctxt examples: menu (Open/Close), dialog (Open), stats (plural), and uncontexted Hello entry.

Sequence Diagram(s)

sequenceDiagram
    participant PO as PO File
    participant Parser as PoParser
    participant Model as PoParsedTranslation
    participant Processor as PoFileProcessor
    participant Storage as Translation<br/>Storage
    
    PO->>Parser: Read msgctxt field
    Parser->>Parser: Set expectMsgCtxt flag
    Parser->>Model: Append to msgctxt StringBuilder
    PO->>Parser: Read msgid field
    Parser->>Model: Store msgid
    PO->>Parser: Read msgstr field
    Parser->>Model: Store msgstr
    Processor->>Model: Retrieve msgctxt & msgid
    Processor->>Processor: buildKeyName()<br/>Result: msgctxt\u0004msgid
    Processor->>Storage: Store key with<br/>msgctxt metadata
    Storage->>Storage: msgctxt field =<br/>PO_FILE_MSG_CTXT_CUSTOM_KEY
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • JanCizmar

🐰 Hopping through gettext fields so spry,
msgctxt now captured, no more goodbye,
Context and messages in harmony blend,
With separator magic, confusion will end! ✨🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: store msgctxt from Gettext (.po) files' accurately describes the main change: adding support for parsing and storing msgctxt from PO files.
Linked Issues check ✅ Passed All coding objectives from issue #3053 are met: msgctxt is imported using key scheme, custom metadata stores raw msgctxt for round-trip export, UI displays msgctxt with context separator, and comprehensive tests validate the implementation.
Out of Scope Changes check ✅ Passed All changes are directly related to msgctxt support: parser/exporter modifications, constant definitions, test resources, and minimal UI display logic for the context separator.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jirikuchynka/po-msgctxt

Comment @coderabbitai help to get the list of available commands and usage tips.

@Anty0
Anty0 marked this pull request as draft February 6, 2026 10:56
@JanCizmar

Copy link
Copy Markdown
Member

Q: Why do you store it also in the meta field?

@github-actions

github-actions Bot commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added stale and removed stale labels Mar 9, 2026
@sybbear

sybbear commented Mar 23, 2026

Copy link
Copy Markdown

Are we able to progress with this? Very important feature and actually a blocker for those who use contexts

@Anty0

Anty0 commented Mar 25, 2026

Copy link
Copy Markdown
Member Author

Hi, @sybbear! ^^
Sorry, I didn't have enough time to finish this during the last cooldown period. It's one of those smaller tasks we handle between main development cycles. I'll try to look at it during the next cooldown period—either this week or in a little over a month. I had to spend most of this cooldown on work that overflowed from the development cycle, so I won't have much time for small tasks this cycle. Sorry about that!

@sybbear

sybbear commented Mar 30, 2026

Copy link
Copy Markdown

@Anty0 Ok, hopefully it will be implemented soon :)
Meanwhile namespaces (domain+context), seem to do the work

@github-actions

Copy link
Copy Markdown
Contributor

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale label Apr 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR was closed because it has been inactive for 14 days since being marked as stale.

@Anty0

Anty0 commented May 26, 2026

Copy link
Copy Markdown
Member Author

Superseeded by #3694

@Anty0 Anty0 closed this May 26, 2026
JanCizmar pushed a commit that referenced this pull request Jun 3, 2026
Supersedes #3459
Fixes #3053

## Summary
- **Import:** the PO parser now captures the optional `msgctxt` field.
On import, Tolgee stores the pair `(msgctxt, msgid)` as a single key
name using the U+0004 EOT separator — the same convention GNU Gettext
uses internally in MO files — so two entries with the same `msgid` but
different `msgctxt` remain distinct keys.
- **Export:** the PO exporter splits the key name on U+0004 and writes
the `msgctxt` line back before `msgid`. Plural fallback now uses the
split `msgid` instead of the raw key name.
- **Parser cleanup:** header detection now requires both `msgid` and
`msgctxt` to be empty, so a malformed msgctxt-only entry can't be
misread as the header block. The legacy `PO_MSGCTXT_NOT_SUPPORTED` file
issue is no longer emitted (the enum value stays in place for
back-compat with previously stored issues).
- **UI display:** a new `<KeyName>` component renders the `msgctxt`
portion as a styled chip in front of the `msgid` across all key-name
display sites (translations grid, simple/list view, single-key page
breadcrumb, activity log, import dialogs, translation-memory rows and
tooltips, branching/merge header, screenshot label tooltips). The window
title intentionally still shows the raw key — the chip cannot render in
a `document.title` string.
- **UI edit:** the `<Editor>` component gains a `keyName` mode that
wires a new CodeMirror plugin (`KeyNamePlugin` from
`@tginternal/editor`) decorating the U+0004 with an atomic widget.
Display and edit are now visually identical — the previous
substitution-based workaround (`keyNameForEditing`/`keyNameFromEdited` +
`␄` visible-char) is removed.
- **Vite config:** added `@codemirror/state`, `@codemirror/view`,
`@codemirror/lint` to `resolve.dedupe` to avoid the "multiple instances
of @codemirror/state" runtime error introduced when `preserveSymlinks:
true` is combined with the linked `@tginternal/editor` worktree.

## Notes
- Re-importing a `.po` file that was imported before this change (when
msgctxt was silently dropped) will create new keys for the `(msgctxt,
msgid)` pairs instead of updating the existing `msgid`-only keys.
Accepted breakage — most users import once.
- An empty `msgctxt ""` collapses to a plain key without a chip,
matching gettext semantics.

## Companion PRs
- editor: tolgee/editor#8
- tolgee-js: tolgee/tolgee-js#3523
- documentation: tolgee/documentation#1111

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Full PO msgctxt support: import/export preserves and uses context to
disambiguate identical strings.
* Key display shows visual context indicators with a tooltip linking to
docs via a new KeyName component.
* Key editor gains a new "keyName" mode for viewing/editing keys with
context.
* Import ignores stray context-only entries to avoid header mis-parsing.

* **Tests**
* Added unit and end-to-end tests for msgctxt parsing, escaping,
import/export, fixtures and UI behavior.

* **Chores**
  * Editor package updated.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/tolgee/tolgee-platform/pull/3694?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
TolgeeMachine added a commit that referenced this pull request Jun 3, 2026
# [3.197.0](v3.196.1...v3.197.0) (2026-06-03)

### Bug Fixes

* bump postgres base image 13.21 → 13.23 ([#3717](#3717)) ([7e45c07](7e45c07))

### Features

* hide QA recheck batch operation from non-privileged users ([#3703](#3703)) ([3dcd142](3dcd142))
* support gettext msgctxt in PO import/export and UI ([#3694](#3694)) ([dfd7775](dfd7775)), closes [#3459](#3459) [#3053](#3053)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Store msgctxt from Gettext (.po) files

3 participants