Skip to content

Fix ANSI (PT_STRING8) text extraction: folder names, subjects, senders, bodies#62

Open
mekinney wants to merge 2 commits into
mooijtech:mainfrom
Private-Recall:fix/ansi-folder-name-string8
Open

Fix ANSI (PT_STRING8) text extraction: folder names, subjects, senders, bodies#62
mekinney wants to merge 2 commits into
mooijtech:mainfrom
Private-Recall:fix/ansi-folder-name-string8

Conversation

@mekinney

Copy link
Copy Markdown

Problem

ANSI .pst files (Outlook 97-2002) are unreadable today: folder names and every message string property (subject, sender, body, …) come back empty, and WalkFolders aborts on the first folder.

ANSI stores text as PropertyTypeString8 (a code-page string) rather than the PropertyTypeString (UTF-16) used by Unicode PSTs. Two places assume Unicode:

  1. Folder walk aborts. GetSubFolders reads PidTagDisplayName (prop 12289) with GetString(), which returns ErrPropertyTypeMismatch on a String8 value — aborting the entire walk on the first folder.

  2. String values silently dropped on decode. The generated property structs key each text field by "<ID><Type>" using the Unicode string type 31 (e.g. Subject is msg:"5531"). WriteMessagePackValue writes a String8 value under a "<ID>30" key, which no struct field matches — so the value is read from the heap correctly and then dropped during msgp decode. Integer/time properties survive because their type is identical across ANSI and Unicode; only strings are affected.

Additionally, GetMessage reads PidTagMessageClass with GetString(), so on ANSI every item fails that read and falls back to properties.Message{} — misclassifying appointments/contacts.

Fix

  • folder.go — branch on the actual property type for PidTagDisplayName: decode String8 via GetString8 (Windows-1252), else GetString. The walk no longer aborts on ANSI.
  • property_reader.go — in WriteMessagePackValue, normalize the String8 message-pack key to the Unicode string type (31) so the value lands in the same struct field the Unicode path populates. Default String8 decoding to Windows-1252 (a superset of ASCII covering Western European text) rather than UTF-8.
  • property_reader.go — add GetStringValue(), a type-agnostic getter (String or String8), for callers that don't control the PST format.
  • message.go — read the message class via GetStringValue() so ANSI items classify correctly.

Windows-1252 is the pragmatic default; a fully correct implementation would honor PidTagMessageCodepage / PidTagInternetCodepage, but 1252 decodes the common case (ASCII + Western European) correctly and matches the existing folder-name path.

Test

Adds TestANSIStringProperties over the existing data/32-bit.pst (ANSI) sample, asserting that a PT_STRING8 PidTagSubject decodes to its expected text. The existing test suite still passes.

Notes

These two commits were developed for and validated by a downstream consumer (private-recall) against real ANSI archives — subjects, senders, sender emails, and bodies all extract; no regression on Unicode PSTs.

Mike Kinney and others added 2 commits June 12, 2026 20:10
PidTagDisplayName (folder name, prop 12289) was read with GetString(),
which requires PropertyTypeString (PT_UNICODE). ANSI PSTs store it as
PropertyTypeString8 (a codepage string), so GetString() returned
ErrPropertyTypeMismatch on the first folder and aborted the entire
WalkFolders — making ANSI .pst files unreadable. Branch on the actual
property type and decode PT_STRING8 via GetString8 (the existing path,
defaulting to Windows-1252). Resolves the in-code TODO at this site.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Building on the folder-name fix, ANSI .pst files still surfaced every
message string (subject, sender, body, ...) as empty. Two causes:

1. The generated property structs key each text field by "<ID><Type>"
   using the Unicode string type 31 (e.g. Subject is msg:"5531"). On
   ANSI PSTs those properties carry PropertyTypeString8 (type 30), so
   WriteMessagePackValue wrote them under a "<ID>30" key that no struct
   field matched -- the values were read from the heap correctly but
   then dropped on decode. Normalize the String8 key to the Unicode
   string type so the value lands in the same field the Unicode path
   populates. Also default String8 decoding to Windows-1252 (a superset
   of ASCII covering Western European text) instead of UTF-8, matching
   the folder-name path.

2. GetMessage read PidTagMessageClass with GetString, which rejects
   String8, so every ANSI item fell back to properties.Message and
   appointments/contacts were misclassified. Add GetStringValue, a
   type-agnostic getter (String or String8), and use it for the class.

Adds TestANSIStringProperties over data/32-bit.pst asserting a
PT_STRING8 PidTagSubject decodes to its expected text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant