Skip to content

Map and decode metadata/post-processing streams (File Property, GUMM_Information, Mass Data Processing) #25

Description

@Nathan-D-R

Summary

Every .lcd file carries dozens of top-level storages beyond raw
spectra and chromatogram/PDA data that crates/openszraw has never
looked at - not in the reader, not in any format doc. A first-pass
survey (2026-07-21) mapped what's there and roughly what format each
piece is in; full writeup in
docs/format/08-metadata-and-processing-streams.md.

What's confirmed

  • File Property/Method File Property - plain UTF-8 XML (4-byte
    prefix + <?xml version="1.0"?>...), free-text fields hex-encoded
    under an @StoX@<hex> scheme (decoded successfully everywhere
    tried). Contains operator name, LabSolutions version, timezone, and
    a FILETIME pair that independently cross-validates the
    existing Acquisition start_timestamp not decoded - real candidate streams exist, none opened yet #9 CFBF-directory-timestamp technique for
    RunMetadata::start_timestamp (agreement within ~140ms/~1s on the
    one file checked).
  • GUMM_Information (GUC.*.CONFIG/METHOD, GUM.CONFIG/
    METHOD) - the full LC+MS instrument method/config. UTF-16LE XML
    wrapper, at least one field of which is a hex-encoded binary
    parameter blob tagged with an ASCII struct name (e.g.
    CTLM3030Parameters) - three encoding layers deep, not decoded past
    identifying the layers. Largest single substream seen: 137KB.
  • Mass Data Processing/Mass Data Load Format - LabSolutions'
    own post-acquisition compound-identification/quantitation results,
    not raw instrument data. Confirmed real (not boilerplate) via
    embedded ASCII compound names in Compound Table
    (m3U, m5U, m1Y, m3Y - standard methylated/pseudouridine RNA
    modification nomenclature) matching MTBLS7425's actual study
    subject exactly. Includes Compound Table/Results/Peak Table,
    calibration curve info, library search params, and a 378KB Fragment Table - the largest unexamined stream found this pass. Record
    layout not identified.
  • Lower-priority, not examined in depth: LSS Data Processing (LC
    detector/baseline processing config), Report Format (looks like
    OLE-embedded print/report templates, plausibly low analytical
    value), Audit Trail, SystemCheckResult.

Suggested priority (see docs/format/08 for full reasoning)

  1. File Property/Method File Property extraction - already
    decodable, smallest effort, real value (operator/version metadata
    plus independent Acquisition start_timestamp not decoded - real candidate streams exist, none opened yet #9 corroboration)
  2. Mass Data Processing compound-ID/quantitation tables - highest
    potential payoff (new capability, not just metadata) but least
    scoped effort-wise
  3. GUMM_Information method/config - large, real, multi-layer; useful
    context for other decode work (including the still-open PDA 3D Raw Data / LSS Raw Data chromatogram: bit-packed segment payload encoding undecoded #2)
  4. Everything else - lower priority

Constraints

Clean-room only, per CONTRIBUTING.md: no LabSolutions software or
vendor SDK/output as ground truth.

Done means

This issue is intentionally broad - treat it as a tracking/roadmap
issue, not a single PR. Reasonable to close once section 1 (File Property extraction) ships and split remaining sections into their
own issues as someone picks them up, rather than trying to land
everything in docs/format/08 in one pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions