Skip to content

Decode LC Raw Data/Chromatogram Ch5 / Ch6 stream (found while investigating #2) #21

Description

@Nathan-D-R

Summary

While investigating #2 (PDA/chromatogram payload decode), a real,
populated chromatogram-shaped stream was found at a different top-level
storage than either name in #2's title: LC Raw Data/Chromatogram Ch5
and Ch6, found in PXD020792/*.LCD files (e.g. UY01-03-01p95.LCD,
14540 and 7422 bytes respectively). It has never been decoded and no
hypotheses from #2's extensive investigation have been tried against it

Full characterization is documented in docs/format/04-lcd-chromatogram-pda.md,
"LC Raw Data - a different, unrelated chromatogram stream" section.

Why this looks tractable

Structurally simpler than PDA's per-wavelength stream, which has
resisted 14 combined investigation sessions across #2:

  • Same 24-byte RC\x00\x00 segment header framing as PDA, but u32[3]
    (block size) spans the entire rest of the stream - the whole
    channel is one giant segment, not thousands of small ones like PDA.
  • Byte content looks qualitatively different and less complex: Ch5 (a
    "quiet"/flat channel) is dominated by one repeating 2-byte pattern
    (82 00 over and over, i.e. a constant u16 value) for almost its
    entire length; Ch6 (a "busier" channel) has small clustered
    single-byte values mostly in 0x00-0x20, with no obvious
    0x3f/0x20 escape-pair pattern that dominates and complicates the
    undecoded PDA payload.
  • Likely a conventional UV/RID detector channel rather than PDA -
    probably a fundamentally simpler encoding given the much lower
    apparent per-sample complexity.

What to try

  • Reuse iter_segments from re/src/analysis/pda_varint_bruteforce.py
    as a starting point (same header framing).
  • Given it's a single giant segment per channel rather than many small
    ones, the usual per-segment sweep approach from PDA 3D Raw Data / LSS Raw Data chromatogram: bit-packed segment payload encoding undecoded #2 won't directly
    apply - likely needs different framing (e.g. treating the whole
    stream as one long token sequence, or looking for internal
    sub-segment structure not yet characterized).
  • Ch5's dominant-constant-pattern content is a good first target -
    likely the easiest real signal in either channel to get a foothold on.

Constraints

Clean-room only, per CONTRIBUTING.md: no LabSolutions software,
vendor SDK, or vendor output as ground truth - same rules as #2.

Done means

A documented, evidence-backed payload decode (in a new or existing
docs/format/ doc), plus - if decoded - chromatogram support wired into
crates/openszraw via openmassspec_core's chromatogram-source trait,
mirroring how sibling readers expose iter_chromatograms. Even a
partial characterization (confirmed structure, even if the exact
per-value grammar remains open) would be useful progress, following
this project's existing precedent for honest partial writeups (#7, #19).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions