Add image/frame reading API, TIFF + NANSEN ImageStack readers (NDI-matlab#823 companion)#106
Merged
Merged
Conversation
Add the imageseries half of the NDR reader interface, the NDR-matlab companion work for VH-Lab/NDI-matlab#823. Image readers are siblings of the regularly-sampled (ephys) readers, not subclasses: they implement only the frame API. - ndr.reader.base / ndr.reader: add the frame API surface (numframes, framesize, dimensionorder, datatype, frametimes, readframes), modeled on nansen.stack.ImageStack (VervaekeLab). epochclock/t0_t1 already exist and carry the clock for movies vs. clockless stacks. - ndr.reader.tiffstack: native multipage-TIFF reader using only built-in MATLAB TIFF support (no external deps). A '<name>_frametimes.txt' sidecar marks a movie (dev_local_time); absence means a clockless ordered stack (no_time). - ndr.reader.imagestack: optional backend wrapping nansen.stack.ImageStack to expose all NANSEN +virtual formats through one reader string. NANSEN stays an optional dependency, loaded only when this reader is used. - Register 'tiffstack' and 'imagestack' in ndr_reader_types.json so they appear in ndr.known_readers(). - tools/tests/+ndr/+unittest/+reader/TestTiffstack.m: CI unit tests for the TIFF frame API (clockless + movie cases), independent of NDI. NANSEN attribution is cited in the class headers; only the API/design is adapted, no NANSEN source is copied. https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
Declare NANSEN (VervaekeLab) in tools/requirements.txt so matbox.installRequirements installs it, including in the CI test workflows. NANSEN is still touched only by the ndr.reader.imagestack backend; native readers do not use it. Fix the imagestack backend against the real ImageStack API: - construct via nansen.stack.open(filename) -> virtual data -> nansen.stack.ImageStack(virtualData); ImageStack's constructor does not accept a filename directly. - restore explicit singleton C/Z dimensions after getFrameSet, which returns YXCZT data squeezed along singleton dimensions, so the NDR frame contract ([Y X C Z T]) holds. Add tools/tests/+ndr/+unittest/+reader/TestImagestack.m: CI unit tests for the NANSEN-backed reader (geometry, frame round-trip, agreement with the native tiffstack reader, clock consistency). Tests are skipped via assumeTrue when NANSEN is not on the path, so local runs without the requirements installed do not fail. https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Analyzer found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Contributor
Test Results170 tests 170 ✅ 7s ⏱️ Results for commit 69a929e. ♻️ This comment has been updated with latest results. |
NANSEN's ImageStack.getFrameTimes returns durations and, for formats whose adapter supplies no timing metadata (TimeIncrement unset), can return an empty/short array instead of erroring. Guarantee the (epochfiles, frameind) -> times contract regardless: - convert duration results to seconds (double), - coerce any result whose length != numel(frameind) to a NaN vector of the correct length. epochclock/t0_t1 already fall back to no_time when the times are not finite, so a format with no timing surfaces as a clockless epoch rather than a fabricated timeline; this just makes frametimes itself length-correct for direct callers. Add TestImagestack.testFrametimesLengthContractAndConsistency: asserts one returned time per requested frame across several index patterns and that frametimes is consistent with the reported epoch clock (no_time => all NaN, dev_local_time => all finite). https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
An imaging acquisition can write thousands of single-plane files per epoch (e.g. Prairie View). Make both image readers accept an epoch described as a single file, a directory, or an explicit list of files, so the epoch can be anchored on its directory instead of enumerating every frame file. ndr.reader.tiffstack (native): - imagefiles(): expand directories to their ordered .tif/.tiff contents; ignore non-image companions. - resolveepoch()/framesource(): map a global frame index to a (file, page) source across the ordered files, assuming a homogeneous stack (geometry/pages taken from the first file). - numframes/framesize/datatype/readframes now span all files; frametimes reads a directory-level 'frametimes.txt' for multi-file epochs (or the '<base>_frametimes.txt' sidecar for a single file). ndr.reader.imagestack (NANSEN): - imagesource()/sameextfiles(): expand a directory to its ordered, same-extension image files and pass that list to nansen.stack.open (which rejects a bare folder); drops companion files like Prairie .xml. Tests: TestTiffstack gains a Prairie-like directory epoch (single-frame TIFFs + directory frametimes.txt + an ignored .xml), checking geometry, ordered frame round-trip, directory-level timestamps, and that passing the directory equals passing the explicit file list. https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
…tory The file navigator can only discover epochs by matching a file, not a bare directory, so an imaging epoch is anchored on a per-epoch marker (e.g. a Prairie '.xml'/'.pcf' config). Make ndr.reader.tiffstack accept such an anchor: any epochstream entry that is neither a TIFF nor a directory is treated as a marker, and the stack's TIFFs are taken from the marker's parent directory. This is the format-agnostic scaffolding; format-specific parsing of the anchor (e.g. reading frame times out of a Prairie config) layers on top. Factor the directory globbing into tiffsindir(). Add TestTiffstack coverage that passing the .xml anchor resolves to the same frames and directory-level timestamps as passing the directory. https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
…tlab)
Add ndr.reader.prairieview for legacy Prairie recordings: a directory of
one-TIFF-per-frame plus a '*_Main.pcf' config whose
'[Image TimeStamp (us)]' section provides true per-frame timestamps. The
epoch surfaces as a 'dev_local_time' movie with those (possibly
irregular) times.
The reader extends ndr.reader.tiffstack, inheriting the directory frame
reading (enumerate TIFFs in name order, framesize, datatype, readframes,
anchor-file resolution) and overriding only the timing (frametimes /
epochclock / t0_t1) to come from the Prairie config.
Config handling is ported and revised from VH-Lab/vhlab-TwoPhoton-matlab
(Platforms/PrairieView readprairieconfig.m, tpconfigfilename.m) into
ndr.format.prairieview.{readconfig,configfilename}: the .pcf section
parser is preserved but made robust to CR/LF/CRLF line endings and uses
str2double. Modern Prairie 2.2+ XML is not parsed here (use
ndr.reader.imagestack / NANSEN for that); an .xml config returns
is_xml=true and the reader falls back to tiffstack timing.
Register reader strings prairieview/prairie/pv. Add TestPrairieView
(synthetic .pcf + frame TIFFs) covering config parsing, config-file
discovery, geometry, frame round-trip, and config-derived timestamps
(including the .pcf passed as the epoch anchor).
v1 assumes a single channel (one TIFF per frame); multi-channel
interleaving is future work.
https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
Each Prairie channel is written as its own TIFF
('..._Cycle<n>_Ch<c>_<frame>...'). Parse the Cycle/Ch/frame tokens from
the file names and group all channels of a timepoint onto the C axis, so
a "frame" is one timepoint with one timestamp, and readframes returns
[Y X C 1 nTimepoints] with channels in ascending Ch order.
ndr.reader.prairieview now overrides the frame layout (framelayout +
numframes/framesize/datatype/readframes) instead of inheriting the
one-TIFF-per-frame layout from tiffstack; timing still comes from the
config's real per-frame timestamps. This mirrors how NANSEN's
PrairieViewTiffs groups channels from file names, but keeps the true
per-frame timestamps (NANSEN derives uniform times from a single frame
period).
Factor the imfinfo->class mapping into a static ndr.reader.tiffstack.tiffclass
shared by tiffstack.datatype and the prairieview layout.
TestPrairieView gains a 2-channel recording fixture and tests for
multi-channel geometry, frame round-trip (channels on C), and one
timestamp per timepoint.
https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NDR-side companion work for VH-Lab/NDI-matlab#823 (imageseries acquisition/ingest/read machinery). Adds a frame-based reading API to
ndr.reader, two concrete image readers, requirements, and CI unit tests.Frame API (
ndr.reader.base/ndr.reader)New methods, each taking
(epochstreams, epoch_select, ...)like the existing ephys API:numframes,framesize(reports[Y X C Z T]without loading pixels),dimensionorder(default'YXCZT', channel kept separate from spatial/ordering axes),datatype,frametimes,readframes. Image readers are siblings of the regularly-sampled readers, not subclasses — they implement only the frame API. Design adapted fromnansen.stack.ImageStack(VervaekeLab), cited in headers; no NANSEN source copied.Concrete readers (registered in
ndr.known_readers())ndr.reader.tiffstack('tiffstack') — native multipage/multichannel TIFF reader using only built-in MATLAB TIFF support. Timing model: a<name>_frametimes.txtsidecar (one time per page, seconds) marks a movie →epochclock=dev_local_time,t0_t1=[first last]; no sidecar → clockless ordered stack →no_time,[NaN NaN].ndr.reader.imagestack('imagestack') — backend wrappingnansen.stack.ImageStack, exposing all of NANSEN's+virtualformat adapters through one reader string. Constructs vianansen.stack.open(file)→ virtual adapter →ImageStack, and restores explicit singleton C/Z dims aftergetFrameSet(which returns squeezed YXCZT). Errors with a clear install message if NANSEN is absent; native readers never touch it.Requirements / CI
tools/requirements.txt: addedhttps://github.com/VervaekeLab/NANSEN@main, installed bymatbox.installRequirementsin both CI workflows (NANSEN was recently made R2025a-compatible).tools/tests/+ndr/+unittest/+reader/TestTiffstack.m— frame API tests against synthesized TIFFs (clockless + movie cases).tools/tests/+ndr/+unittest/+reader/TestImagestack.m— NANSEN-backed reader tests (geometry, frame round-trip, agreement with the native tiffstack reader, clock consistency); skipped viaassumeTruewhen NANSEN is not on the path.Notes for review
external/tree) affecting existing tests.Companion NDI-matlab changes (bridge reader
ndi.daq.reader.image.ndr,ndi.daq.system.image, element/probe, ingestion docs) are on the same branch name in NDI-matlab.https://claude.ai/code/session_01M7f7wSBJeN4QJFvuCwdsSu
Generated by Claude Code