Default NWB export to per-identity files; add --multisubject mode#403
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates JABS NWB export to make per-identity files the default output and adds an opt-in --multisubject mode that writes a single combined .nwb using the ndx-multisubjects extension (with a SubjectsTable in acquisition), while removing support for the legacy combined single-file layout.
Changes:
- Switch NWB export default to per-identity files; remove the CLI
--per-identityflag and reject legacy combined-layout files on read. - Add multisubject export/read support via
ndx-multisubjects(NdxMultiSubjectsNWBFile+SubjectsTable) with ajabs_metadata.multisubjectflag. - Update tests, dependency extras/lockfile, and docs to reflect the new modes and layouts.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Bumps workspace package versions and locks ndx-multisubjects==0.1.1 into the nwb extra. |
packages/jabs-io/pyproject.toml |
Adds ndx-multisubjects>=0.1.1 to the nwb optional dependencies. |
packages/jabs-io/src/jabs/io/internal/pose/nwb.py |
Implements multisubject write path, adds SubjectsTable creation, changes default write mode to per-identity, and updates read behavior to error on legacy layout. |
packages/jabs-io/tests/internal/pose/test_nwb.py |
Re-points “single-file” tests to multisubject mode and adds new multisubject/legacy-layout test coverage. |
src/jabs/scripts/cli/convert_to_nwb.py |
Renames/wires run_conversion to use multisubject flag and forwards it to save(). |
src/jabs/scripts/cli/cli.py |
Replaces --per-identity with --multisubject and updates CLI help/output text accordingly. |
tests/scripts/test_convert_to_nwb.py |
Adds tests ensuring --multisubject and default behavior are correctly forwarded through CLI and run_conversion. |
docs/user-guide/nwb-export.md |
Updates user docs for default per-identity export and new multisubject mode. |
docs/user-guide/file-formats.md |
Updates NWB file-format docs to reflect multisubject layout and metadata flags. |
src/jabs/resources/docs/user_guide/nwb-export.md |
Mirrors the user-guide NWB export doc updates in packaged resources. |
src/jabs/resources/docs/user_guide/file-formats.md |
Mirrors the file-format doc updates in packaged resources. |
docs/development/jabs-nwb-format.md |
Updates developer format spec and rationale to describe how ndx-multisubjects is used. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
# Conflicts: # tests/scripts/test_convert_to_nwb.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes the default
jabs-cli convert-to-nwbbehavior and replaces the legacy combined single-file layout:--per-identityflag is removed.NWBFilewith multiple identities and no layout flag) now raises a clearValueErroron read.--multisubjectflag writes a single, self-contained.nwbusing the ndx-multisubjects extension: anNdxMultiSubjectsNWBFilewith aSubjectsTable(one row per subject) inacquisition, plus the same per-subjectPoseEstimationlayout inprocessing/behavior. Full round-trip (write + read) is supported.Design notes
SubjectsTableattaches viaadd_acquisition(lands atacquisition/SubjectsTable), not asubjects_tableattribute — the extension's 0.1.1 IO mapper for that attribute is disabled, andadd_acquisitionis the path that reliably persists under pynwb 3.x / hdmf 4.x (verified by a smoke test before implementation).jabs_metadatascratch JSON, exactly as before. TheSubjectsTableis write-only redundancy for external/DANDI consumers; JABS recoverssubjects/external_ids/metadatafromjabs_metadata. A newmultisubject: trueflag injabs_metadataselects the read path.load_namespaces=Trueso theNdxMultiSubjectsNWBFilesubclass reconstructs; this is a no-op for plain per-identity files.ndx-multisubjectsonly disables--multisubject(raising a clearImportError); the default per-identity / ndx-pose path still works.SubjectsTablerows: required columns (subject_id,sex,species) are always written (defaulting to the identity name /"U"/""); optional columns are unioned across identities and every cell is coerced to text, since aDynamicTablerequires consistent columns per row.Dependencies
ndx-multisubjects>=0.1.1added to thejabs-ionwbextra;uv.lockregenerated. (The lock diff also syncs the workspace package version to the already-committed0.45.0pyproject version.)Tests
multisubject=True(the default write no longer creates the baseOUTPUTpath).NdxMultiSubjectsNWBFile+ populatedSubjectsTable), full-feature round-trip, heterogeneoussubjectsdicts, themultisubjectmetadata flag, legacy-layout read raises, multisubject/per-identity coexistence in one directory, and the missing-extensionImportError.run_conversionwrite-mode wiring and CLI tests (--multisubjectforwarded;--per-identityremoved).jabs-iotests and 14convert_to_nwbscript tests pass;ruff check/formatclean.Docs
Updated both mirrored doc trees (
docs/user-guide/andsrc/jabs/resources/docs/user_guide/) plusdocs/development/jabs-nwb-format.md: new default,--multisubject, theSubjectsTablelayout, and thejabs_metadata.multisubjectflag. The dev doc's "Why ndx-multisubjects is not used" section is rewritten as "How multisubject files use ndx-multisubjects". (Also corrected a pre-existing inaccuracy:body_partsis not stored injabs_metadata.)🤖 Generated with Claude Code