Skip to content

Default NWB export to per-identity files; add --multisubject mode#403

Merged
gbeane merged 3 commits into
mainfrom
feature/nwb-multisubject-default-per-identity
Jun 23, 2026
Merged

Default NWB export to per-identity files; add --multisubject mode#403
gbeane merged 3 commits into
mainfrom
feature/nwb-multisubject-default-per-identity

Conversation

@gbeane

@gbeane gbeane commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Changes the default jabs-cli convert-to-nwb behavior and replaces the legacy combined single-file layout:

  • Per-identity files are now the default. The --per-identity flag is removed.
  • The legacy combined single-file layout is removed entirely — both the write path and the read fallback. A file written that way (a plain NWBFile with multiple identities and no layout flag) now raises a clear ValueError on read.
  • New --multisubject flag writes a single, self-contained .nwb using the ndx-multisubjects extension: an NdxMultiSubjectsNWBFile with a SubjectsTable (one row per subject) in acquisition, plus the same per-subject PoseEstimation layout in processing/behavior. Full round-trip (write + read) is supported.

Design notes

  • SubjectsTable attaches via add_acquisition (lands at acquisition/SubjectsTable), not a subjects_table attribute — the extension's 0.1.1 IO mapper for that attribute is disabled, and add_acquisition is the path that reliably persists under pynwb 3.x / hdmf 4.x (verified by a smoke test before implementation).
  • Round-trip fidelity rides on the jabs_metadata scratch JSON, exactly as before. The SubjectsTable is write-only redundancy for external/DANDI consumers; JABS recovers subjects/external_ids/metadata from jabs_metadata. A new multisubject: true flag in jabs_metadata selects the read path.
  • Reading a multisubject file uses load_namespaces=True so the NdxMultiSubjectsNWBFile subclass reconstructs; this is a no-op for plain per-identity files.
  • Split import guards: a missing ndx-multisubjects only disables --multisubject (raising a clear ImportError); the default per-identity / ndx-pose path still works.
  • SubjectsTable rows: required columns (subject_id, sex, species) are always written (defaulting to the identity name / "U" / ""); optional columns are unioned across identities and every cell is coerced to text, since a DynamicTable requires consistent columns per row.

Dependencies

ndx-multisubjects>=0.1.1 added to the jabs-io nwb extra; uv.lock regenerated. (The lock diff also syncs the workspace package version to the already-committed 0.45.0 pyproject version.)

Tests

  • The combined-file round-trip/structure tests were re-pointed to multisubject=True (the default write no longer creates the base OUTPUT path).
  • New tests: multisubject structure (NdxMultiSubjectsNWBFile + populated SubjectsTable), full-feature round-trip, heterogeneous subjects dicts, the multisubject metadata flag, legacy-layout read raises, multisubject/per-identity coexistence in one directory, and the missing-extension ImportError.
  • run_conversion write-mode wiring and CLI tests (--multisubject forwarded; --per-identity removed).
  • All 268 jabs-io tests and 14 convert_to_nwb script tests pass; ruff check/format clean.

Docs

Updated both mirrored doc trees (docs/user-guide/ and src/jabs/resources/docs/user_guide/) plus docs/development/jabs-nwb-format.md: new default, --multisubject, the SubjectsTable layout, and the jabs_metadata.multisubject flag. The dev doc's "Why ndx-multisubjects is not used" section is rewritten as "How multisubject files use ndx-multisubjects". (Also corrected a pre-existing inaccuracy: body_parts is not stored in jabs_metadata.)

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates JABS NWB export to make per-identity files the default output and adds an opt-in --multisubject mode that writes a single combined .nwb using the ndx-multisubjects extension (with a SubjectsTable in acquisition), while removing support for the legacy combined single-file layout.

Changes:

  • Switch NWB export default to per-identity files; remove the CLI --per-identity flag and reject legacy combined-layout files on read.
  • Add multisubject export/read support via ndx-multisubjects (NdxMultiSubjectsNWBFile + SubjectsTable) with a jabs_metadata.multisubject flag.
  • Update tests, dependency extras/lockfile, and docs to reflect the new modes and layouts.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
uv.lock Bumps workspace package versions and locks ndx-multisubjects==0.1.1 into the nwb extra.
packages/jabs-io/pyproject.toml Adds ndx-multisubjects>=0.1.1 to the nwb optional dependencies.
packages/jabs-io/src/jabs/io/internal/pose/nwb.py Implements multisubject write path, adds SubjectsTable creation, changes default write mode to per-identity, and updates read behavior to error on legacy layout.
packages/jabs-io/tests/internal/pose/test_nwb.py Re-points “single-file” tests to multisubject mode and adds new multisubject/legacy-layout test coverage.
src/jabs/scripts/cli/convert_to_nwb.py Renames/wires run_conversion to use multisubject flag and forwards it to save().
src/jabs/scripts/cli/cli.py Replaces --per-identity with --multisubject and updates CLI help/output text accordingly.
tests/scripts/test_convert_to_nwb.py Adds tests ensuring --multisubject and default behavior are correctly forwarded through CLI and run_conversion.
docs/user-guide/nwb-export.md Updates user docs for default per-identity export and new multisubject mode.
docs/user-guide/file-formats.md Updates NWB file-format docs to reflect multisubject layout and metadata flags.
src/jabs/resources/docs/user_guide/nwb-export.md Mirrors the user-guide NWB export doc updates in packaged resources.
src/jabs/resources/docs/user_guide/file-formats.md Mirrors the file-format doc updates in packaged resources.
docs/development/jabs-nwb-format.md Updates developer format spec and rationale to describe how ndx-multisubjects is used.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/jabs-io/src/jabs/io/internal/pose/nwb.py Outdated
Comment thread src/jabs/scripts/cli/convert_to_nwb.py
@gbeane gbeane self-assigned this Jun 19, 2026
@gbeane gbeane requested a review from bergsalex June 19, 2026 18:21
# Conflicts:
#	tests/scripts/test_convert_to_nwb.py
@gbeane gbeane merged commit 2833b17 into main Jun 23, 2026
5 checks passed
@gbeane gbeane deleted the feature/nwb-multisubject-default-per-identity branch June 23, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants