Add ndi.setup.lab() for lab-based DAQ system configuration#41
Merged
Conversation
Mirror the new NDI-matlab symmetry tests that create blank sessions with lab-specific DAQ system configurations. Each lab has a makeArtifacts test that creates a session with the lab's DAQ systems (from ndi_common/daq_systems/ JSON configs) and exports artifacts, and a readArtifacts test that verifies the session summary round-trips correctly. New files: - _lab_setup.py: shared helper that reads DAQ config JSONs and creates filenavigator/daqreader/daqsystem documents (Python equivalent of ndi.setup.lab) - _summary_helpers.py: normalizes DAQ system ordering for comparison - 3 make_artifacts tests + 3 read_artifacts tests https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
Move lab setup logic from test helper to public ndi.setup.lab() API, matching the MATLAB ndi.setup.lab() convention. Users can now call ndi.setup.lab(session, "vhlab") to configure a session with lab-specific DAQ systems from the JSON configs in ndi_common/daq_systems/. https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
- Add NDI_FILENAVIGATOR_CLASS = "ndi.file.navigator.epochdir" to the epochdir navigator class - Register ndi_file_navigator_epochdir in the class registry - Use HasEpochDirectories from JSON config to pick the correct filenavigator class in ndi.setup.lab() - Use class registry to reconstruct the correct navigator subclass when loading DAQ systems from documents - Fall back to a generic daqreader with the stored class name when the specific reader class is not in the registry https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
…lsenvisintan) Add lab-specific stimulus DAQ readers: - vhlabvisspike2: extends CED Spike2 reader for VH Lab visual stimulus data - nielsenvisintan: extends Intan reader for Nielsen Lab visual stimulus data Both are registered in the class registry so DAQ systems can reconstruct them from documents, fixing the daqreader_class symmetry mismatches. https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
…AQ readers/navigators When a DAQ reader or file navigator class is not found in the registry, raise ValueError instead of silently falling back to a base class. This ensures missing class implementations are caught immediately. https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
- Fix ruff import sorting in class_registry.py - Fix black formatting in setup/lab.py and session_base.py - Remove element/probe fallback in _document_to_object: raise ValueError for unknown element classes instead of silently falling back to ndi_element or ndi_probe https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
sessionSummary previously relied on daqsystem_load() which silently drops DAQ systems whose reader classes are not implemented in Python (e.g. ndi.daq.reader.mfdaq.ndr). This caused length mismatches when comparing against MATLAB artifacts that include those systems. Now queries all daqsystem documents directly from the database, tries full object reconstruction first, and falls back to extracting class info from raw document properties for unimplemented readers. https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
- Implement ndi.daq.reader.mfdaq.ndr reader (SpikeInterface wrapper for ABF files), fixing marderlab's marder_abf DAQ system reconstruction - Add vhtaste_bpod.json and vhtaste_sync.json configs to match MATLAB's 5 vhlab DAQ systems - Implement VHAudreyBPod stimulus reader and metadata reader for BPod behavioral task data - Register all new classes in class_registry.py - Revert sessionSummary fallback: DAQ systems that fail to reconstruct should surface as errors, not be silently papered over https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
MATLAB's ndi.daq.reader.mfdaq.ndr constructor expects the document to have a daqreader_ndr section with ndr_reader_string. The lab setup was creating a plain daq/daqreader document, causing MATLAB to error with "Unrecognized field name daqreader_ndr" when reading Python artifacts. Now creates daq/daqreader_ndr documents with the ndr_reader_string populated from the config's DaqReaderFileParameters (e.g. "abf"). https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
Python and MATLAB may load DAQ system configs in different filesystem order. Sort by name in sessionSummary output and compare by name in compareSessionSummary so the positional comparison doesn't produce false mismatches. https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
MATLAB's buildDataset exports JSON documents into per-session subdirectories (jsonDocuments/<sessionId>/<docId>.json) rather than flat files. Update both makeArtifacts (write all dataset docs grouped by base.session_id) and readArtifacts (glob **/*.json) to match this structure. https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
MATLAB's buildDataset iterates session_list() and queries each session's database individually. The dataset's database contains extra documents (session, session_in_a_dataset) that aren't in the session DBs. Updated both make_artifacts and read_artifacts to match MATLAB's approach. https://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Python equivalent of MATLAB's
ndi.setup.lab()function, enabling automated DAQ system configuration from JSON files. This allows sessions to be initialized with lab-specific DAQ systems (vhlab, marderlab, kjnielsenlab) by reading configuration files fromndi_common/daq_systems/.Key Changes
New module
ndi.setup: Addedndi/setup/__init__.pyandndi/setup/lab.pyto provide lab configuration functionalitylab(session, lab_name): Main function that reads DAQ system JSON configs and creates corresponding documents (filenavigator, daqreader, daqmetadatareader, daqsystem)_find_daq_configs(): Locates and loads JSON configuration files for a given lab_to_matlab_cell_str(): Converts Python lists to MATLAB cell-array syntax for compatibilitySymmetry test suite: Added comprehensive test coverage for three labs (vhlab, marderlab, kjnielsenlab)
tests/symmetry/make_artifacts/session/: Tests that create blank sessions with lab DAQ systems and export artifactstests/symmetry/read_artifacts/session/: Tests that verify artifacts can be loaded and produce matching session summaries_summary_helpers.py: Utility to normalize DAQ system ordering in session summaries for reliable comparisonUpdated module exports: Modified
src/ndi/__init__.pyto include the newsetupmoduleImplementation Details
ndi_common/daq_systems/<lab_name>/directorydaq/filenavigator: Configured with file parameters and epoch probe map settingsdaq/daqreader: Configured with reader classdaq/daqmetadatareader: Created only if specified in configdaq/daqsystem: Links the above documents via dependency relationshipshttps://claude.ai/code/session_01XKnremisjfbzcM4aK7YMKq