Skip to content

Add Python symmetry tests for ingested DAQ sessions#42

Merged
stevevanhooser merged 3 commits into
mainfrom
claude/add-python-symmetry-tests-ImbOM
Mar 19, 2026
Merged

Add Python symmetry tests for ingested DAQ sessions#42
stevevanhooser merged 3 commits into
mainfrom
claude/add-python-symmetry-tests-ImbOM

Conversation

@stevevanhooser

Copy link
Copy Markdown
Contributor

Summary

This PR adds comprehensive Python symmetry tests for ingested DAQ sessions, mirroring the existing MATLAB test suite. The changes enable cross-language validation of session ingestion workflows for both Intan and Axon data formats using native and NDR reader implementations.

Key Changes

  • New helper module (_ingestion_helpers.py): Provides shared utilities for ingestion-based tests

    • _find_intan_rhd() and _find_axon_abf(): Locate example data files from multiple well-known locations (environment variables, CI layout, home directory, package resources)
    • setup_intan_session() and setup_axon_session(): Set up session directories with real DAQ data and required epochprobemap files
    • delete_raw_files(): Clean up raw data files after ingestion, keeping only the .ndi database (mirrors MATLAB behavior)
  • Make artifacts tests (3 new test classes):

    • TestIngestionIntan: Creates and exports artifacts for native Intan reader ingestion
    • TestIngestionIntanNDR: Creates and exports artifacts for Intan data via NDR wrapper reader
    • TestIngestionAxonNDR: Creates and exports artifacts for Axon ABF data via NDR wrapper reader
    • Each test: ingests a real DAQ session, strips raw files, exports session database and JSON documents to persistent artifact directories
  • Read artifacts tests (3 new test classes):

    • TestIngestionIntan, TestIngestionIntanNDR, TestIngestionAxonNDR: Parameterized tests that load artifacts from either MATLAB or Python sources
    • Verify session summaries match stored JSON
    • Validate all exported documents can be loaded from the ingested database
    • Enable cross-language validation of ingestion workflows

Notable Implementation Details

  • Tests are conditionally skipped if example data files are not available, allowing graceful degradation in CI environments without required datasets
  • Artifact directories follow the same naming convention as MATLAB tests (including British spelling "Artefacts" for consistency)
  • Session setup mirrors MATLAB test structure: creates filenavigator, daqreader, and daqsystem documents with proper dependencies
  • Read artifact tests are parameterized to validate both MATLAB-generated and Python-generated artifacts with identical test logic
  • Supports multiple data discovery mechanisms: explicit environment variables, CI layout assumptions, home directory conventions, and package-bundled resources

https://claude.ai/code/session_01DqZDq59zdpbunYovuPKahY

claude added 3 commits March 19, 2026 14:01
Mirror the MATLAB symmetry tests for session ingestion artifacts:
- ingestionIntan: native Intan reader with .rhd data
- ingestionIntanNDR: NDR wrapper reader with Intan .rhd data
- ingestionAxonNDR: NDR wrapper reader with Axon .abf data

Each test type has both makeArtifacts (generate) and readArtifacts
(verify) halves. makeArtifacts tests skip when example data is not
available; readArtifacts tests skip when artifacts don't exist.
Shared helpers in _ingestion_helpers.py handle data discovery and
session setup.

https://claude.ai/code/session_01DqZDq59zdpbunYovuPKahY
Adds `query = ndi_query` in __init__.py so that `ndi.query('','isa','base')`
works directly, matching MATLAB conventions. All existing import patterns
(`from ndi.query import ndi_query`) continue to work unchanged.

https://claude.ai/code/session_01DqZDq59zdpbunYovuPKahY
When reconstructing objects from documents (filenavigator, daqreader,
metadatareader, subject), `self.identifier = base_id` set a plain
attribute instead of updating `self._id` which is what `ndi_ido.id`
reads. This caused freshly generated IDs to be used in database queries
(e.g. find_ingested_documents) instead of the stored document IDs,
making epochfiles_ingested lookups fail silently.

https://claude.ai/code/session_01DqZDq59zdpbunYovuPKahY
@stevevanhooser stevevanhooser merged commit 0e4cc6e into main Mar 19, 2026
5 checks passed
@stevevanhooser stevevanhooser deleted the claude/add-python-symmetry-tests-ImbOM branch March 19, 2026 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants