diff --git a/MATLAB_MAPPING.md b/MATLAB_MAPPING.md deleted file mode 100644 index 7f52c28..0000000 --- a/MATLAB_MAPPING.md +++ /dev/null @@ -1,554 +0,0 @@ -# MATLAB to Python Mapping - -Complete reference mapping every MATLAB NDI function/class to its Python equivalent. - -**MATLAB source**: [VH-Lab/NDI-matlab](https://github.com/VH-Lab/NDI-matlab) -**Python port**: [Waltham-Data-Science/NDI-python](https://github.com/Waltham-Data-Science/NDI-python) - ---- - -## Core - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.document` | `ndi.Document` | `ndi.document` | -| `ndi.query` | `ndi.Query` | `ndi.query` | -| `ndi.ido` | `ndi.Ido` | `ndi.ido` | -| `ndi.database` | `ndi.Database` | `ndi.database` | -| `ndi.version` | `ndi.version()` | `ndi.__init__` | -| `ndi.validate` | `ndi.validate` | `ndi.validate` | -| `ndi.documentservice` | `ndi.DocumentService` | `ndi.documentservice` | - -## Common Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.common.PathConstants` | `ndi.common.PathConstants` | `ndi.common` | -| `ndi.common.getLogger` | `ndi.common.getLogger()` | `ndi.common` | -| `ndi.common.getCache` | `ndi.common.getCache()` | `ndi.common` | -| `ndi.common.getDatabaseHierarchy` | `ndi.common.getDatabaseHierarchy()` | `ndi.common` | -| `ndi.common.assertDIDInstalled` | `ndi.common.assertDIDInstalled()` | `ndi.common` | - -## Time Synchronization - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.time.clocktype` | `ndi.time.ClockType` | `ndi.time.clocktype` | -| `ndi.time.timemapping` | `ndi.time.TimeMapping` | `ndi.time.timemapping` | -| `ndi.time.timereference` | `ndi.time.TimeReference` | `ndi.time.timereference` | -| `ndi.time.syncgraph` | `ndi.time.SyncGraph` | `ndi.time.syncgraph` | -| `ndi.time.syncrule` | `ndi.time.SyncRule` | `ndi.time.syncrule` | -| `ndi.time.syncrule_filematch` | `ndi.time.syncrule.FileMatchRule` | `ndi.time.syncrule.filematch` | - -## DAQ (Data Acquisition) - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.daq.system` | `ndi.daq.DAQSystem` | `ndi.daq.system` | -| `ndi.daq.system.mfdaq` | `ndi.daq.MFDAQSystem` | `ndi.daq.system_mfdaq` | -| `ndi.daq.daqsystemstring` | `ndi.daq.DAQSystemString` | `ndi.daq.daqsystemstring` | -| `ndi.daq.reader` | `ndi.daq.DAQReader` | `ndi.daq.reader` | -| `ndi.daq.reader.mfdaq` | `ndi.daq.MFDAQReader` | `ndi.daq.reader.mfdaq` | -| `ndi.daq.reader.mfdaq.intan` | `ndi.daq.reader.mfdaq.IntanReader` | `ndi.daq.reader.mfdaq.intan` | -| `ndi.daq.reader.mfdaq.blackrock` | `ndi.daq.reader.mfdaq.BlackrockReader` | `ndi.daq.reader.mfdaq.blackrock` | -| `ndi.daq.reader.mfdaq.cedspike2` | `ndi.daq.reader.mfdaq.CEDSpike2Reader` | `ndi.daq.reader.mfdaq.cedspike2` | -| `ndi.daq.reader.mfdaq.spikegadgets` | `ndi.daq.reader.mfdaq.SpikeGadgetsReader` | `ndi.daq.reader.mfdaq.spikegadgets` | -| `ndi.daq.metadatareader.NewStimStims` | `ndi.daq.metadatareader.NewStimStimsReader` | `ndi.daq.metadatareader.newstim_stims` | -| `ndi.daq.metadatareader.NielsenLabStims` | `ndi.daq.metadatareader.NielsenLabStimsReader` | `ndi.daq.metadatareader.nielsenlab_stims` | - -## Epoch, Element, Probe - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.epoch.epochprobemap_daqsystem` | `ndi.epoch.EpochProbeMapDAQSystem` | `ndi.epoch.epochprobemap_daqsystem` | -| `ndi.element` | `ndi.Element` | `ndi.element` | -| `ndi.probe` | `ndi.Probe` | `ndi.probe` | -| `ndi.probe.timeseries` | `ndi.probe.ProbeTimeseries` | `ndi.probe.timeseries` | -| `ndi.probe.timeseries.mfdaq` | `ndi.probe.ProbeTimeseriesMFDAQ` | `ndi.probe.timeseries_mfdaq` | -| `ndi.probe.timeseries.stimulator` | `ndi.probe.ProbeTimeseriesStimulator` | `ndi.probe.timeseries_stimulator` | -| `ndi.probe.fun.initProbeTypeMap` | `ndi.probe.initProbeTypeMap()` | `ndi.probe` | -| `ndi.probe.fun.getProbeTypeMap` | `ndi.probe.getProbeTypeMap()` | `ndi.probe` | - -## Session & Dataset - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.session` | `ndi.Session` | `ndi.session` | -| `ndi.session.dir` | `ndi.session.dir` / `ndi.DirSession` | `ndi.session.dir` | -| `ndi.session.sessiontable` | `ndi.session.SessionTable` | `ndi.session.sessiontable` | -| MockSession (conceptual) | `ndi.session.MockSession` | `ndi.session.mock` | -| `ndi.dataset` | `ndi.Dataset` | `ndi.dataset` (`.cloud_client` property for on-demand file fetching) | -| `ndi.dataset.dir` | `ndi.dataset.dir` / `ndi.Dataset` | `ndi.dataset` | -| `ndi.subject` | `ndi.Subject` | `ndi.subject` | -| `ndi.neuron` | `ndi.Neuron` | `ndi.neuron` | -| `ndi.element_timeseries` | `ndi.ElementTimeseries` | `ndi.element_timeseries` | - -## File Navigation - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.file.navigator` | `ndi.file.navigator` / `ndi.file.FileNavigator` | `ndi.file.navigator` | -| `ndi.file.navigator_epochdir` | `ndi.file.navigator_epochdir` / `ndi.file.EpochDirNavigator` | `ndi.file.navigator.epochdir` | -| `ndi.file.type.mfdaq_epoch_channel` | `ndi.file.type.mfdaq_epoch_channel` / `ndi.file.type.MFDAQEpochChannel` | `ndi.file.type.mfdaq_epoch_channel` | -| `ndi.file.pfilemirror` | `ndi.file.pfilemirror()` | `ndi.file.pfilemirror` | - -## App Framework - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.app` | `ndi.App` | `ndi.app` | -| `ndi.app.appdoc` | `ndi.app.AppDoc` | `ndi.app.appdoc` | -| `ndi.app.markgarbage` | `ndi.app.MarkGarbage` | `ndi.app.markgarbage` | -| `ndi.app.spikeextractor` | `ndi.app.SpikeExtractor` | `ndi.app.spikeextractor` | -| `ndi.app.spikesorter` | `ndi.app.SpikeSorter` | `ndi.app.spikesorter` | -| `ndi.app.oridirtuning` | `ndi.app.OriDirTuning` | `ndi.app.oridirtuning` | -| `ndi.app.stimulus.decoder` | `ndi.app.stimulus.StimulusDecoder` | `ndi.app.stimulus.decoder` | -| `ndi.app.stimulus.tuning_response` | `ndi.app.stimulus.TuningResponse` | `ndi.app.stimulus.tuning_response` | - -## Calculator Framework - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.calculator` | `ndi.Calculator` | `ndi.calculator` | -| `ndi.calc.example.simple` | `ndi.calc.example.SimpleCalc` | `ndi.calc.example.simple` | -| `ndi.calc.stimulus.tuningcurve` | `ndi.calc.stimulus.TuningCurveCalc` | `ndi.calc.stimulus.tuningcurve` | -| `ndi.calc.tuning_fit` | `ndi.calc.TuningFit` | `ndi.calc.tuning_fit` | - -## Database Functions (`ndi.database.fun.*`) - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.database.fun.findallantecedents` | `ndi.database_fun.find_all_antecedents()` | `ndi.database_fun` | -| `ndi.database.fun.findalldependencies` | `ndi.database_fun.find_all_dependencies()` | `ndi.database_fun` | -| `ndi.database.fun.docs_from_ids` | `ndi.database_fun.docs_from_ids()` | `ndi.database_fun` | -| `ndi.database.fun.docs2graph` | `ndi.database_fun.docs_to_graph()` | `ndi.database_fun` | -| `ndi.database.fun.find_ingested_docs` | `ndi.database_fun.find_ingested_docs()` | `ndi.database_fun` | -| `ndi.database.fun.finddocs_elementEpochType` | `ndi.database_fun.finddocs_elementEpochType()` | `ndi.database_fun` | -| `ndi.database.fun.ndi_document2ndi_object` | `ndi.database_fun.ndi_document2ndi_object()` | `ndi.database_fun` | -| `ndi.database.fun.copy_session_to_dataset` | `ndi.database_fun.copy_session_to_dataset()` | `ndi.database_fun` | -| `ndi.database.fun.finddocs_missing_dependencies` | `ndi.database_fun.finddocs_missing_dependencies()` | `ndi.database_fun` | -| `ndi.database.fun.write_presentation_time_structure` | `ndi.database_fun.write_presentation_time_structure()` | `ndi.database_fun` | -| `ndi.database.fun.read_presentation_time_structure` | `ndi.database_fun.read_presentation_time_structure()` | `ndi.database_fun` | -| `ndi.database.fun.database2json` | `ndi.database_fun.database2json()` | `ndi.database_fun` | -| `ndi.database.fun.copydocfile2temp` | `ndi.database_fun.copydocfile2temp()` | `ndi.database_fun` | -| `ndi.database.fun.extract_doc_files` | `ndi.database_fun.extract_doc_files()` | `ndi.database_fun` | - -## Database Ingestion - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.database.implementations.fun.ingest_plan` | `ndi.database_ingestion.ingest_plan()` | `ndi.database_ingestion` | -| `ndi.database.implementations.fun.ingest` | `ndi.database_ingestion.ingest()` | `ndi.database_ingestion` | -| `ndi.database.implementations.fun.expell_plan` | `ndi.database_ingestion.expell_plan()` | `ndi.database_ingestion` | -| `ndi.database.implementations.fun.expell` | `ndi.database_ingestion.expell()` | `ndi.database_ingestion` | - -## Document Comparison - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.database.doctools.docComparison` | `ndi.doc_comparison.DocComparison` | `ndi.doc_comparison` | - -## Utility Functions (`ndi.fun.*`) - -### Doc Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.doc.allTypes` | `ndi.fun.doc.allTypes()` | `ndi.fun.doc` | -| `ndi.fun.doc.getDocTypes` | `ndi.fun.doc.getDocTypes()` | `ndi.fun.doc` | -| `ndi.fun.doc.findFuid` | `ndi.fun.doc.findFuid()` | `ndi.fun.doc` | -| `ndi.fun.doc.subject.makeSpeciesStrainSex` | `ndi.fun.doc.makeSpeciesStrainSex()` | `ndi.fun.doc` | -| `ndi.fun.doc.probe.probeLocations4probes` | `ndi.fun.doc.probeLocations4probes()` | `ndi.fun.doc` | -| `ndi.fun.doc.diff` | `ndi.fun.doc.diff()` | `ndi.fun.doc` | -| `ndi.fun.doc.ontologyTableRowVars` | `ndi.fun.doc.ontologyTableRowVars()` | `ndi.fun.doc` | - -### DocTable Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.docTable.docCellArray2Table` | `ndi.fun.doc_table.docCellArray2Table()` | `ndi.fun.doc_table` | -| `ndi.fun.doc.ontologyTableRowDoc2Table` | `ndi.fun.doc_table.ontologyTableRowDoc2Table()` | `ndi.fun.doc_table` | -| `ndi.fun.docTable.element` | `ndi.fun.doc_table.element()` | `ndi.fun.doc_table` | -| `ndi.fun.docTable.subject` | `ndi.fun.doc_table.subject()` | `ndi.fun.doc_table` | -| `ndi.fun.docTable.probe` | `ndi.fun.doc_table.probe()` | `ndi.fun.doc_table` | -| `ndi.fun.docTable.epoch` | `ndi.fun.doc_table.epoch()` | `ndi.fun.doc_table` | -| `ndi.fun.docTable.openminds` | `ndi.fun.doc_table.openminds()` | `ndi.fun.doc_table` | -| `ndi.fun.docTable.treatment` | `ndi.fun.doc_table.treatment()` | `ndi.fun.doc_table` | - -### Epoch Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.epoch.epochid2element` | `ndi.fun.epoch.epochid2element()` | `ndi.fun.epoch` | -| `ndi.fun.epoch.filename2epochid` | `ndi.fun.epoch.filename2epochid()` | `ndi.fun.epoch` | -| `ndi.fun.doc.t0_t1cell2array` | `ndi.fun.epoch.t0_t1cell2array()` | `ndi.fun.epoch` | - -### File Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.file.MD5` | `ndi.fun.file.MD5()` | `ndi.fun.file` | -| `ndi.fun.file.dateCreated` | `ndi.fun.file.dateCreated()` | `ndi.fun.file` | -| `ndi.fun.file.dateUpdated` | `ndi.fun.file.dateUpdated()` | `ndi.fun.file` | - -### Data Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.data.readngrid` | `ndi.fun.data.readngrid()` | `ndi.fun.data` | -| `ndi.fun.data.writengrid` | `ndi.fun.data.writengrid()` | `ndi.fun.data` | -| `ndi.fun.data.mat2ngrid` | `ndi.fun.data.mat2ngrid()` | `ndi.fun.data` | -| — | `ndi.fun.data.evaluate_fitcurve()` | `ndi.fun.data` | -| `ndi.fun.data.readImageStack` | `ndi.fun.data.readImageStack()` | `ndi.fun.data` | - -### Stimulus Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.stimulus.tuning_curve_to_response_type` | `ndi.fun.stimulus.tuning_curve_to_response_type()` | `ndi.fun.stimulus` | -| `ndi.fun.stimulus.f0_f1_responses` | `ndi.fun.stimulus.f0_f1_responses()` | `ndi.fun.stimulus` | -| `ndi.fun.stimulus.findMixtureName` | `ndi.fun.stimulus.findMixtureName()` | `ndi.fun.stimulus` | -| `ndi.fun.stimulustemporalfrequency` | `ndi.fun.stimulus.stimulustemporalfrequency()` | `ndi.fun.stimulus` | -| `ndi.fun.calc.stimulus_tuningcurve_log` | `ndi.fun.stimulus.stimulus_tuningcurve_log()` | `ndi.fun.stimulus` | - -### Table Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.table.identifyMatchingRows` | `ndi.fun.table.identifyMatchingRows()` | `ndi.fun.table` | -| `ndi.fun.table.identifyValidRows` | `ndi.fun.table.identifyValidRows()` | `ndi.fun.table` | -| `ndi.fun.table.join` | `ndi.fun.table.join()` | `ndi.fun.table` | -| `ndi.fun.table.moveColumnsLeft` | `ndi.fun.table.moveColumnsLeft()` | `ndi.fun.table` | -| `ndi.fun.table.vstack` | `ndi.fun.table.vstack()` | `ndi.fun.table` | - -### Probe Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.probe.export_binary` | `ndi.fun.probe.export_binary()` | `ndi.fun.probe.export_binary` | -| `ndi.fun.probe.export_all_binary` | `ndi.fun.probe.export_all_binary()` | `ndi.fun.probe.export_binary` | -| `ndi.fun.probe.location` | `ndi.fun.probe.location()` | `ndi.fun.probe.location` | - -### Plot Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.plot.bar3` | `ndi.fun.plot.bar3()` | `ndi.fun.plot` | -| `ndi.fun.plot.multichan` | `ndi.fun.plot.multichan()` | `ndi.fun.plot` | -| `ndi.fun.plot.stimulusTimeseries` | `ndi.fun.plot.stimulusTimeseries()` | `ndi.fun.plot` | - -### General Utilities - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.channelname2prefixnumber` | `ndi.fun.channelname2prefixnumber()` | `ndi.fun.utils` | -| `ndi.fun.name2variableName` | `ndi.fun.name2variableName()` | `ndi.fun.utils` / `ndi.fun.name_utils` | -| `ndi.fun.pseudorandomint` | `ndi.fun.pseudorandomint()` | `ndi.fun.utils` | -| `ndi.fun.timestamp` | `ndi.fun.timestamp()` | `ndi.fun.utils` | - -### Session & Dataset Diff - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.fun.session.diff` | `ndi.fun.session.diff()` | `ndi.fun.session` | -| `ndi.fun.dataset.diff` | `ndi.fun.dataset.diff()` | `ndi.fun.dataset` | - -## Cloud API - -> For a detailed analysis of structural differences between the MATLAB and Python -> cloud modules (including why `+implementation/` was eliminated, where auth -> functions live, etc.), see [REPO_AUDIT.md](REPO_AUDIT.md). - -### Authentication - -Auth functions are re-exported from `ndi.cloud.__init__` so `from ndi.cloud import login` works. - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `ndi.cloud.authenticate` | `ndi.cloud.authenticate()` | | -| `ndi.cloud.api.auth.login` | `ndi.cloud.login()` | Also at `ndi.cloud.auth.login()` | -| `ndi.cloud.api.auth.logout` | `ndi.cloud.logout()` | Also at `ndi.cloud.auth.logout()` | -| `ndi.cloud.api.auth.changePassword` | `ndi.cloud.changePassword()` | | -| `ndi.cloud.api.auth.resetPassword` | `ndi.cloud.resetPassword()` | | -| `ndi.cloud.api.auth.verifyUser` | `ndi.cloud.verifyUser()` | | -| `ndi.cloud.api.auth.resendConfirmation` | `ndi.cloud.resendConfirmation()` | | -| `ndi.cloud.uilogin` | — | GUI-only, no Python equivalent | - -### Client & Config - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `ndi.cloud.api.url()` | `ndi.cloud.CloudClient` | HTTP client wrapping `requests.Session` | -| `ndi.cloud.api.call` (abstract classdef) | `ndi.cloud.CloudClient._request()` | See [REPO_AUDIT.md](REPO_AUDIT.md) §1 | -| `ndi.cloud.api.implementation.*` | — | Eliminated; `CloudClient` replaces all 60 impl classes | -| Cloud config constants | `ndi.cloud.CloudConfig` | Dataclass with `from_env()` classmethod | -| Cloud error IDs | `ndi.cloud.exceptions.*` | `CloudError`, `CloudAPIError`, `CloudAuthError`, `CloudNotFoundError`, `CloudSyncError`, `CloudUploadError` | - -### Datasets API (`ndi.cloud.api.datasets`) - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `getDataset` | `getDataset(dataset_id)` | | -| `createDataset` | `createDataset(org_id, name, ...)` | | -| `updateDataset` | `updateDataset(dataset_id, **fields)` | | -| `deleteDataset` | `deleteDataset(dataset_id, when='7d')` | `when` param for soft-delete | -| `listDatasets` | `listDatasets(org_id, ...)` | | -| — | `listAllDatasets(org_id)` | Auto-paginator (Python-only) | -| `getPublished` | `getPublished(...)` | | -| `getUnpublished` | `getUnpublished(...)` | | -| `publishDataset` | `publishDataset(dataset_id)` | | -| `unpublishDataset` | `unpublishDataset(dataset_id)` | | -| `submitDataset` | `submitDataset(dataset_id)` | | -| `createDatasetBranch` | `createDatasetBranch(dataset_id)` | | -| `getBranches` | `getBranches(dataset_id)` | | -| `undeleteDataset` | `undeleteDataset(dataset_id)` | | -| `listDeletedDatasets` | `listDeletedDatasets(...)` | | - -### Documents API (`ndi.cloud.api.documents`) - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `getDocument` | `getDocument(dataset_id, doc_id)` | | -| `addDocument` | `addDocument(dataset_id, doc_json)` | | -| `addDocumentAsFile` | `addDocumentAsFile(dataset_id, path)` | | -| `updateDocument` | `updateDocument(dataset_id, doc_id, doc_json)` | | -| `deleteDocument` | `deleteDocument(dataset_id, doc_id, when='7d')` | `when` param for soft-delete | -| `listDatasetDocuments` | `listDatasetDocuments(dataset_id, ...)` | | -| `listDatasetDocumentsAll` | `listDatasetDocumentsAll(dataset_id, ...)` | | -| `countDocuments` / `documentCount` | `countDocuments(dataset_id)` | Single function with fallback | -| `getBulkUploadURL` | `getBulkUploadURL(dataset_id)` | | -| `getBulkDownloadURL` | `getBulkDownloadURL(dataset_id, ...)` | | -| `bulkDeleteDocuments` | `bulkDeleteDocuments(dataset_id, doc_ids, when='7d')` | `when` param for soft-delete | -| `ndiquery` | `ndiquery(scope, search_structure, ...)` | | -| `ndiqueryAll` | `ndiqueryAll(scope, search_structure, ...)` | | -| — | `bulkUpload(dataset_id, zip_path)` | Python-only | -| `listDeletedDocuments` | `listDeletedDocuments(dataset_id, ...)` | | - -### Files API (`ndi.cloud.api.files`) - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `getFile` | `getFile(url, target_path, ...)` | | -| `getFileUploadURL` | `getFileUploadURL(org_id, dataset_id, uid)` | | -| `getFileCollectionUploadURL` | `getFileCollectionUploadURL(...)` | | -| `getFileDetails` | `getFileDetails(dataset_id, uid)` | Used by `fetch_cloud_file` for on-demand download | -| `listFiles` | `listFiles(dataset_id)` | | -| `putFiles` | `putFiles(url, file_path, ...)` | | -| — | `putFileBytes(url, data, ...)` | Python-only (raw bytes) | -| — | `getBulkUploadURL(org_id, dataset_id)` | Python-only | - -### Users API (`ndi.cloud.api.users`) - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `createUser` | `createUser(email, name, password)` | | -| `GetUser` | `GetUser(user_id)` | | -| `me` | `me()` | | - -### Compute API (`ndi.cloud.api.compute`) - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `startSession` | `startSession(pipeline_id, ...)` | | -| `getSessionStatus` | `getSessionStatus(session_id)` | | -| `triggerStage` | `triggerStage(session_id, stage_id)` | | -| `finalizeSession` | `finalizeSession(session_id)` | | -| `abortSession` | `abortSession(session_id)` | | -| `listSessions` | `listSessions()` | | - -### Top-Level Convenience Functions - -These match MATLAB's `ndi.cloud.*` functions and are importable directly from `ndi.cloud`: - -```python -from ndi.cloud import downloadDataset, uploadDataset, syncDataset, uploadSingleFile -``` - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `ndi.cloud.downloadDataset` | `ndi.cloud.downloadDataset(...)` | Also at `ndi.cloud.orchestration.downloadDataset()` | -| `ndi.cloud.uploadDataset` | `ndi.cloud.uploadDataset(...)` | Also at `ndi.cloud.orchestration.uploadDataset()` | -| `ndi.cloud.syncDataset` | `ndi.cloud.syncDataset(...)` | Also at `ndi.cloud.orchestration.syncDataset()` | -| `ndi.cloud.uploadSingleFile` | `ndi.cloud.uploadSingleFile(...)` | Also at `ndi.cloud.upload.uploadSingleFile()` | -| `ndi.cloud.upload.newDataset` | `ndi.cloud.orchestration.newDataset(...)` | | -| `ndi.cloud.upload.scanForUpload` | `ndi.cloud.orchestration.scanForUpload(...)` | | -| *(customFileHandler in didsqlite.m)* | `ndi.cloud.fetch_cloud_file(ndic_uri, path, ...)` | On-demand binary file download via `ndic://` protocol | - -### Download - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `ndi.cloud.download.dataset` | `download.downloadFullDataset(...)` | | -| `ndi.cloud.download.downloadDatasetFiles` | `download.downloadDatasetFiles(...)` | | -| `ndi.cloud.download.downloadDocumentCollection` | `download.downloadDocumentCollection(...)` | | -| `ndi.cloud.download.jsons2documents` | `download.jsons2documents(doc_jsons)` | | -| `ndi.cloud.download.datasetDocuments` | — | Handled inside orchestration | -| `ndi.cloud.download.internal.*` | — | Folded into main functions | -| `+sync/+internal/updateFileInfoForRemoteFiles` | `filehandler.rewrite_file_info_for_cloud()` | Rewrites file_info to `ndic://` URIs | -| `+download/+internal/setFileInfo` | `filehandler.rewrite_file_info_for_cloud()` | Same function handles both modes | - -### Upload - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `ndi.cloud.upload.uploadDocumentCollection` | `upload.uploadDocumentCollection(...)` | | -| `ndi.cloud.upload.zipForUpload` | `upload.zipForUpload(docs, ...)` | | -| `ndi.cloud.upload.uploadFilesForDatasetDocuments` | `upload.uploadFilesForDatasetDocuments(...)` | | -| `ndi.cloud.upload.uploadToNDICloud` | — | Subsumed by `uploadDataset()` | -| `ndi.cloud.upload.internal.*` | — | Promoted or folded inline | - -### Sync - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `SyncOptions` classdef | `ndi.cloud.sync.SyncOptions` | Dataclass | -| `SyncMode` enum | `ndi.cloud.sync.SyncMode` | Python Enum | -| `downloadNew` | `ndi.cloud.sync.downloadNew(...)` | | -| `uploadNew` | `ndi.cloud.sync.uploadNew(...)` | | -| `mirrorFromRemote` | `ndi.cloud.sync.mirrorFromRemote(...)` | | -| `mirrorToRemote` | `ndi.cloud.sync.mirrorToRemote(...)` | | -| `twoWaySync` | `ndi.cloud.sync.twoWaySync(...)` | | -| `validate` | `ndi.cloud.sync.validate(...)` | | -| — | `ndi.cloud.sync.sync(..., mode)` | Dispatch by SyncMode (Python-only) | -| `+sync/+internal/Constants` | — | Inlined | -| `+sync/+internal/index.*` (5 funcs) | `ndi.cloud.sync.SyncIndex` | Collapsed into dataclass | - -### Internal Helpers - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `+internal/listRemoteDocumentIds` | `internal.listRemoteDocumentIds()` | | -| `+internal/getCloudDatasetIdForLocalDataset` | `internal.getCloudDatasetIdForLocalDataset()` | | -| `+internal/createRemoteDatasetDoc` | `internal.createRemoteDatasetDoc()` | | -| `+internal/decodeJwt` | `auth.decodeJwt()` | Moved to auth | -| `+internal/getActiveToken` | `auth.getActiveToken()` | Moved to auth | -| `+internal/getTokenExpiration` | `auth.getTokenExpiration()` | Moved to auth | -| `+internal/getWeboptionsWithAuthHeader` | — | Replaced by `CloudClient` | -| `+internal/getUploadedDocumentIds` | — | Via `listRemoteDocumentIds()` | -| `+internal/getUploadedFileIds` | — | Via `listFiles()` | -| `+internal/dropDuplicateDocsFromJsonDecode` | — | Not needed (Python JSON is exact) | -| `+internal/duplicateDocuments` | `internal.duplicateDocuments()` | | -| `+sync/+internal/listLocalDocuments` | `internal.listLocalDocuments()` | | -| `+sync/+internal/getFileUidsFromDocuments` | `internal.getFileUidsFromDocuments()` | | -| `+sync/+internal/filesNotYetUploaded` | `internal.filesNotYetUploaded()` | | -| `+sync/+internal/datasetSessionIdFromDocs` | `internal.datasetSessionIdFromDocs()` | | -| `+sync/+internal/deleteLocalDocuments` | `sync.operations._delete_local_docs()` | Private | -| `+sync/+internal/deleteRemoteDocuments` | Inline in `mirrorToRemote()` | | -| `+sync/+internal/downloadNdiDocuments` | `sync.operations._download_docs_by_ids()` | Private | -| `+sync/+internal/uploadFilesForDatasetDocuments` | `upload.uploadFilesForDatasetDocuments()` | | -| *(ndic:// URI parsing in didsqlite.m)* | `filehandler.parse_ndic_uri()` | `ndic://dataset_id/file_uid` → tuple | - -### Admin (DOI & Crossref) - -| MATLAB | Python | Notes | -|--------|--------|-------| -| `ndi.cloud.admin.createNewDOI` | `admin.doi.createNewDOI()` | | -| `ndi.cloud.admin.registerDatasetDOI` | `admin.doi.registerDatasetDOI()` | | -| `ndi.cloud.admin.checkSubmission` | `admin.doi.checkSubmission()` | | -| `+crossref/Constants` | `admin.crossref.CrossrefConstants` | Frozen dataclass | -| `+crossref/createDoiBatchSubmission` | `admin.crossref.createDoiBatchSubmission()` | | -| `+crossref/convertCloudDatasetToCrossrefDataset` | `admin.crossref.convertCloudDatasetToCrossrefDataset()` | | -| `+crossref/createDatabaseMetadata` | — | Inline in `createDoiBatchSubmission()` | -| `+crossref/createDoiBatchHeadElement` | — | Inline in `createDoiBatchSubmission()` | -| `+crossref/+conversion/convertContributors` | `admin.crossref.convertContributors()` | | -| `+crossref/+conversion/convertDatasetDate` | `admin.crossref.convertDatasetDate()` | | -| `+crossref/+conversion/convertFunding` | `admin.crossref.convertFunding()` | | -| `+crossref/+conversion/convertLicense` | `admin.crossref.convertLicense()` | | -| `+crossref/+conversion/convertRelatedPublications` | `admin.crossref.convertRelatedPublications()` | | - -### Cloud: Not Ported - -| MATLAB | Reason | -|--------|--------| -| `ndi.cloud.uilogin` | MATLAB GUI | -| `ndi.cloud.ui.dialog.selectCloudDataset` | MATLAB GUI dialog | -| `ndi.cloud.utility.createCloudMetadataStruct` | MATLAB struct validator; `CloudConfig` replaces | -| `ndi.cloud.utility.mustBeValidMetadata` | MATLAB struct validator; type hints replace | - -## Validators (`ndi.validators.*`) - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.validators.mustBeCellArrayOfClass` | `ndi.validators.mustBeCellArrayOfClass()` | `ndi.validators.mustBeCellArrayOfClass` | -| `ndi.validators.mustBeCellArrayOfNdiSessions` | `ndi.validators.mustBeCellArrayOfNdiSessions()` | `ndi.validators.mustBeCellArrayOfNdiSessions` | -| `ndi.validators.mustBeCellArrayOfNonEmptyCharacterArrays` | `ndi.validators.mustBeCellArrayOfNonEmptyCharacterArrays()` | `ndi.validators.mustBeCellArrayOfNonEmptyCharacterArrays` | -| `ndi.validators.mustBeClassnameOfType` | `ndi.validators.mustBeClassnameOfType()` | `ndi.validators.mustBeClassnameOfType` | -| `ndi.validators.mustBeEpochInput` | `ndi.validators.mustBeEpochInput()` | `ndi.validators.mustBeEpochInput` | -| `ndi.validators.mustBeID` | `ndi.validators.mustBeID()` | `ndi.validators.mustBeID` | -| `ndi.validators.mustBeNumericClass` | `ndi.validators.mustBeNumericClass()` | `ndi.validators.mustBeNumericClass` | -| `ndi.validators.mustBeTextLike` | `ndi.validators.mustBeTextLike()` | `ndi.validators.mustBeTextLike` | -| `ndi.validators.mustHaveFields` | `ndi.validators.mustHaveFields()` | `ndi.validators.mustHaveFields` | -| `ndi.validators.mustHaveRequiredColumns` | `ndi.validators.mustHaveRequiredColumns()` | `ndi.validators.mustHaveRequiredColumns` | -| `ndi.validators.mustMatchRegex` | `ndi.validators.mustMatchRegex()` | `ndi.validators.mustMatchRegex` | - -## Utilities (`ndi.util.*`) - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.util.datestamp2datetime` | `ndi.util.datestamp2datetime()` | `ndi.util.datestamp2datetime` | -| `ndi.util.downsampleTimeseries` | `ndi.util.downsampleTimeseries()` | `ndi.util.downsampleTimeseries` | -| `ndi.util.hexDiff` | `ndi.util.hexDiff()` | `ndi.util.hexDiff` | -| `ndi.util.hexDiffBytes` | `ndi.util.hexDiffBytes()` | `ndi.util.hexDiffBytes` | -| `ndi.util.getHexDiffFromFileObj` | `ndi.util.getHexDiffFromFileObj()` | `ndi.util.getHexDiffFromFileObj` | -| `ndi.util.hexDump` | `ndi.util.hexDump()` | `ndi.util.hexDump` | -| `ndi.util.rehydrateJSONNanNull` | `ndi.util.rehydrateJSONNanNull()` | `ndi.util.rehydrateJSONNanNull` | -| `ndi.util.unwrapTableCellContent` | `ndi.util.unwrapTableCellContent()` | `ndi.util.unwrapTableCellContent` | -| `ndi.util.openminds.find_instance_name` | `ndi.openminds_convert.find_controlled_instance()` | `ndi.openminds_convert` | -| `ndi.util.openminds.find_techniques_names` | `ndi.openminds_convert.find_technique_names()` | `ndi.openminds_convert` | - -### Util: Not Ported - -| MATLAB | Reason | -|--------|--------| -| `ndi.util.choosefile` | MATLAB GUI dialog | -| `ndi.util.choosefileordir` | MATLAB GUI dialog | -| `ndi.util.toolboxdir` | MATLAB-specific path resolution | - -## Ontology - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.ontology` | `ndi.ontology.OntologyProvider` | `ndi.ontology` | -| `ndi.ontology.lookup` | `ndi.ontology.lookup()` | `ndi.ontology` | - -## OpenMINDS Integration - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.database.fun.openMINDSobj2struct` | `ndi.openminds_convert.openminds_obj_to_dict()` | `ndi.openminds_convert` | -| `ndi.database.fun.openMINDSobj2ndi_document` | `ndi.openminds_convert.openminds_obj_to_ndi_document()` | `ndi.openminds_convert` | -| `ndi.util.openminds.find_instance_name` | `ndi.openminds_convert.find_controlled_instance()` | `ndi.openminds_convert` | -| `ndi.util.openminds.find_techniques_names` | `ndi.openminds_convert.find_technique_names()` | `ndi.openminds_convert` | - -## Mock / Testing - -| MATLAB | Python | Module | -|--------|--------|--------| -| `ndi.mock.fun.subject_stimulator_neuron` | `ndi.mock.subject_stimulator_neuron()` | `ndi.mock` | -| `ndi.mock.fun.stimulus_presentation` | `ndi.mock.stimulus_presentation()` | `ndi.mock` | -| `ndi.mock.fun.clear` | `ndi.mock.clear_mock()` | `ndi.mock` | -| `ndi.mock.ctest` | `ndi.mock.CalculatorTest` | `ndi.mock` | - -## Not Ported (MATLAB-Specific) - -The following MATLAB components were intentionally not ported (GUI, MATLAB-specific tooling, one-time scripts): - -| MATLAB | Reason | -|--------|--------| -| `ndi.gui.*` (10+ files) | MATLAB GUI infrastructure | -| `ndi.database.fun.databasehierarchyinit` | MATLAB `eval()` for dynamic class creation | -| `ndi.fun.projectvardef` | MATLAB cell array helper | -| `metadata_ds_core/*` | MATLAB metadata editor GUI | -| `createNIFbrainareas.m` | One-time data preparation script | -| `readGenBankNames.m` / `readGenBankNodes.m` | Batch taxonomy scripts | -| `createGenBankControlledVocabulary.m` | Batch vocabulary builder | -| `find_calc_directories.m` | MATLAB path/toolbox discovery | -| `ndi.fun.assertAddonOnPath` | MATLAB addon/path checker | -| `ndi.fun.check_Matlab_toolboxes` | MATLAB toolbox checker | -| `ndi.fun.console` / `debuglog` / `errlog` / `syslog` | MATLAB console/logging | -| `ndi.fun.convertoldnsd2ndi` | Legacy NSD→NDI migration | -| `ndi.fun.run_Linux_checks` | MATLAB Linux environment checks | -| `ndi.fun.plot_extracellular_spikeshapes` | MATLAB GUI plotting | -| `ndi.util.choosefile` | MATLAB GUI dialog | -| `ndi.util.choosefileordir` | MATLAB GUI dialog | -| `ndi.util.toolboxdir` | MATLAB-specific path resolution | diff --git a/src/ndi/gui/component/ndi_matlab_python_bridge.yaml b/src/ndi/gui/component/ndi_matlab_python_bridge.yaml index 48c58de..4e66dba 100644 --- a/src/ndi/gui/component/ndi_matlab_python_bridge.yaml +++ b/src/ndi/gui/component/ndi_matlab_python_bridge.yaml @@ -85,6 +85,47 @@ classes: output_arguments: [] decision_log: "Abstract method. Subclasses implement completion display." + - name: setProgressTracker + input_arguments: + - name: tracker + type_matlab: "ndi.gui.component.internal.ProgressTracker" + type_python: "ProgressTracker" + output_arguments: [] + decision_log: > + Exact match. Detaches from previous tracker and attaches + listeners to the new one. Synchronized 2026-03-13. + + - name: getProgressValue + input_arguments: [] + output_arguments: + - name: value + type_python: "float" + decision_log: > + Exact match. Returns current progress as a fraction 0-1 + from the attached ProgressTracker. Synchronized 2026-03-13. + + - name: getProgressTitle + input_arguments: [] + output_arguments: + - name: title + type_python: "str" + decision_log: > + Exact match. Returns the tracker's rendered message, or + falls back to the Title property. Synchronized 2026-03-13. + + - name: formatMessage + input_arguments: + - name: message + type_matlab: "char" + type_python: "str" + output_arguments: + - name: formattedMessage + type_python: "str" + decision_log: > + Exact match. Appends estimated remaining time to the + message string if DisplayRemainingTime is enabled. + Synchronized 2026-03-13. + # ----------------------------------------------------------------------- # ndi.gui.component.internal.ProgressTracker # ----------------------------------------------------------------------- @@ -129,6 +170,12 @@ classes: type_matlab: "char" type_python: "str" decision_log: "Exact match." + - name: DumpFilePath + type_matlab: "char" + type_python: "str | None" + decision_log: > + Exact match. Path for async file-based progress dumps. + Used by AsynchProgressTracker subclass. Synchronized 2026-03-13. methods: - name: ProgressTracker @@ -186,6 +233,20 @@ classes: Overrides parent. Adds file-based progress dumping for cross-process monitoring. + - name: getAsynchTaskProgress + kind: static + input_arguments: + - name: filepath + type_matlab: "char" + type_python: "str" + output_arguments: + - name: progressData + type_python: "dict" + decision_log: > + Exact match. Reads progress state from a JSON dump file + written by updateProgress. Returns dict with CurrentStep, + TotalSteps, TemplateMessage keys. Synchronized 2026-03-13. + # ----------------------------------------------------------------------- # ndi.gui.component.internal.event.ProgressUpdatedEventData # ----------------------------------------------------------------------- diff --git a/src/ndi/ndi_matlab_python_bridge.yaml b/src/ndi/ndi_matlab_python_bridge.yaml index 012533a..b4c23ae 100644 --- a/src/ndi/ndi_matlab_python_bridge.yaml +++ b/src/ndi/ndi_matlab_python_bridge.yaml @@ -71,6 +71,38 @@ classes: type_python: "Ido" decision_log: "Exact match. Generates new ID if none provided." + - name: unique_id + kind: static + input_arguments: [] + output_arguments: + - name: id + type_python: "str" + decision_log: > + Exact match. Generates a new unique hex ID from current time + and random number. Synchronized 2026-03-13. + + - name: is_valid + kind: static + input_arguments: + - name: id_value + type_matlab: "char" + type_python: "str" + output_arguments: + - name: b + type_python: "bool" + decision_log: > + Exact match. Validates ID format (hex_hex or UUID). + Synchronized 2026-03-13. + + - name: eq + input_arguments: + - name: other + type_python: "Any" + output_arguments: + - name: b + type_python: "bool" + decision_log: "Mapped to Python __eq__. Compares by ID string." + decision_log: > Thin wrapper over did.ido. Python Ido class provides the same unique ID generation. Synchronized 2026-03-13. @@ -114,6 +146,31 @@ classes: Abstract-like method. Subclasses override to return queries matching their documents. + - name: load_element_doc + input_arguments: + - name: session + type_matlab: "ndi.session" + type_python: "Any" + output_arguments: + - name: doc + type_python: "Document | None" + decision_log: > + Exact match. Loads this object's document from the database + via searchquery(). Returns None if not found. + Synchronized 2026-03-13. + + - name: document_id + input_arguments: + - name: session + type_matlab: "ndi.session" + type_python: "Any" + output_arguments: + - name: doc_id + type_python: "str | None" + decision_log: > + Exact match. Returns the document ID for this object, + or None if not found. Synchronized 2026-03-13. + decision_log: > Interface class for document persistence. Any NDI class that can be stored as a document inherits from this. @@ -137,6 +194,7 @@ classes: decision_log: "MATLAB struct; Python uses a dict-like DotDict." methods: + # --- Constructor --- - name: document kind: constructor input_arguments: @@ -151,6 +209,29 @@ classes: type_python: "Document" decision_log: "Exact match." + # --- Identity --- + - name: id + input_arguments: [] + output_arguments: + - name: doc_id + type_python: "str" + decision_log: "Exact match. Returns base.id. Implemented as @property." + + - name: session_id + input_arguments: [] + output_arguments: + - name: sid + type_python: "str" + decision_log: "Exact match. Returns base.session_id. Implemented as @property." + + - name: doc_unique_id + input_arguments: [] + output_arguments: + - name: uid + type_python: "str" + decision_log: "Exact match. Alias for id()." + + # --- Session --- - name: set_session_id input_arguments: - name: session_id @@ -161,6 +242,7 @@ classes: type_python: "Document" decision_log: "Exact match. Returns self for chaining." + # --- Dependency Management --- - name: set_dependency_value input_arguments: - name: dependency_name @@ -169,32 +251,276 @@ classes: - name: dependency_value type_matlab: "char" type_python: "str" + - name: error_if_not_found + type_matlab: "logical" + type_python: "bool" + default: "True" output_arguments: - name: ndi_document_obj type_python: "Document" decision_log: "Exact match. Returns self for chaining." - - name: id + - name: dependency input_arguments: [] output_arguments: - - name: doc_id + - name: names + type_python: "list[str]" + - name: dependency_structures + type_python: "list[dict]" + decision_log: > + Exact match. Returns 2-tuple (names, dependency_structures). + Synchronized 2026-03-13. + + - name: dependency_value + input_arguments: + - name: dependency_name + type_matlab: "char" + type_python: "str" + - name: error_if_not_found + type_matlab: "logical" + type_python: "bool" + default: "True" + output_arguments: + - name: value + type_python: "str | None" + decision_log: > + Exact match. Returns dependency value by name. Raises + KeyError if not found and error_if_not_found is True. + Synchronized 2026-03-13. + + - name: dependency_value_n + input_arguments: + - name: dependency_name + type_matlab: "char" type_python: "str" - decision_log: "Exact match. Returns base.id." + - name: error_if_not_found + type_matlab: "logical" + type_python: "bool" + default: "True" + output_arguments: + - name: values + type_python: "list[str]" + decision_log: > + Exact match. Returns values from numbered dependency list + (e.g., element_1, element_2). Synchronized 2026-03-13. - - name: session_id + - name: add_dependency_value_n + input_arguments: + - name: dependency_name + type_matlab: "char" + type_python: "str" + - name: value + type_matlab: "char" + type_python: "str" + output_arguments: + - name: ndi_document_obj + type_python: "Document" + decision_log: > + Exact match. Appends to numbered dependency list. + Returns self for chaining. Synchronized 2026-03-13. + + - name: remove_dependency_value_n + input_arguments: + - name: dependency_name + type_matlab: "char" + type_python: "str" + - name: index + type_matlab: "integer" + type_python: "int | None" + default: "None" + output_arguments: + - name: ndi_document_obj + type_python: "Document" + decision_log: > + Exact match. Removes numbered dependency by index, or all + if index is None. Returns self for chaining. + Synchronized 2026-03-13. + + # --- File Management --- + - name: has_files input_arguments: [] output_arguments: - - name: sid + - name: b + type_python: "bool" + decision_log: > + Exact match. Checks if document has associated files. + Synchronized 2026-03-13. + + - name: add_file + input_arguments: + - name: name + type_matlab: "char" type_python: "str" - decision_log: "Exact match. Returns base.session_id." + - name: location + type_matlab: "char" + type_python: "str" + - name: ingest + type_matlab: "logical" + type_python: "bool | None" + default: "None" + - name: delete_original + type_matlab: "logical" + type_python: "bool | None" + default: "None" + - name: location_type + type_matlab: "char" + type_python: "str | None" + default: "None" + output_arguments: + - name: ndi_document_obj + type_python: "Document" + decision_log: > + Exact match. Adds a file to the document's file_info. + Auto-detects location_type from URL scheme. + Returns self for chaining. Synchronized 2026-03-13. - - name: doc_unique_id + - name: remove_file + input_arguments: + - name: name + type_matlab: "char" + type_python: "str" + - name: location + type_matlab: "char" + type_python: "str | None" + default: "None" + output_arguments: + - name: ndi_document_obj + type_python: "Document" + decision_log: > + Exact match. Removes a file or specific location from + the document. Returns self for chaining. + Synchronized 2026-03-13. + + - name: current_file_list input_arguments: [] output_arguments: - - name: uid + - name: file_names + type_python: "list[str]" + decision_log: > + Exact match. Returns list of file names currently + associated with this document. Synchronized 2026-03-13. + + # --- Document Class Information --- + - name: doc_class + input_arguments: [] + output_arguments: + - name: class_name type_python: "str" - decision_log: "Exact match. Alias for id()." + decision_log: > + Exact match. Returns the document class name + (e.g., 'base', 'element'). Synchronized 2026-03-13. + + - name: doc_superclass + input_arguments: [] + output_arguments: + - name: superclass_names + type_python: "list[str]" + decision_log: > + Exact match. Returns list of superclass names by + walking the document_class hierarchy. + Synchronized 2026-03-13. + + - name: doc_isa + input_arguments: + - name: document_class + type_matlab: "char" + type_python: "str" + output_arguments: + - name: b + type_python: "bool" + decision_log: > + Exact match. Checks if document is of a class or + inherits from it. Synchronized 2026-03-13. + # --- Serialization --- + - name: write + input_arguments: + - name: filename + type_matlab: "char" + type_python: "str" + - name: indent + type_python: "int" + default: "2" + output_arguments: [] + decision_log: > + Exact match. Writes document to JSON file. + Python adds indent parameter. Synchronized 2026-03-13. + + - name: setproperties + input_arguments: + - name: varargin + type_matlab: "name-value pairs" + type_python: "**kwargs" + output_arguments: + - name: ndi_document_obj + type_python: "Document" + decision_log: > + Exact match. Sets multiple properties at once using + dot-notation paths. Returns self for chaining. + Synchronized 2026-03-13. + + - name: validate + input_arguments: + - name: session + type_matlab: "ndi.session" + type_python: "Any" + default: "None" + output_arguments: + - name: result + type_python: "ValidationResult" + decision_log: > + Exact match. Three-tier validation: this-class properties, + superclass properties, dependency references. + Synchronized 2026-03-13. + + # --- Conversion --- + - name: to_table + input_arguments: [] + output_arguments: + - name: T + type_matlab: "table" + type_python: "pd.DataFrame" + decision_log: > + Exact match. Converts document to tabular form. + MATLAB returns table; Python returns pandas DataFrame. + Synchronized 2026-03-13. + + - name: to_dict + input_arguments: [] + output_arguments: + - name: d + type_python: "dict" + decision_log: > + Python-specific convenience. Returns deep copy of + document_properties. Synchronized 2026-03-13. + + - name: to_json + input_arguments: + - name: indent + type_python: "int" + default: "2" + output_arguments: + - name: json_str + type_python: "str" + decision_log: > + Python-specific convenience. Returns JSON string + representation. Synchronized 2026-03-13. + + # --- Merging --- + - name: plus + input_arguments: + - name: other + type_matlab: "ndi.document" + type_python: "Document" + output_arguments: + - name: merged_doc + type_python: "Document" + decision_log: > + Exact match. Merges two documents. Fields in self take + precedence. Mapped to Python __add__. Merges superclasses, + dependencies, and file_lists. Synchronized 2026-03-13. + + # --- Equality --- - name: eq input_arguments: - name: other @@ -202,7 +528,59 @@ classes: output_arguments: - name: b type_python: "bool" - decision_log: "Mapped to Python __eq__." + decision_log: "Mapped to Python __eq__. Compares by document ID." + + # --- Static Methods --- + - name: find_doc_by_id + kind: static + input_arguments: + - name: doc_array + type_matlab: "cell array of ndi.document" + type_python: "list[Document]" + - name: doc_id + type_matlab: "char" + type_python: "str" + output_arguments: + - name: doc + type_python: "Document | None" + - name: index + type_python: "int | None" + decision_log: > + Exact match. Finds a document in a list by ID. Returns + 2-tuple (document, index) or (None, None). + Synchronized 2026-03-13. + + - name: find_newest + kind: static + input_arguments: + - name: doc_array + type_matlab: "cell array of ndi.document" + type_python: "list[Document]" + output_arguments: + - name: newest_doc + type_python: "Document" + - name: index + type_python: "int" + - name: datestamp + type_python: "datetime" + decision_log: > + Exact match. Finds the newest document by datestamp. + Returns 3-tuple (doc, index, datestamp). + Synchronized 2026-03-13. + + - name: read_blank_definition + kind: static + input_arguments: + - name: document_type + type_matlab: "char" + type_python: "str" + output_arguments: + - name: definition + type_python: "dict" + decision_log: > + Exact match. Reads a blank document definition from + JSON schema. Recursively merges superclass properties. + Synchronized 2026-03-13. decision_log: > Core document class. Both languages use JSON schema-based @@ -223,8 +601,29 @@ classes: type_matlab: "struct" type_python: "list[dict]" decision_log: "Inherited from did.query." + - name: field + type_python: "str" + decision_log: > + Computed property. Returns the field being queried from + search_structure. Synchronized 2026-03-13. + - name: operator + type_python: "str | None" + decision_log: > + Computed property. Returns the Python-style operator + mapped from DID operation. Synchronized 2026-03-13. + - name: value + type_python: "Any" + decision_log: > + Computed property. Returns the query value (param1). + Synchronized 2026-03-13. + - name: queries + type_python: "list[Query]" + decision_log: > + List of sub-queries for composite (AND/OR) queries. + Synchronized 2026-03-13. methods: + # --- Constructor --- - name: query kind: constructor input_arguments: @@ -233,8 +632,8 @@ classes: type_python: "str" - name: op type_matlab: "char" - type_python: "str" - default: "''" + type_python: "str | None" + default: "None" - name: param1 type_matlab: "any" type_python: "Any" @@ -248,6 +647,7 @@ classes: type_python: "Query" decision_log: "Exact match." + # --- NDI-specific query types --- - name: isa input_arguments: - name: class_name @@ -266,16 +666,272 @@ classes: - name: dependency_value type_matlab: "char" type_python: "str" + default: "''" output_arguments: - name: q type_python: "Query" decision_log: "Exact match. Shorthand for depends_on query." + # --- Comparison operators (MATLAB overloads -> Python __eq__ etc.) --- + - name: eq + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + MATLAB == operator overload. Mapped to Python __eq__. + Resolves to exact_string or exact_number. + Synchronized 2026-03-13. + + - name: ne + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + MATLAB ~= operator overload. Mapped to Python __ne__. + Synchronized 2026-03-13. + + - name: lt + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "MATLAB < overload. Mapped to Python __lt__. Synchronized 2026-03-13." + + - name: le + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "MATLAB <= overload. Mapped to Python __le__. Synchronized 2026-03-13." + + - name: gt + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "MATLAB > overload. Mapped to Python __gt__. Synchronized 2026-03-13." + + - name: ge + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "MATLAB >= overload. Mapped to Python __ge__. Synchronized 2026-03-13." + + # --- Logical operators --- + - name: and + input_arguments: + - name: other + type_matlab: "ndi.query" + type_python: "Query" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + MATLAB & overload. Mapped to Python __and__. Concatenates + search_structures for AND semantics. Synchronized 2026-03-13. + + - name: or + input_arguments: + - name: other + type_matlab: "ndi.query" + type_python: "Query" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + MATLAB | overload. Mapped to Python __or__. Creates nested + 'or' operation in search_structure. Synchronized 2026-03-13. + + - name: not + input_arguments: [] + output_arguments: + - name: q + type_python: "Query" + decision_log: > + MATLAB ~ prefix. Mapped to Python __invert__. Negates all + operations by toggling ~ prefix. Synchronized 2026-03-13. + + # --- String/pattern methods --- + - name: contains + input_arguments: + - name: value + type_matlab: "char" + type_python: "str" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Exact match. Resolves to contains_string operation. + Synchronized 2026-03-13. + + - name: match + input_arguments: + - name: pattern + type_matlab: "char" + type_python: "str" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Exact match. Resolves to regexp operation. + Synchronized 2026-03-13. + + - name: equals + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Explicit equality method. Same as == operator. + Synchronized 2026-03-13. + + # --- Comparison methods (explicit) --- + - name: less_than + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "Explicit less-than method. Synchronized 2026-03-13." + + - name: less_than_or_equal_to + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "Explicit less-than-or-equal method. Synchronized 2026-03-13." + + - name: greater_than + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "Explicit greater-than method. Synchronized 2026-03-13." + + - name: greater_than_or_equal_to + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: "Explicit greater-than-or-equal method. Synchronized 2026-03-13." + + # --- Field existence --- + - name: has_field + input_arguments: [] + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Exact match. Resolves to hasfield operation. + Synchronized 2026-03-13. + + - name: has_member + input_arguments: + - name: value + type_python: "Any" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Exact match. Resolves to hasmember operation. + Synchronized 2026-03-13. + + # --- Static factory methods --- + - name: all + kind: static + input_arguments: [] + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Exact match. Returns query matching all documents + (isa 'base'). Synchronized 2026-03-13. + + - name: none + kind: static + input_arguments: [] + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Exact match. Returns query matching no documents. + Synchronized 2026-03-13. + + - name: from_search + kind: classmethod + input_arguments: + - name: field + type_matlab: "char" + type_python: "str" + - name: operation + type_matlab: "char" + type_python: "str" + - name: param1 + type_matlab: "any" + type_python: "Any" + default: "''" + - name: param2 + type_matlab: "any" + type_python: "Any" + default: "''" + output_arguments: + - name: q + type_python: "Query" + decision_log: > + Exact match. MATLAB-compatible factory method. Creates + query using DID-style operation strings. + Synchronized 2026-03-13. + + # --- Serialization --- + - name: to_searchstructure + input_arguments: [] + output_arguments: + - name: ss + type_python: "dict" + decision_log: > + Converts query to search structure dict with Python-style + operators. Synchronized 2026-03-13. + + - name: to_search_structure + input_arguments: [] + output_arguments: + - name: ss + type_python: "dict | list[dict]" + decision_log: > + Converts to DID-python compatible search structure. + Unwraps single-element lists. Synchronized 2026-03-13. + decision_log: > Database query class supporting operators: regexp, exact_string, contains_string, exact_number, lessthan, lessthaneq, greaterthan, greaterthaneq, hasfield, isa, depends_on, hasmember, and negated versions. + Module also exposes top-level convenience functions all(), + none(), from_search() for MATLAB parity. Synchronized 2026-03-13. # ========================================================================= @@ -312,3 +968,38 @@ not_applicable: decision_log: > MATLAB calculation pipeline registry with GUI editing. Not ported to Python. + + # ========================================================================= + # ndi.data namespace — MATLAB-only + # ========================================================================= + - name: data.evaluate_fitcurve + matlab_path: "+ndi/+data/evaluate_fitcurve.m" + status: not_applicable + decision_log: > + MATLAB utility for evaluating fitted curves from + ndi.data fitting results. Python equivalent uses + scipy.optimize curve fitting directly; no wrapper needed. + + # ========================================================================= + # ndi.setup namespace — MATLAB-only lab configurations + # ========================================================================= + - name: setup + matlab_path: "+ndi/+setup/" + status: not_applicable + decision_log: > + Entire MATLAB package containing lab-specific DAQ configurations + (angeluccilab, dbkatzlab, marderlab, vhlab, yangyangwang) and + sub-packages (+NDIMaker, +conv, +daq, +epoch, +stimulus, + @DaqSystemConfiguration). These are MATLAB-specific lab setup + wizards. Python labs configure via JSON/YAML files instead. + + # ========================================================================= + # ndi.docs namespace — MATLAB documentation generation + # ========================================================================= + - name: docs + matlab_path: "+ndi/+docs/" + status: not_applicable + decision_log: > + MATLAB documentation generation utilities (all_documents2markdown, + build, calcbuild, concatenateFiles, docfun, document2markdown, + schemastructure2docstructure). Python uses Sphinx/mkdocs instead. diff --git a/src/ndi/ndi_matlab_python_bridge_database.yaml b/src/ndi/ndi_matlab_python_bridge_database.yaml index 8f1a205..4564017 100644 --- a/src/ndi/ndi_matlab_python_bridge_database.yaml +++ b/src/ndi/ndi_matlab_python_bridge_database.yaml @@ -193,17 +193,117 @@ classes: the session object (session.database_closebinarydoc). Synchronized 2026-03-13. - # --- Additional Python methods not in MATLAB base class --- - # These are implementation-level or convenience methods in Python - # that correspond to the MATLAB didsqlite subclass behavior. + # --- Additional Python methods (from didsqlite subclass) --- + + - name: update + input_arguments: + - name: ndi_document_obj + type_python: "Document" + output_arguments: + - name: ndi_document_obj + type_python: "Document" + decision_log: > + From didsqlite subclass. Updates an existing document. + Raises ValueError if not found. Synchronized 2026-03-13. + + - name: add_or_replace + input_arguments: + - name: ndi_document_obj + type_python: "Document" + output_arguments: + - name: ndi_document_obj + type_python: "Document" + decision_log: > + Python convenience. Upsert: adds if new, replaces if exists. + Corresponds to MATLAB add() with Update=1. + Synchronized 2026-03-13. + + - name: find_by_id + input_arguments: + - name: doc_id + type_python: "str" + output_arguments: + - name: ndi_document_obj + type_python: "Document | None" + decision_log: > + Alias for read(). Convenience for MATLAB compatibility. + Synchronized 2026-03-13. + + - name: numdocs + input_arguments: [] + output_arguments: + - name: n + type_python: "int" + decision_log: > + Python convenience. Returns count of documents in database. + Synchronized 2026-03-13. + + - name: find_depends_on + input_arguments: + - name: document + type_python: "Document | str" + output_arguments: + - name: dependent_docs + type_python: "list[Document]" + decision_log: > + Python convenience. Finds all documents that depend on + the given document. Synchronized 2026-03-13. + + - name: find_dependencies + input_arguments: + - name: document + type_python: "Document | str" + output_arguments: + - name: dependency_docs + type_python: "list[Document]" + decision_log: > + Python convenience. Finds all documents that the given + document depends on. Synchronized 2026-03-13. + + - name: add_many + input_arguments: + - name: documents + type_python: "list[Document]" + output_arguments: + - name: added_docs + type_python: "list[Document]" + decision_log: > + Python convenience. Batch add. Stops on first error. + Synchronized 2026-03-13. + + - name: remove_many + input_arguments: + - name: query + type_python: "Query | None" + default: "None" + - name: documents + type_python: "list[Document] | None" + default: "None" + output_arguments: + - name: count + type_python: "int" + decision_log: > + Python convenience. Removes documents matching query + and/or explicit list. Returns count removed. + Synchronized 2026-03-13. + + - name: get_binary_path + input_arguments: + - name: document + type_python: "Document" + - name: file_name + type_python: "str" + output_arguments: + - name: path + type_python: "Path" + decision_log: > + Python convenience. Returns path for storing a + document's binary file. Synchronized 2026-03-13. + decision_log: > Python Database class merges the abstract ndi.database base class with the concrete ndi.database.implementations.database.didsqlite - subclass. Additional Python methods: update(), add_or_replace(), - find_by_id(), numdocs(), find_depends_on(), find_dependencies(), - add_many(), remove_many(), get_binary_path(). These are convenience - wrappers not present in the MATLAB abstract base but consistent - with didsqlite behavior. Synchronized 2026-03-13. + subclass. Synchronized 2026-03-13. # ========================================================================= # Standalone functions diff --git a/src/ndi/ndi_matlab_python_bridge_database_fun.yaml b/src/ndi/ndi_matlab_python_bridge_database_fun.yaml index c924a5c..498b0b5 100644 --- a/src/ndi/ndi_matlab_python_bridge_database_fun.yaml +++ b/src/ndi/ndi_matlab_python_bridge_database_fun.yaml @@ -347,3 +347,120 @@ functions: Note: the MATLAB .m file is named extract_docs_files.m but the function inside is named extract_doc_files. Using the function name. Synchronized 2026-03-13. + +# ========================================================================= +# Not Applicable — MATLAB functions intentionally not ported to Python +# ========================================================================= +not_applicable: + + - name: createGenBankControlledVocabulary + matlab_path: "+ndi/+database/+fun/createGenBankControlledVocabulary.m" + status: not_applicable + decision_log: > + MATLAB utility for creating a controlled vocabulary from GenBank + taxonomy data. Bioinformatics-specific helper for populating + ontology tables. Not needed in Python; equivalent functionality + available via BioPython or direct GenBank API access. + + - name: readGenBankNames + matlab_path: "+ndi/+database/+fun/readGenBankNames.m" + status: not_applicable + decision_log: > + MATLAB utility for reading GenBank taxonomy names from flat files. + Bioinformatics-specific parser. Python users can use BioPython + (Bio.Entrez) or pandas to parse GenBank taxonomy files directly. + + - name: readGenBankNodes + matlab_path: "+ndi/+database/+fun/readGenBankNodes.m" + status: not_applicable + decision_log: > + MATLAB utility for reading GenBank taxonomy node hierarchy from + flat files. Bioinformatics-specific parser. Python users can use + BioPython or pandas to parse GenBank taxonomy files directly. + + - name: createNIFbrainareas + matlab_path: "+ndi/+database/+fun/createNIFbrainareas.m" + status: not_applicable + decision_log: > + MATLAB utility for creating NIF (Neuroscience Information + Framework) brain area vocabulary entries. One-time data import + script for populating ontology tables. Not needed as a library + function in Python. + + - name: create_new_database + matlab_path: "+ndi/+database/+fun/create_new_database.m" + status: not_applicable + decision_log: > + MATLAB factory for instantiating a new database based on the + database hierarchy preferences. In Python, this functionality is + absorbed into Database.__init__() which directly creates the + SQLite backend. No separate factory needed. + + - name: databasehierarchyinit + matlab_path: "+ndi/+database/+fun/databasehierarchyinit.m" + status: not_applicable + decision_log: > + MATLAB function for initializing the database implementation + hierarchy (preference list of database backends). In Python, the + equivalent is ndi.common.getDatabaseHierarchy() which returns the + single SQLite backend. The MATLAB multi-backend discovery pattern + is not needed. + + - name: dataset_metadata + matlab_path: "+ndi/+database/+fun/dataset_metadata.m" + status: not_applicable + decision_log: > + MATLAB utility for reading/writing dataset metadata structures. + In Python, dataset metadata is handled natively via dict/JSON + serialization in the Document class. No separate utility needed. + + - name: openMINDSobj2ndi_document + matlab_path: "+ndi/+database/+fun/openMINDSobj2ndi_document.m" + status: not_applicable + decision_log: > + MATLAB function for converting openMINDS objects to NDI documents. + In Python, this functionality is handled by the dedicated + ndi/openminds_convert.py module which provides a more Pythonic + approach to openMINDS integration. + + - name: openMINDSobj2struct + matlab_path: "+ndi/+database/+fun/openMINDSobj2struct.m" + status: not_applicable + decision_log: > + MATLAB function for converting openMINDS objects to MATLAB structs. + In Python, openMINDS objects are naturally represented as dicts. + The ndi/openminds_convert.py module handles conversion directly. + + - name: plotinteractivedocgraph + matlab_path: "+ndi/+database/+fun/plotinteractivedocgraph.m" + status: not_applicable + decision_log: > + MATLAB interactive GUI for visualizing document dependency graphs. + Uses MATLAB-specific figure/axes/callback infrastructure. Python + users can use networkx + matplotlib or graphviz for equivalent + visualization from the docs2graph adjacency dict output. + + - name: projectvardef + matlab_path: "+ndi/+database/+fun/projectvardef.m" + status: not_applicable + decision_log: > + MATLAB utility for defining project-level variable definitions + used in the MATLAB workspace/project system. No equivalent + concept in the Python package; configuration is handled via + JSON/YAML files and ndi.common settings. + + - name: readtablechar + matlab_path: "+ndi/+database/+fun/readtablechar.m" + status: not_applicable + decision_log: > + MATLAB utility for reading table data from character-delimited + files into MATLAB table objects. Python equivalent is + pandas.read_csv() which is more capable and widely used. + + - name: writetablechar + matlab_path: "+ndi/+database/+fun/writetablechar.m" + status: not_applicable + decision_log: > + MATLAB utility for writing MATLAB table objects to + character-delimited files. Python equivalent is + pandas.DataFrame.to_csv() which is more capable and widely used.