Port utility functions, validators, and enforce strict MATLAB naming#15
Merged
Merged
Conversation
MATLAB function names are the canonical reference. Renames: - save_to_file → savetofile (epochprobemap_daqsystem) - load_from_file → loadfromfile (epochprobemap_daqsystem) - clear_cache → resetepochtable (epochset) - find_epoch_node → findepochnode (epoch/functions) - init_probe_type_map → initProbeTypeMap (probe) - get_probe_type_map → getProbeTypeMap (probe) - spikes_for_probe → spikesForProbe (element/functions) Also adds probestring() method to Probe class (exists in MATLAB probe.m but was missing from Python port), and updates PYTHON_PORTING_GUIDE.md with the "MATLAB function names are the source of truth" instruction. https://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp
- Add Namespace Coverage Status section to PYTHON_PORTING_GUIDE.md with verified per-submodule coverage table for ndi.cloud.api (100% across datasets, documents, files, users, compute, auth) - Fix MATLAB_MAPPING.md: undeleteDataset, listDeletedDatasets, and listDeletedDocuments exist in MATLAB (not Python-only) - Fix MATLAB_MAPPING.md: duplicateDocuments is ported (internal.py) https://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp
…TLAB ndi.validators: All 11 MATLAB argument-block validators ported 1:1 - mustBeCellArrayOfClass, mustBeCellArrayOfNdiSessions, mustBeCellArrayOfNonEmptyCharacterArrays, mustBeClassnameOfType, mustBeEpochInput, mustBeID, mustBeNumericClass, mustBeTextLike, mustHaveFields, mustHaveRequiredColumns, mustMatchRegex ndi.util: All 8 portable utility functions ported 1:1 - datestamp2datetime, downsampleTimeseries, hexDiff, hexDiffBytes, getHexDiffFromFileObj, hexDump, rehydrateJSONNanNull, unwrapTableCellContent - GUI-only (choosefile, choosefileordir) and MATLAB-specific (toolboxdir) intentionally skipped Also updates MATLAB_MAPPING.md and PYTHON_PORTING_GUIDE.md with coverage tables for both namespaces. https://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp
Per PYTHON_PORTING_GUIDE Rule 4, all public-facing API functions must use @pydantic.validate_call to mirror MATLAB arguments blocks. Replaces manual isinstance checks with Pydantic validation. Uses Annotated + pydantic.Field for constraints (ge=0, gt=0) and arbitrary_types_allowed=True where numpy/IO types are needed. https://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp
…uide Create ndi.validators.is_ndarray and ndi.validators.is_iso8601 as reusable validation functions following the MATLAB +ndi/+validators/ pattern. Update PYTHON_PORTING_GUIDE Section 4 with new subsection 4a documenting the requirement to centralise custom validators in ndi/validators/ rather than writing inline checks. https://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp
Add Section 5 (Code Style & Linting) to PYTHON_PORTING_GUIDE requiring black --check before every commit. Run black to fix the 12 files that were out of compliance. https://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp
Add ruff check requirement alongside black in Section 5 of PYTHON_PORTING_GUIDE. Fix unused `math` import in unwrapTableCellContent.py caught by ruff. https://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR ports a comprehensive set of utility functions and validators from the MATLAB NDI codebase to Python, while enforcing strict adherence to MATLAB function naming conventions. The changes include new modules for hex diffing/dumping, timeseries downsampling, JSON rehydration, and a complete validator framework that mirrors MATLAB's
argumentsblock validation.Key Changes
New Utility Functions (
ndi.util)hexDiff/hexDump/hexDiffBytes: Hex comparison and display utilities for binary file inspectiongetHexDiffFromFileObj: Chunk-by-chunk comparison of file-like objectsdownsampleTimeseries: Anti-aliasing low-pass filter + downsampling using scipydatestamp2datetime: ISO 8601 datestamp parsing to PythondatetimerehydrateJSONNanNull: Sentinel string replacement for NaN/Infinity in JSONunwrapTableCellContent: Recursive unwrapping of nested lists (MATLAB cell arrays)New Validator Framework (
ndi.validators)Comprehensive validation functions mirroring MATLAB's custom validators:
mustBeID(33-char format with underscore at position 17)mustBeNumericClass,mustBeClassnameOfType,is_ndarraymustBeCellArrayOfClass,mustBeCellArrayOfNonEmptyCharacterArrays,mustBeCellArrayOfNdiSessionsmustHaveFields,mustHaveRequiredColumnsmustBeEpochInput,mustBeTextLike,mustMatchRegex,is_iso8601Strict MATLAB Naming Enforcement
Renamed functions to match MATLAB source exactly (no snake_case translation):
find_epoch_node→findepochnodespikes_for_probe→spikesForProbeclear_cache→resetepochtablesave_to_file→savetofileload_from_file→loadfromfileinit_probe_type_map→initProbeTypeMapget_probe_type_map→getProbeTypeMapUpdated
PYTHON_PORTING_GUIDE.mdto emphasize that MATLAB function names are the source of truth and must not be translated to PEP 8 conventions.Implementation Details
@pydantic.validate_callfor robust input validationAnnotated[type, pydantic.Field(...)]for constraints (e.g.,gt=0,ge=0)_format_chunk,_print_header) are shared betweenhexDiffandhexDumpTest Updates
Updated test files to use new function names:
test_batch_e.py: Updatedfind_epoch_nodereferencestest_batch_a.py: Updatedsave_to_file/load_from_filereferencestest_probe.py: Updated probe type map function namestest_element.py: Updated element function namesDocumentation Updates
MATLAB_MAPPING.md: Corrected probe function mappings and soft-delete API entriesPYTHON_PORTING_GUIDE.md: Added section 4a on reusable validators and clarified naming rules with verification guidancehttps://claude.ai/code/session_016oPH9EoxCLcsUSTpgN9iZp