|
| 1 | +# DID MATLAB-to-Python Porting Instructions |
| 2 | + |
| 3 | +This document describes how to keep DID-python synchronized with DID-matlab |
| 4 | +using the YAML bridge files. |
| 5 | + |
| 6 | +## Bridge Files |
| 7 | + |
| 8 | +The bridge files live in `src/did/` and define the contract between MATLAB and |
| 9 | +Python implementations: |
| 10 | + |
| 11 | +| Bridge file | Scope | |
| 12 | +|---|---| |
| 13 | +| `did_matlab_python_bridge.yaml` | Core classes: database, document, query, ido, documentservice, binarydoc | |
| 14 | +| `did_matlab_python_bridge_implementations.yaml` | Implementation classes: sqlitedb, doc2sql, binarydoc_matfid | |
| 15 | +| `did_matlab_python_bridge_file.yaml` | File I/O: fileobj, readonly_fileobj, binaryTable, utilities | |
| 16 | +| `did_matlab_python_bridge_util.yaml` | Utilities: databaseSummary, compareDatabaseSummary, fun, datastructures, db, common | |
| 17 | + |
| 18 | +## Checking for Drift |
| 19 | + |
| 20 | +To check whether a MATLAB file has changed since the last Python sync, use the |
| 21 | +`matlab_last_sync_hash` field from the bridge YAML: |
| 22 | + |
| 23 | +```bash |
| 24 | +# For a single file: |
| 25 | +git -C /path/to/DID-matlab log <matlab_last_sync_hash>..HEAD -- <matlab_path> |
| 26 | + |
| 27 | +# Example: |
| 28 | +git -C /path/to/DID-matlab log 205d34b..HEAD -- src/did/+did/+file/fileobj.m |
| 29 | +``` |
| 30 | + |
| 31 | +If the command produces output, the MATLAB file has changed since the last port. |
| 32 | + |
| 33 | +### Bulk drift check |
| 34 | + |
| 35 | +Run this to check all bridge files at once: |
| 36 | + |
| 37 | +```bash |
| 38 | +cd /path/to/DID-matlab |
| 39 | +for yaml in /path/to/DID-python/src/did/did_matlab_python_bridge*.yaml; do |
| 40 | + echo "=== $(basename $yaml) ===" |
| 41 | + # Extract matlab_path and matlab_last_sync_hash pairs |
| 42 | + python3 -c " |
| 43 | +import yaml, sys |
| 44 | +with open('$yaml') as f: |
| 45 | + data = yaml.safe_load(f) |
| 46 | +for section in ['classes', 'functions']: |
| 47 | + for item in data.get(section, []): |
| 48 | + path = item.get('matlab_path', '') |
| 49 | + sync_hash = item.get('matlab_last_sync_hash', '') |
| 50 | + name = item.get('name', '') |
| 51 | + if path and sync_hash: |
| 52 | + print(f'{name}|src/did/{path}|{sync_hash}') |
| 53 | +" | while IFS='|' read name path hash; do |
| 54 | + changes=$(git log --oneline "$hash"..HEAD -- "$path" 2>/dev/null) |
| 55 | + if [ -n "$changes" ]; then |
| 56 | + echo " DRIFT: $name ($path)" |
| 57 | + echo "$changes" | sed 's/^/ /' |
| 58 | + fi |
| 59 | + done |
| 60 | +done |
| 61 | +``` |
| 62 | + |
| 63 | +## Porting a MATLAB Change to Python |
| 64 | + |
| 65 | +### Step 1: Identify the change |
| 66 | + |
| 67 | +```bash |
| 68 | +git -C /path/to/DID-matlab log <sync_hash>..HEAD -- src/did/<matlab_path> |
| 69 | +git -C /path/to/DID-matlab diff <sync_hash>..HEAD -- src/did/<matlab_path> |
| 70 | +``` |
| 71 | + |
| 72 | +### Step 2: Locate the Python counterpart |
| 73 | + |
| 74 | +Use the bridge YAML to find `python_path` and `python_class` / `python_name`. |
| 75 | + |
| 76 | +### Step 3: Apply the change |
| 77 | + |
| 78 | +Follow these conventions when porting: |
| 79 | + |
| 80 | +| MATLAB | Python | |
| 81 | +|---|---| |
| 82 | +| `camelCase` method names | `snake_case` method names | |
| 83 | +| `struct` | `dict` | |
| 84 | +| `cell array` | `list` | |
| 85 | +| `char` / `string` | `str` | |
| 86 | +| `logical` | `bool` | |
| 87 | +| `[]` (empty) | `None` or `[]` depending on context | |
| 88 | +| `nargin`, `varargin` | `*args`, `**kwargs` | |
| 89 | +| `arguments` block | Type hints + validation | |
| 90 | +| Name-value pairs | `**kwargs` | |
| 91 | +| 1-based indexing | 0-based indexing | |
| 92 | + |
| 93 | +### Step 4: Update the bridge YAML |
| 94 | + |
| 95 | +After porting, update the entry in the bridge YAML: |
| 96 | + |
| 97 | +1. Set `matlab_last_sync_hash` to the current MATLAB commit hash for that file: |
| 98 | + ```bash |
| 99 | + git -C /path/to/DID-matlab log -1 --format="%h" -- src/did/<matlab_path> |
| 100 | + ``` |
| 101 | +2. Remove `matlab_current_hash` and `out_of_sync` / `out_of_sync_reason` if present. |
| 102 | +3. Update the `decision_log` with the sync date. |
| 103 | + |
| 104 | +### Step 5: Run symmetry tests |
| 105 | + |
| 106 | +```bash |
| 107 | +# Python tests |
| 108 | +pytest -m make_artifacts -v |
| 109 | +pytest -m read_artifacts -v |
| 110 | +``` |
| 111 | + |
| 112 | +If MATLAB is available, run the full 3-step symmetry cycle: |
| 113 | +1. MATLAB `makeArtifacts` tests |
| 114 | +2. Python `makeArtifacts` + `readArtifacts` tests |
| 115 | +3. MATLAB `readArtifacts` tests |
| 116 | + |
| 117 | +## Bridge YAML Field Reference |
| 118 | + |
| 119 | +| Field | Required | Description | |
| 120 | +|---|---|---| |
| 121 | +| `name` | Yes | MATLAB function/class name | |
| 122 | +| `type` | Yes | `class` or `function` | |
| 123 | +| `matlab_path` | Yes | Path relative to `src/did/` in DID-matlab | |
| 124 | +| `matlab_last_sync_hash` | Yes | Short SHA of the MATLAB commit last ported to Python | |
| 125 | +| `matlab_current_hash` | No | Current MATLAB hash when out of sync (for tracking) | |
| 126 | +| `python_path` | Yes | Path relative to `src/did/` in DID-python | |
| 127 | +| `python_class` | If class | Python class name | |
| 128 | +| `python_name` | If function | Python function name | |
| 129 | +| `inherits_matlab` | No | MATLAB parent class(es) | |
| 130 | +| `inherits_python` | No | Python parent class(es) | |
| 131 | +| `out_of_sync` | No | `true` if MATLAB has diverged | |
| 132 | +| `out_of_sync_reason` | No | Human-readable explanation of the divergence | |
| 133 | +| `decision_log` | Yes | Sync status, dates, deviation rationale | |
| 134 | +| `properties` | No | List of property mappings | |
| 135 | +| `methods` | No | List of method mappings | |
| 136 | + |
| 137 | +## Adding a New MATLAB File |
| 138 | + |
| 139 | +When a new file is added to DID-matlab that needs a Python counterpart: |
| 140 | + |
| 141 | +1. Create the Python implementation following the conventions above. |
| 142 | +2. Add an entry to the appropriate bridge YAML file. |
| 143 | +3. Set `matlab_last_sync_hash` to the MATLAB commit that introduced the file. |
| 144 | +4. Run symmetry tests to verify cross-language compatibility. |
| 145 | + |
| 146 | +## Current Sync Status |
| 147 | + |
| 148 | +As of 2026-04-13, the repositories are **in sync** for all core functionality. |
| 149 | + |
| 150 | +### Recently resolved (no Python changes needed) |
| 151 | +The following MATLAB changes (March 29-31, 2026) were verified to already be |
| 152 | +handled correctly by Python: |
| 153 | + |
| 154 | +- **fileobj / readonly_fileobj / binaryTable / binarydoc_matfid**: MATLAB |
| 155 | + changed default permission `'r'` -> `'rb'` for Linux binary-mode |
| 156 | + compatibility. Python's `Fileobj.fopen()` already appends `'b'` to the mode |
| 157 | + string if not present (line 88-89 of `file.py`), so all files are opened in |
| 158 | + binary mode regardless. **Behaviorally in sync.** |
| 159 | +- **fileobj fread**: MATLAB changed default precision from `'char'` to |
| 160 | + `'uint8'`. Python's `fread()` returns raw `bytes`, which is equivalent to |
| 161 | + `uint8`. **No change needed.** |
| 162 | +- **fileobj fwrite**: MATLAB updated permission check to allow `'r+'` mode. |
| 163 | + Python relies on native file objects to reject writes on read-only files. |
| 164 | + **No change needed.** |
| 165 | +- **mustBeValidPermission**: MATLAB added binary-mode variants. Python's |
| 166 | + `must_be_valid_permission()` already accepts `rb`, `wb`, `ab`, etc. |
| 167 | + **Already in sync.** |
| 168 | +- **sqlitedb**: MATLAB replaced `websave` with `ndi.cloud.api.files.getFile` |
| 169 | + for URL downloads. This is MATLAB-ecosystem-specific; Python uses its own |
| 170 | + download mechanism. **Not applicable to Python.** |
| 171 | + |
| 172 | +## Not Yet Ported from MATLAB |
| 173 | + |
| 174 | +These MATLAB features do not yet have Python counterparts: |
| 175 | + |
| 176 | +| MATLAB feature | Bridge file | Priority | |
| 177 | +|---|---|---| |
| 178 | +| `database.freeze_branch` | bridge.yaml | Low | |
| 179 | +| `database.is_branch_editable` | bridge.yaml | Low | |
| 180 | +| `database.display_branches` | bridge.yaml | Low | |
| 181 | +| `database.exist_doc` | bridge.yaml | Medium | |
| 182 | +| `database.close_doc` | bridge.yaml | Low | |
| 183 | +| `document.validate` | bridge.yaml | Medium | |
| 184 | +| `document.dependency_value_n` | bridge.yaml | Low | |
| 185 | +| `document.add_dependency_value_n` | bridge.yaml | Low | |
| 186 | +| `document.remove_dependency_value_n` | bridge.yaml | Low | |
| 187 | +| `binaryTable` write methods | bridge_file.yaml | Medium | |
0 commit comments