Skip to content

Commit ff6aa01

Browse files
committed
Add MATLAB-Python bridge YAML files and porting instructions
Introduce four bridge YAML files (following NDI-python pattern) that map every DID-matlab source file to its Python counterpart, tracking sync status via matlab_last_sync_hash. Also add PORTING_INSTRUCTIONS.md with drift-check commands, porting conventions, and current sync status. All recent MATLAB changes (binary-mode fixes, fread/fwrite updates) were verified to already be handled correctly by Python's Fileobj.fopen() which appends 'b' to mode strings automatically. https://claude.ai/code/session_015g95pQvxDUnTWxHYmTnreG
1 parent e871ffa commit ff6aa01

5 files changed

Lines changed: 1928 additions & 0 deletions

PORTING_INSTRUCTIONS.md

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
# DID MATLAB-to-Python Porting Instructions
2+
3+
This document describes how to keep DID-python synchronized with DID-matlab
4+
using the YAML bridge files.
5+
6+
## Bridge Files
7+
8+
The bridge files live in `src/did/` and define the contract between MATLAB and
9+
Python implementations:
10+
11+
| Bridge file | Scope |
12+
|---|---|
13+
| `did_matlab_python_bridge.yaml` | Core classes: database, document, query, ido, documentservice, binarydoc |
14+
| `did_matlab_python_bridge_implementations.yaml` | Implementation classes: sqlitedb, doc2sql, binarydoc_matfid |
15+
| `did_matlab_python_bridge_file.yaml` | File I/O: fileobj, readonly_fileobj, binaryTable, utilities |
16+
| `did_matlab_python_bridge_util.yaml` | Utilities: databaseSummary, compareDatabaseSummary, fun, datastructures, db, common |
17+
18+
## Checking for Drift
19+
20+
To check whether a MATLAB file has changed since the last Python sync, use the
21+
`matlab_last_sync_hash` field from the bridge YAML:
22+
23+
```bash
24+
# For a single file:
25+
git -C /path/to/DID-matlab log <matlab_last_sync_hash>..HEAD -- <matlab_path>
26+
27+
# Example:
28+
git -C /path/to/DID-matlab log 205d34b..HEAD -- src/did/+did/+file/fileobj.m
29+
```
30+
31+
If the command produces output, the MATLAB file has changed since the last port.
32+
33+
### Bulk drift check
34+
35+
Run this to check all bridge files at once:
36+
37+
```bash
38+
cd /path/to/DID-matlab
39+
for yaml in /path/to/DID-python/src/did/did_matlab_python_bridge*.yaml; do
40+
echo "=== $(basename $yaml) ==="
41+
# Extract matlab_path and matlab_last_sync_hash pairs
42+
python3 -c "
43+
import yaml, sys
44+
with open('$yaml') as f:
45+
data = yaml.safe_load(f)
46+
for section in ['classes', 'functions']:
47+
for item in data.get(section, []):
48+
path = item.get('matlab_path', '')
49+
sync_hash = item.get('matlab_last_sync_hash', '')
50+
name = item.get('name', '')
51+
if path and sync_hash:
52+
print(f'{name}|src/did/{path}|{sync_hash}')
53+
" | while IFS='|' read name path hash; do
54+
changes=$(git log --oneline "$hash"..HEAD -- "$path" 2>/dev/null)
55+
if [ -n "$changes" ]; then
56+
echo " DRIFT: $name ($path)"
57+
echo "$changes" | sed 's/^/ /'
58+
fi
59+
done
60+
done
61+
```
62+
63+
## Porting a MATLAB Change to Python
64+
65+
### Step 1: Identify the change
66+
67+
```bash
68+
git -C /path/to/DID-matlab log <sync_hash>..HEAD -- src/did/<matlab_path>
69+
git -C /path/to/DID-matlab diff <sync_hash>..HEAD -- src/did/<matlab_path>
70+
```
71+
72+
### Step 2: Locate the Python counterpart
73+
74+
Use the bridge YAML to find `python_path` and `python_class` / `python_name`.
75+
76+
### Step 3: Apply the change
77+
78+
Follow these conventions when porting:
79+
80+
| MATLAB | Python |
81+
|---|---|
82+
| `camelCase` method names | `snake_case` method names |
83+
| `struct` | `dict` |
84+
| `cell array` | `list` |
85+
| `char` / `string` | `str` |
86+
| `logical` | `bool` |
87+
| `[]` (empty) | `None` or `[]` depending on context |
88+
| `nargin`, `varargin` | `*args`, `**kwargs` |
89+
| `arguments` block | Type hints + validation |
90+
| Name-value pairs | `**kwargs` |
91+
| 1-based indexing | 0-based indexing |
92+
93+
### Step 4: Update the bridge YAML
94+
95+
After porting, update the entry in the bridge YAML:
96+
97+
1. Set `matlab_last_sync_hash` to the current MATLAB commit hash for that file:
98+
```bash
99+
git -C /path/to/DID-matlab log -1 --format="%h" -- src/did/<matlab_path>
100+
```
101+
2. Remove `matlab_current_hash` and `out_of_sync` / `out_of_sync_reason` if present.
102+
3. Update the `decision_log` with the sync date.
103+
104+
### Step 5: Run symmetry tests
105+
106+
```bash
107+
# Python tests
108+
pytest -m make_artifacts -v
109+
pytest -m read_artifacts -v
110+
```
111+
112+
If MATLAB is available, run the full 3-step symmetry cycle:
113+
1. MATLAB `makeArtifacts` tests
114+
2. Python `makeArtifacts` + `readArtifacts` tests
115+
3. MATLAB `readArtifacts` tests
116+
117+
## Bridge YAML Field Reference
118+
119+
| Field | Required | Description |
120+
|---|---|---|
121+
| `name` | Yes | MATLAB function/class name |
122+
| `type` | Yes | `class` or `function` |
123+
| `matlab_path` | Yes | Path relative to `src/did/` in DID-matlab |
124+
| `matlab_last_sync_hash` | Yes | Short SHA of the MATLAB commit last ported to Python |
125+
| `matlab_current_hash` | No | Current MATLAB hash when out of sync (for tracking) |
126+
| `python_path` | Yes | Path relative to `src/did/` in DID-python |
127+
| `python_class` | If class | Python class name |
128+
| `python_name` | If function | Python function name |
129+
| `inherits_matlab` | No | MATLAB parent class(es) |
130+
| `inherits_python` | No | Python parent class(es) |
131+
| `out_of_sync` | No | `true` if MATLAB has diverged |
132+
| `out_of_sync_reason` | No | Human-readable explanation of the divergence |
133+
| `decision_log` | Yes | Sync status, dates, deviation rationale |
134+
| `properties` | No | List of property mappings |
135+
| `methods` | No | List of method mappings |
136+
137+
## Adding a New MATLAB File
138+
139+
When a new file is added to DID-matlab that needs a Python counterpart:
140+
141+
1. Create the Python implementation following the conventions above.
142+
2. Add an entry to the appropriate bridge YAML file.
143+
3. Set `matlab_last_sync_hash` to the MATLAB commit that introduced the file.
144+
4. Run symmetry tests to verify cross-language compatibility.
145+
146+
## Current Sync Status
147+
148+
As of 2026-04-13, the repositories are **in sync** for all core functionality.
149+
150+
### Recently resolved (no Python changes needed)
151+
The following MATLAB changes (March 29-31, 2026) were verified to already be
152+
handled correctly by Python:
153+
154+
- **fileobj / readonly_fileobj / binaryTable / binarydoc_matfid**: MATLAB
155+
changed default permission `'r'` -> `'rb'` for Linux binary-mode
156+
compatibility. Python's `Fileobj.fopen()` already appends `'b'` to the mode
157+
string if not present (line 88-89 of `file.py`), so all files are opened in
158+
binary mode regardless. **Behaviorally in sync.**
159+
- **fileobj fread**: MATLAB changed default precision from `'char'` to
160+
`'uint8'`. Python's `fread()` returns raw `bytes`, which is equivalent to
161+
`uint8`. **No change needed.**
162+
- **fileobj fwrite**: MATLAB updated permission check to allow `'r+'` mode.
163+
Python relies on native file objects to reject writes on read-only files.
164+
**No change needed.**
165+
- **mustBeValidPermission**: MATLAB added binary-mode variants. Python's
166+
`must_be_valid_permission()` already accepts `rb`, `wb`, `ab`, etc.
167+
**Already in sync.**
168+
- **sqlitedb**: MATLAB replaced `websave` with `ndi.cloud.api.files.getFile`
169+
for URL downloads. This is MATLAB-ecosystem-specific; Python uses its own
170+
download mechanism. **Not applicable to Python.**
171+
172+
## Not Yet Ported from MATLAB
173+
174+
These MATLAB features do not yet have Python counterparts:
175+
176+
| MATLAB feature | Bridge file | Priority |
177+
|---|---|---|
178+
| `database.freeze_branch` | bridge.yaml | Low |
179+
| `database.is_branch_editable` | bridge.yaml | Low |
180+
| `database.display_branches` | bridge.yaml | Low |
181+
| `database.exist_doc` | bridge.yaml | Medium |
182+
| `database.close_doc` | bridge.yaml | Low |
183+
| `document.validate` | bridge.yaml | Medium |
184+
| `document.dependency_value_n` | bridge.yaml | Low |
185+
| `document.add_dependency_value_n` | bridge.yaml | Low |
186+
| `document.remove_dependency_value_n` | bridge.yaml | Low |
187+
| `binaryTable` write methods | bridge_file.yaml | Medium |

0 commit comments

Comments
 (0)