⚡ Bolt: In-place Dictionary Mutations for JSON File Adapters

.jules/bolt.md

@@ -13,3 +13,7 @@
 ## 2024-03-16 - Batch JSON File I/O Operations
 **Learning:** When using JSON file-backed repositories, iterating over items sequentially and calling `save_item` or `record_attempt` inside a loop leads to N+1 file read/write operations. This creates a significant performance bottleneck, especially when importing progress snapshots with numerous items and attempts.
 **Action:** Prefer batch processing methods (e.g., `save_items`, `record_attempts`) so file-backed adapters can load storage once, update it in memory, and write it back in a single pass.
+
+## 2025-03-22 - Optimize JSON file-backed adapters with in-place dict updates
+**Learning:** Fully deserializing and reserializing stored records to domain models for batch write operations in JSON file-backed adapters causes a massive $O(N)$ CPU/memory penalty. Iterating through the large stored JSON payload to construct heavy domain model lists before mutating values is very inefficient compared to directly mutating the dictionary objects in place.
+**Action:** Always update raw dictionary entries directly in-place rather than instantiating entire domain models, mapping them back, and overwriting, significantly boosting serialization/write speeds.

src/python_learning_orchestrated/adapters/json_file_practice_repository.py

@@ -46,15 +46,22 @@ def save_items(self, items: list[LearningItem]) -> None:
 
         storage = self._load_storage()
         raw_items = storage.get("items", [])
-        existing_items = []
-        if isinstance(raw_items, list):
-            existing_items = [
-                _item_from_dict(entry) for entry in raw_items if isinstance(entry, dict)
-            ]
-        by_id = {existing.id: existing for existing in existing_items}
-        for item in items:
-            by_id[item.id] = item
-        storage["items"] = [_item_to_dict(entry) for entry in by_id.values()]
+        if not isinstance(raw_items, list):
+            raw_items = []
+
+        items_to_save = {str(item.id): _item_to_dict(item) for item in items}
+
+        updated_raw_items = []
+        for entry in raw_items:
+            if isinstance(entry, dict):
+                entry_id = str(entry.get("id"))
+                if entry_id in items_to_save:
+                    updated_raw_items.append(items_to_save.pop(entry_id))
+                else:
+                    updated_raw_items.append(entry)
+
+        updated_raw_items.extend(items_to_save.values())
+        storage["items"] = updated_raw_items
         self._save_storage(storage)
 
     def list_attempts(self) -> list[Attempt]: