Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/bolt.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,7 @@
## 2024-03-16 - Batch JSON File I/O Operations
**Learning:** When using JSON file-backed repositories, iterating over items sequentially and calling `save_item` or `record_attempt` inside a loop leads to N+1 file read/write operations. This creates a significant performance bottleneck, especially when importing progress snapshots with numerous items and attempts.
**Action:** Prefer batch processing methods (e.g., `save_items`, `record_attempts`) so file-backed adapters can load storage once, update it in memory, and write it back in a single pass.

## 2025-03-27 - Batch JSON File I/O Updates
**Learning:** In JSON file-backed adapters, deserializing all stored records to domain models and then reserializing them inside `save_items` causes an O(N) CPU/memory penalty.
**Action:** When updating JSON storage for batch methods like `save_items`, always update the raw dictionary entries in-place.
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,24 @@ def save_items(self, items: list[LearningItem]) -> None:

storage = self._load_storage()
raw_items = storage.get("items", [])
existing_items = []
if isinstance(raw_items, list):
existing_items = [
_item_from_dict(entry) for entry in raw_items if isinstance(entry, dict)
]
by_id = {existing.id: existing for existing in existing_items}
for item in items:
by_id[item.id] = item
storage["items"] = [_item_to_dict(entry) for entry in by_id.values()]
if not isinstance(raw_items, list):
raw_items = []

# ⚑ Bolt: Optimize batch saves by updating raw dicts in-place
# instead of full deserialize/serialize loop (avoids O(N) penalty)
new_items_by_id = {str(item.id): _item_to_dict(item) for item in items}

for entry in raw_items:
if isinstance(entry, dict):
entry_id = str(entry.get("id"))
if entry_id in new_items_by_id:
entry.clear()
entry.update(new_items_by_id.pop(entry_id))
Comment on lines +59 to +61

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update every matching entry before popping new item

If storage["items"] contains duplicate records with the same id, this loop updates only the first one and then removes the pending update via pop, leaving later duplicates stale. That is a behavior regression from the previous implementation, which rebuilt a by-id map and collapsed duplicates on each save; with the new logic, list_items() can return conflicting versions of the same item and downstream code that materializes {item.id: item} may end up using an outdated record.

Useful? React with πŸ‘Β / πŸ‘Ž.

Comment on lines +58 to +61

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current logic str(entry.get("id")) converts a missing ID (None) to the string "None". This could lead to a bug where a new item with id="None" overwrites an existing item that is missing an ID in the JSON file. It's safer to explicitly skip entries that do not have an ID to prevent incorrect data mapping.

Suggested change
entry_id = str(entry.get("id"))
if entry_id in new_items_by_id:
entry.clear()
entry.update(new_items_by_id.pop(entry_id))
entry_id_val = entry.get("id")
if entry_id_val is None:
continue
entry_id = str(entry_id_val)
if entry_id in new_items_by_id:
entry.clear()
entry.update(new_items_by_id.pop(entry_id))


for new_item in new_items_by_id.values():
raw_items.append(new_item)

storage["items"] = raw_items
self._save_storage(storage)

def list_attempts(self) -> list[Attempt]:
Expand Down
Loading