⚡ Bolt: [performance improvement] Optimize solution lookup from O(N*M) to O(N)#149
⚡ Bolt: [performance improvement] Optimize solution lookup from O(N*M) to O(N)#149glacy wants to merge 1 commit into
Conversation
- Refactored `get_all_exercises` in `evolutia/material_extractor.py` to use a pre-computed dictionary for solution lookups instead of a nested loop. - Applied the same optimization to `index_materials` in `evolutia/rag/rag_indexer.py`. - Formatted modified files with Black and cleaned up unused variables using Ruff. Co-authored-by: glacy <1131951+glacy@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
There was a problem hiding this comment.
Pull request overview
This PR improves performance in the material extraction + RAG indexing pipeline by replacing per-exercise nested solution searches with a precomputed dictionary keyed by exercise_label, reducing solution matching from O(N*M) to O(N+M) per material.
Changes:
- Precompute
solutions_by_labelfor each material to enableO(1)solution lookup inMaterialExtractor.get_all_exercises. - Precompute
solutions_by_labelfor each material to enableO(1)solution lookup inRAGIndexer.index_materials. - Minor refactors/formatting updates (string quoting, typing imports, small cleanup).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| evolutia/material_extractor.py | Builds a per-material solutions_by_label dict to avoid nested loops when attaching solutions to exercises. |
| evolutia/rag/rag_indexer.py | Builds a per-material solutions_by_label dict to avoid nested loops when indexing exercises with their solutions. |
Comments suppressed due to low confidence (1)
evolutia/material_extractor.py:333
- La validación de caché puede devolver True aunque el archivo haya cambiado, porque compara
file_mtimecontra_last_scan_timestamp(máximo global) en vez del timestamp almacenado para ese archivo. Esto puede hacer que se sirva contenido obsoleto cuando otro archivo fue cacheado con un mtime mayor.
file_mtime = file_path.stat().st_mtime
# Usar el timestamp de escaneo más reciente para verificar
if file_mtime > self._last_scan_timestamp:
return False
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| collection_name = self.collection.name | ||
| self.client.delete_collection(name=collection_name) | ||
| _ = self.config.get("vector_store", {}) | ||
| self.collection = self.client.create_collection( | ||
| name=collection_name, metadata={"hnsw:space": "cosine"} | ||
| ) | ||
| logger.info(f"Colección {collection_name} limpiada") |
💡 What: Replaced nested loops searching for solutions
O(N*M)with a pre-computed dictionary lookupO(N)inevolutia/material_extractor.pyandevolutia/rag/rag_indexer.py.🎯 Why: Extracting and indexing materials involves looping over large arrays of exercises and solutions. Nested loops can become a performance bottleneck as the size of the curriculum grows.
📊 Impact: Changes the time complexity of matching solutions from
O(N*M)toO(N+M). This will significantly speed up file parsing and RAG indexing.🔬 Measurement: Run
pyteston the test suite to verify no regressions in material extraction or RAG indexing logic. Baseline tests remain fully compliant.PR created automatically by Jules for task 7467297981266915265 started by @glacy