⚡ Bolt: O(1) dictionary lookup for exercises and solutions#138
Conversation
Replace O(N*M) nested loops with O(N) hash map lookup for matching solutions to exercises in material_extractor.py and rag_indexer.py. Co-authored-by: glacy <1131951+glacy@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
There was a problem hiding this comment.
Pull request overview
This PR improves performance when matching solutions to exercises by replacing nested-loop searches with per-material dictionaries, aiming to reduce extraction/indexing time for large material sets while preserving “first-match” semantics.
Changes:
- Refactors solution lookup in
MaterialExtractor.get_all_exercisesto use an O(1) map keyed byexercise_label. - Refactors solution lookup in
RAGIndexer.index_materialsto use an O(1) solution map instead of scanning solutions per exercise. - Applies broad formatting/quoting cleanups in both modules.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
evolutia/rag/rag_indexer.py |
Uses a per-material solution_map for O(1) solution matching during indexing; includes formatting cleanups. |
evolutia/material_extractor.py |
Uses a per-material solution_map for O(1) solution matching when aggregating exercises; includes formatting cleanups. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Verificar si el archivo fue modificado | ||
| try: | ||
| cache_entry = self._file_cache[file_path] | ||
| _ = self._file_cache[file_path] | ||
| file_mtime = file_path.stat().st_mtime | ||
|
|
| elif self.embedding_provider == "sentence-transformers": | ||
| return self.embedding_model.encode( | ||
| texts, show_progress_bar=True, batch_size=32 | ||
| ).tolist() |
| collection_name = self.collection.name | ||
| self.client.delete_collection(name=collection_name) | ||
| _ = self.config.get("vector_store", {}) | ||
| self.collection = self.client.create_collection( | ||
| name=collection_name, metadata={"hnsw:space": "cosine"} | ||
| ) | ||
| logger.info(f"Colección {collection_name} limpiada") |
💡 What: Replaced O(NM) nested loops with O(N) hash map lookup when matching solutions to exercises in
MaterialExtractor.get_all_exercisesandRAGIndexer.index_materials.🎯 Why: The original nested loop iterated through all solutions for each exercise, causing an N^2 performance scaling issue when parsing large assignment sets.
📊 Impact: Expected to significantly reduce extraction and indexing times for directories containing many exercises and solutions. Scales linearly O(N) instead of O(NM).
🔬 Measurement: Verified functionality using test suite. No logic changed. First-match
breaklogic from the nested loop is perfectly mirrored usingif sol["exercise_label"] not in solution_map:when populating the dictionary.PR created automatically by Jules for task 12829625720123419232 started by @glacy