Skip to content

⚡ Bolt: [performance improvement] Optimize nested loops in material extraction and RAG indexing#135

Open
glacy wants to merge 1 commit into
mainfrom
perf/optimize-nested-loops-14618264829651679475
Open

⚡ Bolt: [performance improvement] Optimize nested loops in material extraction and RAG indexing#135
glacy wants to merge 1 commit into
mainfrom
perf/optimize-nested-loops-14618264829651679475

Conversation

@glacy

@glacy glacy commented Jun 16, 2026

Copy link
Copy Markdown
Owner

💡 What:

Converted O(N*M) nested loop searches in evolutia/material_extractor.py (get_all_exercises) and evolutia/rag/rag_indexer.py (index_materials) into O(N) hash map lookups by pre-computing solution_map dictionaries.

🎯 Why:

When iterating over exercises and repeatedly searching through a list of solutions for a matching label, the time complexity scales quadratically (O(N*M)). For repositories with large amounts of exercises and solutions, this becomes a noticeable bottleneck during extraction and RAG indexing.

📊 Impact:

Reduces the time complexity of exercise-to-solution matching from quadratic O(N*M) to linear O(N), significantly speeding up extraction and index generation pipelines.

🔬 Measurement:

To verify the improvement, generate materials or index using a large subset of exercises. Performance profiles before and after will show a stark decrease in cumulative execution time within get_all_exercises and index_materials functions. Testing verified the exact same output generation correctly preserving first-match semantics.


PR created automatically by Jules for task 14618264829651679475 started by @glacy

…dict lookups

- Convert nested O(N*M) list searches in `get_all_exercises` and `index_materials` to O(1) hash map lookups.
- Preserves the first-match logic of the original `break` statement by validating `if key not in dict`.
- Cleans up unused variables in `rag_indexer.py` and `material_extractor.py` caught by Ruff.

Co-authored-by: glacy <1131951+glacy@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Copilot AI review requested due to automatic review settings June 16, 2026 18:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Este PR busca mejorar el rendimiento al eliminar búsquedas O(N*M) al asociar ejercicios con sus soluciones durante la extracción de materiales y el indexado RAG, reemplazándolas por lookups O(1) mediante diccionarios precomputados, preservando la semántica de “primera coincidencia”.

Changes:

  • Optimiza el matching ejercicio→solución en MaterialExtractor.get_all_exercises usando un solution_map por material.
  • Optimiza el matching ejercicio→solución en RAGIndexer.index_materials usando un solution_map por material.
  • Reformat/normaliza estilo (comillas, wrapping, etc.) en los módulos afectados.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.

File Description
tests/test_exercise_analyzer.py Sin cambios funcionales relevantes; principalmente normalización de diff/format.
evolutia/rag/rag_indexer.py Introduce solution_map para evitar nested loops en indexación; incluye refactors/formatting.
evolutia/material_extractor.py Introduce solution_map para evitar nested loops al consolidar ejercicios; incluye refactors/formatting.
Comments suppressed due to low confidence (1)

evolutia/material_extractor.py:336

  • La validación de caché usa _last_scan_timestamp (global) en lugar del timestamp del entry para este archivo, por lo que puede devolver datos obsoletos si un archivo cambia con mtime menor al de otro archivo escaneado. Además, el acceso _= self._file_cache[file_path] no aporta nada. Conviene validar por mtime del propio archivo y, si el stat() falla, usar el TTL para mantener el caché de errores y evitar reintentos constantes.
        if file_path not in self._file_cache:
            return False

        # Verificar si el archivo fue modificado
        try:
            _ = self._file_cache[file_path]
            file_mtime = file_path.stat().st_mtime

            # Usar el timestamp de escaneo más reciente para verificar
            if file_mtime > self._last_scan_timestamp:
                return False

            return True

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +288 to +295
# Generar embeddings
embeddings = self._generate_embeddings_batch(chunks)

# Sincronizar chunks con embeddings (por si se filtraron vacíos en _generate_embeddings_batch)
# Aunque aquí preferimos filtrar antes para mantener consistencia
valid_indices = [i for i, chunk in enumerate(chunks) if chunk and chunk.strip()]
chunks = [chunks[i] for i in valid_indices]

Comment on lines +348 to +354
# Generar embeddings
embeddings = self._generate_embeddings_batch(chunks)

# Sincronizar chunks con embeddings
valid_indices = [i for i, chunk in enumerate(chunks) if chunk and chunk.strip()]
chunks = [chunks[i] for i in valid_indices]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants