📦 Optional reranker deps + lazy langchain_classic.agents import - #2
Merged
Merged
Conversation
… import Move sentence-transformers/torch/transformers out of the default backend install into requirements-rerank.txt (opt-in via USE_RERANKER). Also defer the langchain_classic.agents import to first agent invocation instead of process startup, since it eagerly loads every legacy agent implementation. Combined, this cuts routers.chat import time from ~7.0s to ~1.6s. The Docker image built from the default install still shipped with USE_RERANKER=true, so a container could come up cleanly and fail reranking at request time instead of at build/boot. Add an INSTALL_RERANK build arg (default false) that conditionally installs requirements-rerank.txt in the Dockerfile.backend builder stage, wire it through docker-compose.yml's build.args, and document the INSTALL_RERANK/USE_RERANKER pairing in backend/.env.example and the README so the two flags can't silently drift out of sync.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Πρόβλημα
Το
import routers.chatέπαιρνε ~7s. Η αλυσίδα ήτανagent/executor.py → langchain_classic.agents → sentence_transformers → transformers → torch. Παράλληλα, τα βαριά ML πακέτα ήταν υποχρεωτικά στοdefault install ενώ ο reranker είναι opt-in (
USE_RERANKER=false).Αλλαγές
backend/requirements-rerank.txt(νέο):sentence-transformers>=3.0.0.Τα torch/transformers/tokenizers έρχονται transitively - δεν υπήρχαν ποτέ
explicit pins.
backend/requirements.txt: αφαίρεση της γραμμής + pointer στο optional file.rag/reranker.py: ImportError → RuntimeError με το ακριβές install command.Τα υπόλοιπα exceptions γίνονται
log.errorμε ρητή αναφορά ότι τοretrieval συνεχίζει χωρίς reranking.
agent/executor.py: τοfrom langchain_classic.agents import ...μετακινήθηκε μέσα στο
_build_agent_executor(), με TYPE_CHECKING guard.Dockerfile.backend:ARG INSTALL_RERANK=false, conditional install τουrequirements-rerank.txt στο builder stage.
docker-compose.yml:build.args.INSTALL_RERANK: ${INSTALL_RERANK:-false}.backend/.env.example+ README: τεκμηρίωση του build-time/runtimeζεύγους INSTALL_RERANK / USE_RERANKER.
backend/tests/test_optional_reranker.py(νέο, 3 tests): AST check ότι δενυπάρχει module-level
langchain_classicimport, import με μπλοκαρισμένοsentence_transformersμέσω meta_path, RuntimeError χωρίς το package.Αποτέλεσμα
import routers.chat: 7.00s → 1.59s (-77%). Καθαρό venv με το νέοrequirements.txt: 134 πακέτα, κανένα torch/transformers/sentence-transformers.
Σημείωση
Η αρχική υπόθεση ότι έφταιγε το
langchain_text_splitters.sentence_transformersήταν λάθος - έχει ήδη try/except ImportError. Η πραγματική καθυστέρηση ήταν ότι
το
langchain_classic/agents/__init__.py(5.66s) φορτώνει eagerly όλους τουςlegacy agent types, οπότε narrower import path δεν βοηθάει. Η λύση ήταν το lazy
import, όχι μόνο η αφαίρεση του sentence-transformers.
Επαλήθευση
docker compose build backendχωρίς override → κανένα torch στο image.