Add CIJ_PROXY global download proxy (explicit config injection)

.env.template

@@ -12,6 +12,11 @@ COPILOTJ_VLM_API_KEY=sk-xxxxxxxx
 # When enabled, COPILOTJ_VLM_MODEL and COPILOTJ_VLM_API_KEY must be set.
 # COPILOTJ_VISION_ENABLED=1
 # COPILOTJ_VLM_PROXY=http://127.0.0.1:8080
+# Global download proxy — passed explicitly to all backend download clients (HuggingFace,
+# model DB, bioimage model zoo, web research/Tavily/DDGS, cellpose/stardist) and used as the
+# fallback for the LLM/VLM API clients when COPILOTJ_LLM_PROXY/COPILOTJ_VLM_PROXY are unset.
+# Passed via library proxy parameters (no global proxy env vars are set).
+# CIJ_PROXY=http://127.0.0.1:8080
 COPILOTJ_VLM_BASE_URL=https://api.siliconflow.cn/v1
 # VLM provider (optional, falls back to COPILOTJ_LLM_PROVIDER)
 # COPILOTJ_VLM_PROVIDER=gemini

AGENTS.md

@@ -53,6 +53,7 @@ The core agent framework follows a layered architecture:
 - `COPILOTJ_VLM_MODEL`: Vision model for image analysis (requires `COPILOTJ_VISION_ENABLED=1`)
 - `COPILOTJ_VLM_API_KEY`: API key for the vision model
 - `COPILOTJ_LLM_PROXY`: HTTP proxy
+- `CIJ_PROXY`: Global download proxy for all backend downloads (HuggingFace, model DB, bioimage model zoo, research/Tavily/DDGS, cellpose/stardist); also the fallback proxy for the LLM/VLM clients when their own proxy is unset. Passed explicitly to each download client (no global proxy env vars are set)
 - `COPILOTJ_KB_AUTOSAVE`: Set `1` to auto-ingest dialog summaries into knowledge bank
 - `LANGFUSE_SECRET_KEY`/`LANGFUSE_PUBLIC_KEY`: Optional Langfuse observability
 

DEVELOPMENT.md

@@ -217,24 +217,25 @@ Note: Ollama models generally do not support image input. If image understanding
 
 ### All configuration variables
 
-| Variable                  | Description                                                                     |
-| ------------------------- | ------------------------------------------------------------------------------- |
-| `COPILOTJ_LLM_MODEL`      | Main LLM model name (required)                                                  |
-| `COPILOTJ_LLM_API_KEY`    | API key for the main model (required)                                           |
-| `COPILOTJ_LLM_BASE_URL`   | Override API endpoint for the main model                                        |
-| `COPILOTJ_LLM_PROVIDER`   | Explicit LLM provider selection (optional; auto-detected if not set)            |
-| `COPILOTJ_VLM_MODEL`      | Vision-language model name (optional)                                           |
-| `COPILOTJ_VLM_API_KEY`    | API key for the VLM (falls back to `COPILOTJ_LLM_API_KEY`)                      |
-| `COPILOTJ_VLM_BASE_URL`   | Override API endpoint for the VLM (falls back to `COPILOTJ_LLM_BASE_URL`)       |
-| `COPILOTJ_VLM_PROVIDER`   | Explicit VLM provider (optional; falls back to `COPILOTJ_LLM_PROVIDER`)         |
-| `COPILOTJ_LLM_PROXY`      | HTTP/HTTPS proxy for LLM outbound API requests                                  |
-| `COPILOTJ_VLM_PROXY`      | HTTP/HTTPS proxy for VLM outbound requests (falls back to `COPILOTJ_LLM_PROXY`) |
-| `COPILOTJ_TAVILY_API_KEY` | Tavily API key for live web search                                              |
-| `COPILOTJ_KB_AUTOSAVE`    | Set `1` to auto-ingest dialog summaries into the knowledge bank                 |
-| `COPILOTJ_DEV`            | Development mode (presence-based flag)                                          |
-| `LANGFUSE_SECRET_KEY`     | Langfuse secret key for observability                                           |
-| `LANGFUSE_PUBLIC_KEY`     | Langfuse public key for observability                                           |
-| `LANGFUSE_HOST`           | Langfuse host URL (e.g. `https://us.cloud.langfuse.com`)                        |
+| Variable                  | Description                                                                                                                              |
+| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
+| `COPILOTJ_LLM_MODEL`      | Main LLM model name (required)                                                                                                           |
+| `COPILOTJ_LLM_API_KEY`    | API key for the main model (required)                                                                                                    |
+| `COPILOTJ_LLM_BASE_URL`   | Override API endpoint for the main model                                                                                                 |
+| `COPILOTJ_LLM_PROVIDER`   | Explicit LLM provider selection (optional; auto-detected if not set)                                                                     |
+| `COPILOTJ_VLM_MODEL`      | Vision-language model name (optional)                                                                                                    |
+| `COPILOTJ_VLM_API_KEY`    | API key for the VLM (falls back to `COPILOTJ_LLM_API_KEY`)                                                                               |
+| `COPILOTJ_VLM_BASE_URL`   | Override API endpoint for the VLM (falls back to `COPILOTJ_LLM_BASE_URL`)                                                                |
+| `COPILOTJ_VLM_PROVIDER`   | Explicit VLM provider (optional; falls back to `COPILOTJ_LLM_PROVIDER`)                                                                  |
+| `COPILOTJ_LLM_PROXY`      | HTTP/HTTPS proxy for LLM outbound API requests                                                                                           |
+| `COPILOTJ_VLM_PROXY`      | HTTP/HTTPS proxy for VLM outbound requests (falls back to `CIJ_PROXY`)                                                                   |
+| `CIJ_PROXY`               | Global download proxy for HuggingFace, model DB, bioimage zoo, research/Tavily/DDGS, cellpose/stardist; fallback for the LLM/VLM proxies |
+| `COPILOTJ_TAVILY_API_KEY` | Tavily API key for live web search                                                                                                       |
+| `COPILOTJ_KB_AUTOSAVE`    | Set `1` to auto-ingest dialog summaries into the knowledge bank                                                                          |
+| `COPILOTJ_DEV`            | Development mode (presence-based flag)                                                                                                   |
+| `LANGFUSE_SECRET_KEY`     | Langfuse secret key for observability                                                                                                    |
+| `LANGFUSE_PUBLIC_KEY`     | Langfuse public key for observability                                                                                                    |
+| `LANGFUSE_HOST`           | Langfuse host URL (e.g. `https://us.cloud.langfuse.com`)                                                                                 |
 
 A complete `.env.local` template:
 
@@ -246,6 +247,10 @@ COPILOTJ_LLM_API_KEY=sk-xxxxxxxx
 #COPILOTJ_LLM_PROXY=http://PATH_TO_YOUR_PROXY
 #COPILOTJ_LLM_PROVIDER=openai
 
+# Global download proxy (HuggingFace, model DB, bioimage, web research, cellpose/stardist).
+# Also the fallback for the LLM/VLM API proxies above. Passed explicitly — no global proxy env vars.
+#CIJ_PROXY=http://127.0.0.1:8080
+
 # Vision-language model (image understanding) — optional, choose one provider
 #COPILOTJ_VLM_MODEL=gemini-2.5-flash
 #COPILOTJ_VLM_API_KEY=AI-xxxxxxxx

copilotj/appose_worker.py

@@ -42,6 +42,7 @@
 
 from copilotj.core import load_config
 from copilotj.core.config import bootstrap_assets, load_managed_config, save_managed_config
+from copilotj.core.embedding import configure_download_proxy
 from copilotj.core.lifecycle import run_cleanup as _run_cleanup
 from copilotj.server import Server
 
@@ -101,6 +102,8 @@ def start_server() -> None:
         saved_port = _extract_port(load_managed_config().get("server_url"))
 
         cfg = load_config()
+        configure_download_proxy(cfg)
+
         _server = Server(cfg)
         _loop = asyncio.new_event_loop()
 

copilotj/core/config.py

@@ -69,6 +69,11 @@ class Config:
     vlm_proxy: str | None = None
     vlm_provider: str | None = None
 
+    # Global download proxy — passed explicitly to all backend download clients (HF,
+    # model DB, bioimage zoo, research/Tavily/DDGS, cellpose/stardist). Also the
+    # fallback proxy for the LLM/VLM API clients when their own proxy is unset.
+    cij_proxy: str | None = None
+
     # Tool keys
     tavily_api_key: str | None = None
 
@@ -142,6 +147,7 @@ def load_config() -> Config:
         llm_provider=os.getenv("COPILOTJ_LLM_PROVIDER", None),
         vlm_proxy=os.getenv("COPILOTJ_VLM_PROXY", None),
         vlm_provider=os.getenv("COPILOTJ_VLM_PROVIDER", None),
+        cij_proxy=os.getenv("CIJ_PROXY"),
         tavily_api_key=os.getenv("COPILOTJ_TAVILY_API_KEY", None),
         kb_autosave=os.getenv("COPILOTJ_KB_AUTOSAVE", "0") == "1",
         dev=os.getenv("COPILOTJ_DEV") is not None,
@@ -176,7 +182,7 @@ def resolve_vision_config(cfg: Config) -> Config:
     from copilotj.core.model_info import get_model_capabilities
 
     # Detect main model vision capability
-    main_caps = get_model_capabilities(cfg.llm_model) if cfg.llm_model else None
+    main_caps = get_model_capabilities(cfg.llm_model, cfg) if cfg.llm_model else None
     llm_supports_vision = bool(main_caps and main_caps.supports_vision)
 
     # Check if a separate VLM is explicitly configured

copilotj/core/embedding.py

@@ -6,20 +6,27 @@
 
 from langchain_core.embeddings import Embeddings
 
+from copilotj.core.config import Config
+
 logger = logging.getLogger(__name__)
 
 RAG_EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
 _embeddings: Embeddings | None = None
 
 __all__ = [
     "RAG_EMBEDDING_MODEL",
+    "configure_download_proxy",
     "get_embeddings",
     "new_local_embeddings",
 ]
 
 
 def new_local_embeddings() -> Embeddings:
-    """Create the local embedding model used by the bundled FAISS index."""
+    """Create the local embedding model used by the bundled FAISS index.
+
+    The HuggingFace download proxy is configured once at process startup via
+    :func:`configure_download_proxy`, so this function needs no ``Config``.
+    """
     from langchain_huggingface import HuggingFaceEmbeddings
 
     device = _detect_embedding_device()
@@ -32,6 +39,25 @@ def new_local_embeddings() -> Embeddings:
     )
 
 
+def configure_download_proxy(cfg: Config) -> None:
+    """Route HuggingFace downloads through ``CIJ_PROXY`` (explicit, no env vars).
+
+    ``huggingface_hub`` (httpx-based) downloads weights via a client produced by its
+    client factory. Installing a factory that returns a proxied ``httpx.Client`` makes
+    sentence-transformers/huggingface_hub fetch the embedding weights through the
+    proxy without touching ``os.environ``. Call once at process startup (server /
+    appose_worker / rag_builder) before the first HF download; a no-op when no
+    download proxy is configured.
+    """
+    proxy = cfg.cij_proxy
+    if not proxy:
+        return
+    import httpx
+    from huggingface_hub import set_client_factory
+
+    set_client_factory(lambda: httpx.Client(proxy=proxy))
+
+
 def get_embeddings() -> Embeddings:
     """Get the local embeddings used for ImageJ RAG."""
     global _embeddings

copilotj/core/model_client/__init__.py

@@ -65,10 +65,11 @@ def new_vlm_model_client(cfg: Config) -> ModelClient:
     return _new_model_client(
         cfg.vlm_model,
         cfg.vlm_api_key,
-        proxy=cfg.llm_proxy,
+        proxy=cfg.vlm_proxy or cfg.cij_proxy,
         base_url=cfg.vlm_base_url,
         cfg=cfg,
         provider=cfg.vlm_provider or cfg.llm_provider,
+        allow_llm_proxy_fallback=False,
     )
 
 
@@ -127,9 +128,13 @@ def _new_model_client(
     base_url: str | None = None,
     cfg: Config | None = None,
     provider: str | None = None,
+    allow_llm_proxy_fallback: bool = True,
 ) -> ModelClient:
     cfg = cfg or load_config()
-    proxy = proxy or cfg.llm_proxy
+    if allow_llm_proxy_fallback:
+        proxy = proxy or cfg.llm_proxy or cfg.cij_proxy
+    else:
+        proxy = proxy or cfg.cij_proxy
 
     # If provider is explicitly given, use it directly.
     if provider is not None:

copilotj/core/model_info.py

@@ -23,7 +23,7 @@
 
 import aiohttp
 
-from copilotj.core.config import get_home
+from copilotj.core.config import Config, get_home, load_config
 
 __all__ = [
     "CatalogModel",
@@ -232,7 +232,7 @@ def _store_db(data: dict[str, Any]) -> None:
 # ---------------------------------------------------------------------------
 
 
-async def download_db(*, force: bool = False) -> Path | None:
+async def download_db(*, force: bool = False, cfg: Config | None = None) -> Path | None:
     """Download the model database from GitHub.
 
     Skips the download if the cache is fresh (unless *force* is ``True``).
@@ -250,9 +250,10 @@ async def download_db(*, force: bool = False) -> Path | None:
     _log.info("Downloading model DB from %s", LITELLM_DB_URL)
 
     try:
+        proxy = (cfg or load_config()).cij_proxy
         timeout = aiohttp.ClientTimeout(total=_DOWNLOAD_TIMEOUT)
         async with aiohttp.ClientSession(timeout=timeout) as session:
-            async with session.get(LITELLM_DB_URL) as resp:
+            async with session.get(LITELLM_DB_URL, proxy=proxy) as resp:
                 resp.raise_for_status()
                 data = await resp.json(content_type=None)
 
@@ -268,12 +269,17 @@ async def download_db(*, force: bool = False) -> Path | None:
         return path if path.exists() else None
 
 
-def _download_db_sync() -> dict[str, Any]:
+def _download_db_sync(cfg: Config | None = None) -> dict[str, Any]:
     """Synchronous fallback download using ``urllib`` (for CLI / first-run)."""
     _log.info("Downloading model DB (sync) from %s", LITELLM_DB_URL)
     try:
+        proxy = (cfg or load_config()).cij_proxy
         req = urllib.request.Request(LITELLM_DB_URL)
-        with urllib.request.urlopen(req, timeout=_DOWNLOAD_TIMEOUT) as resp:
+        if proxy:
+            opener = urllib.request.build_opener(urllib.request.ProxyHandler({"http": proxy, "https": proxy}))
+        else:
+            opener = urllib.request.build_opener()
+        with opener.open(req, timeout=_DOWNLOAD_TIMEOUT) as resp:
             data = json.loads(resp.read().decode("utf-8"))
 
         if isinstance(data, dict) and len(data) >= 100:
@@ -382,7 +388,7 @@ def _ollama_vision_heuristic(model: str) -> bool:
 # ---------------------------------------------------------------------------
 
 
-def get_model_capabilities(model: str) -> ModelCapabilities:
+def get_model_capabilities(model: str, cfg: Config | None = None) -> ModelCapabilities:
     """Return capabilities for *model*.
 
     Loads the cached model database, looks up the model (with normalisation),
@@ -402,7 +408,7 @@ def get_model_capabilities(model: str) -> ModelCapabilities:
     db = _load_db()
     if not db:
         # No cache available — try a blocking download on first use.
-        db = _download_db_sync()
+        db = _download_db_sync(cfg)
 
     entry = _lookup_model(db, model) if db else None
 
@@ -464,7 +470,7 @@ def _merge_supplements(provider: str, catalog_models: list[CatalogModel]) -> lis
     return merged
 
 
-def list_catalog_models(provider: str) -> list[CatalogModel]:
+def list_catalog_models(provider: str, cfg: Config | None = None) -> list[CatalogModel]:
     """Return chat models from the cached LiteLLM catalog for *provider*.
 
     Filters entries whose ``litellm_provider`` equals *provider* and whose
@@ -479,7 +485,7 @@ def list_catalog_models(provider: str) -> list[CatalogModel]:
     """
     db = _load_db()
     if not db:
-        db = _download_db_sync()
+        db = _download_db_sync(cfg)
     if not db:
         return _merge_supplements(provider, [])
 
@@ -514,11 +520,11 @@ def list_catalog_models(provider: str) -> list[CatalogModel]:
     return _merge_supplements(provider, models)
 
 
-async def ensure_model_db_async() -> bool:
+async def ensure_model_db_async(cfg: Config | None = None) -> bool:
     """Ensure the model DB is available (download if stale).
 
     Called at server startup as a background task.  Returns ``True``
     if the DB is available (fresh or stale).
     """
-    path = await download_db()
+    path = await download_db(cfg=cfg)
     return path is not None and path.exists()

copilotj/core/model_listing.py

@@ -19,6 +19,7 @@
 
 import aiohttp
 
+from copilotj.core.config import Config
 from copilotj.core.model_info import CatalogModel, get_model_capabilities, list_catalog_models
 
 __all__ = [
@@ -50,7 +51,9 @@ async def _fetch_ollama_tags(base_url: str, *, timeout: float) -> dict[str, Any]
         return None
 
 
-async def list_ollama_models(base_url: str, *, timeout: float = 2.0) -> list[CatalogModel] | None:
+async def list_ollama_models(
+    base_url: str, *, timeout: float = 2.0, cfg: Config | None = None
+) -> list[CatalogModel] | None:
     """List models installed on a local Ollama instance.
 
     Performs a live ``GET {base_url}/api/tags``.  Ollama runs locally, so we
@@ -69,7 +72,7 @@ async def list_ollama_models(base_url: str, *, timeout: float = 2.0) -> list[Cat
         name = entry.get("name") or entry.get("model")
         if not name:
             continue
-        caps = get_model_capabilities(f"ollama/{name}")
+        caps = get_model_capabilities(f"ollama/{name}", cfg)
         models.append(
             CatalogModel(
                 id=name,
@@ -94,7 +97,9 @@ def _model_to_dict(m: CatalogModel) -> dict[str, Any]:
     }
 
 
-async def list_provider_models(provider: str, *, base_url: str | None = None) -> dict[str, Any]:
+async def list_provider_models(
+    provider: str, *, base_url: str | None = None, cfg: Config | None = None
+) -> dict[str, Any]:
     """Resolve available models for *provider* into a uniform dict.
 
     - ``ollama`` → live ``/api/tags`` at ``base_url`` (default
@@ -107,7 +112,7 @@ async def list_provider_models(provider: str, *, base_url: str | None = None) ->
     Never raises — callers can ``asyncio.gather`` over providers safely.
     """
     if provider == "ollama":
-        models = await list_ollama_models(base_url or DEFAULT_OLLAMA_URL)
+        models = await list_ollama_models(base_url or DEFAULT_OLLAMA_URL, cfg=cfg)
         if models is None:
             return {"provider": provider, "source": "unreachable", "models": []}
         return {
@@ -116,5 +121,5 @@ async def list_provider_models(provider: str, *, base_url: str | None = None) ->
             "models": [_model_to_dict(m) for m in models],
         }
 
-    models = list_catalog_models(provider)
+    models = list_catalog_models(provider, cfg)
     return {"provider": provider, "source": "catalog", "models": [_model_to_dict(m) for m in models]}

copilotj/multiagent/agent_configs/research_agent.toml

@@ -147,7 +147,7 @@ Use this tool when you need to:
 
 [[tools]]
 display_name = "DuckDuckGo Search"
-function = "copilotj.multiagent.research_tools.ddg_search"
+factory = "copilotj.multiagent.research_tools.make_ddg_search"
 description = """
 Use this tool to search for current information and image links using DuckDuckGo.
 

copilotj/multiagent/agent_configs/tool_agent.toml

@@ -133,7 +133,7 @@ Use this tool for specialized tools fail or need debugging. **ERROR RECOVERY ONL
 """
 
 [[tools]]
-function = "copilotj.multiagent.py_tools.cellpose_segmentation"
+factory = "copilotj.multiagent.py_tools.make_cellpose_segmentation"
 description = """
 Deep learning-based cell segmentation using Cellpose. Supports nuclei, cytoplasm models with GPU \
 acceleration. Optimized with native cellpose image loading for maximum compatibility.
@@ -161,7 +161,7 @@ signals.
 
 [[tools]]
 display_name = "StarDist Segmentation"
-function = "copilotj.multiagent.py_tools.stardist_segmentation"
+factory = "copilotj.multiagent.py_tools.make_stardist_segmentation"
 description = """
 StarDist instance segmentation for cells and nuclei.
 

copilotj/multiagent/leader_multiagent.py

@@ -784,6 +784,7 @@ async def run(self, task: str, trace_ctx: Langfuse | None = None) -> None:
                         "Reply with a single Thought + Action, or a Final Answer.",
                         trace_ctx=trace_ctx,
                     )
+                    syntax_error_counter = 0  # Reset syntax error counter on success
                 except ModelSyntaxError as e:
                     syntax_error_counter += 1
                     self.log_error(