Skip to content

fix(processing): lazy-load provider imports in response.py to reduce import-time RAM usage#2268

Open
chaosreload wants to merge 1 commit into567-labs:mainfrom
chaosreload:fix/lazy-provider-imports-in-response
Open

fix(processing): lazy-load provider imports in response.py to reduce import-time RAM usage#2268
chaosreload wants to merge 1 commit into567-labs:mainfrom
chaosreload:fix/lazy-provider-imports-in-response

Conversation

@chaosreload
Copy link
Copy Markdown

Summary

Continuing the work started in #2206.

PR #2206 moved the top-level provider imports in __init__.py to lazy loading — great first step. However, instructor/processing/response.py still eagerly imports all 11 provider utils at module load time, which in turn pulls in the full provider SDKs (anthropic, bedrock, cohere, gemini, mistral, etc.).

As analyzed by @kiyeonjeon21 in #2205:

import instructor loads ~1670 modules (anthropic alone accounts for ~550)

Root cause

instructor/processing/response.py had 11 top-level provider import blocks:

from ..providers.anthropic.utils import handle_anthropic_tools, ...
from ..providers.gemini.utils import handle_gemini_json, ...
# ... 9 more providers

These are only needed when a specific provider's mode handler is actually invoked — not at import time.

Changes

instructor/processing/response.py — only file changed:

  1. All 11 provider import blocks moved into TYPE_CHECKING block (type checkers still work correctly)
  2. Added a _lazy(module_path, func_name) helper backed by _provider_cache — uses importlib.import_module on first call, cached thereafter
  3. All three dispatch structures (PARALLEL_MODES, mode_handlers, REASK_HANDLERS) updated to use _lazy(...) wrappers

instructor/batch/providers/__init__.py — already uses importlib.util.find_spec conditional imports, no changes needed.

Verification

import instructor  # baseline (from __init__.py)
baseline = set(sys.modules.keys())

from instructor.processing.response import handle_response_model, handle_reask_kwargs
new = set(sys.modules.keys()) - baseline

# Zero new provider modules loaded by response.py
assert not any('instructor.providers' in m for m in new)
# ✅ SUCCESS

Provider SDKs are now only loaded when the corresponding mode handler is first invoked.

Fixes #2205

…RAM on import

Move all 11 provider util imports (anthropic, bedrock, cerebras, cohere,
fireworks, gemini, mistral, openai, perplexity, writer, xai) behind
TYPE_CHECKING and load them on demand via importlib.import_module.

This prevents ~1670 modules from being eagerly loaded when importing
instructor, significantly reducing initial RAM usage.

Closes 567-labs#2205

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RAM Spike when I import instructor

1 participant