fix: eliminate Linux Docker crash and restore MLX on native macOS - #3
Open
king-dopey wants to merge 1 commit into
Open
fix: eliminate Linux Docker crash and restore MLX on native macOS#3king-dopey wants to merge 1 commit into
king-dopey wants to merge 1 commit into
Conversation
## Problem
Linux Docker containers crashed immediately on startup with:
ImportError: libmlx.so: cannot open shared object file: No such file or directory
Root cause: `__init__.py` eagerly imported `JinaMLXReranker` at module level,
which triggered a top-level `import mlx.core as mx` before backend selection
could occur. On Linux, MLX shared libraries do not exist.
A secondary regression caused MLX to no longer auto-install on native macOS
after dependencies were restructured as optional extras.
## Changes
### src/local_reranker/__init__.py
- Replaced eager `from .jina_mlx_reranker import JinaMLXReranker` with a
`__getattr__` lazy-loading implementation so MLX is only imported when
actually accessed, making package import safe on Linux/Docker.
### src/local_reranker/config.py
- Added `get_mlx_platform_error()`: returns an error string when platform is
not macOS arm64, None otherwise.
- Added `validate_backend_type()`: raises ValueError with an actionable message
if mlx backend is selected on an unsupported platform.
### src/local_reranker/api.py
- Moved MLX import inside the `mlx` branch of the lifespan startup function,
after `validate_backend_type()` is called, so it is never reached on Linux.
### src/local_reranker/cli.py
- Added `validate_backend_type()` call before server startup so CLI fails fast
with a clear error message if mlx is requested on a non-macOS platform.
### src/local_reranker/reranker_mlx.py
- Added init-time platform check via `get_mlx_platform_error()`; raises
RuntimeError with an actionable fallback suggestion if not on macOS arm64.
- Moved `JinaMLXReranker` import inside `_load_mlx_reranker()` with
try/except for ImportError/OSError.
### src/local_reranker/reranker_pytorch.py
- Added `warnings.filterwarnings` to suppress the harmless
`torch_dtype is deprecated, use dtype instead` warning emitted by Jina
model files downloaded from HuggingFace.
### pyproject.toml
- Moved MLX dependencies from `[project.optional-dependencies]` back to main
`[project.dependencies]` with platform markers:
mlx>=0.30.0; sys_platform == 'darwin' and platform_machine == 'arm64'
This restores auto-installation on native macOS Apple Silicon without
requiring a separate install step or extra flag.
- Pinned `transformers>=4.57.0,<5` to avoid breaking API changes in 5.x
that affect Jina model loading inside Docker.
### Dockerfile
- Added `RERANKER_BACKEND_TYPE=pytorch` environment variable.
- CMD already hardcodes `--backend pytorch` ensuring the container never
attempts to load MLX regardless of any env var override.
### tests/test_cli.py
- Added `test_main_unsupported_mlx_backend`: verifies mlx backend fails with
SystemExit on unsupported platforms.
- Added `test_package_import_keeps_mlx_lazy`: confirms jina_mlx_reranker is
not present in sys.modules after a bare package import.
### README.md
- Clarified that MLX auto-installs only on native macOS Apple Silicon.
- Updated Backend Support table to distinguish native macOS (supported) from
Docker Desktop on Apple Silicon (not supported, Linux runtime).
- Removed outdated `uv pip install -e ".[dev,mlx]"` install instructions.
- Updated Troubleshooting section to reflect new dependency structure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Linux Docker containers crashed immediately on startup with:
ImportError: libmlx.so: cannot open shared object file: No such file or directory
Root cause:
__init__.pyeagerly importedJinaMLXRerankerat module level, which triggered a top-levelimport mlx.core as mxbefore backend selection could occur. On Linux, MLX shared libraries do not exist.A secondary regression caused MLX to no longer auto-install on native macOS after dependencies were restructured as optional extras.
Changes
src/local_reranker/init.py
from .jina_mlx_reranker import JinaMLXRerankerwith a__getattr__lazy-loading implementation so MLX is only imported when actually accessed, making package import safe on Linux/Docker.src/local_reranker/config.py
get_mlx_platform_error(): returns an error string when platform is not macOS arm64, None otherwise.validate_backend_type(): raises ValueError with an actionable message if mlx backend is selected on an unsupported platform.src/local_reranker/api.py
mlxbranch of the lifespan startup function, aftervalidate_backend_type()is called, so it is never reached on Linux.src/local_reranker/cli.py
validate_backend_type()call before server startup so CLI fails fast with a clear error message if mlx is requested on a non-macOS platform.src/local_reranker/reranker_mlx.py
get_mlx_platform_error(); raises RuntimeError with an actionable fallback suggestion if not on macOS arm64.JinaMLXRerankerimport inside_load_mlx_reranker()with try/except for ImportError/OSError.src/local_reranker/reranker_pytorch.py
warnings.filterwarningsto suppress the harmlesstorch_dtype is deprecated, use dtype insteadwarning emitted by Jina model files downloaded from HuggingFace.pyproject.toml
[project.optional-dependencies]back to main[project.dependencies]with platform markers: mlx>=0.30.0; sys_platform == 'darwin' and platform_machine == 'arm64' This restores auto-installation on native macOS Apple Silicon without requiring a separate install step or extra flag.transformers>=4.57.0,<5to avoid breaking API changes in 5.x that affect Jina model loading inside Docker.Dockerfile
RERANKER_BACKEND_TYPE=pytorchenvironment variable.--backend pytorchensuring the container never attempts to load MLX regardless of any env var override.tests/test_cli.py
test_main_unsupported_mlx_backend: verifies mlx backend fails with SystemExit on unsupported platforms.test_package_import_keeps_mlx_lazy: confirms jina_mlx_reranker is not present in sys.modules after a bare package import.README.md
uv pip install -e ".[dev,mlx]"install instructions.