Hebrus

Local inference built first for Apple Metal and SSD streaming.

Hebrus is a focused inference engine for running a small, explicitly qualified set of large mixture-of-experts models on Apple Silicon. It combines mmap-backed GGUF loading, an embedded ExpertMajor v2 routed-weight store, Metal execution, and adaptive SSD streaming so unified memory is a performance tier rather than the only capacity boundary.

Quick start · Supported models · Measured results · Documentation · Contributing

Note

Hebrus is the public engine name and canonical executable namespace. The engine is published at andreaborio/hebrus, with Hebrus Studio as its companion macOS application. Durable ds4 and DSBox identifiers remain only where changing their bytes would break model, application, or user-data compatibility; see ADR 0005.

Important

Hebrus began as a fork of antirez/ds4 and retains substantial core-engine implementation, Git history, and design work from that project. The fork has since diverged toward Apple Metal, embedded ExpertMajor storage, and SSD-first execution. This attribution does not imply endorsement by or an official partnership with the upstream maintainer. See Acknowledgments and Third-party notices for the precise source scopes.

What Hebrus is for

Hebrus is designed around one practical question: how effectively can large MoE models run locally when Metal and the SSD are treated as one coordinated memory system?

• Metal-first inference. Production model execution targets Apple Metal.
• SSD streaming as a first-class path. Routed experts can be cached and streamed without keeping the complete model in unified memory.
• Hardware-aware AUTO policy. Residency and cache plans use model geometry, context memory, Metal's working-set recommendation, and live memory pressure.
• One validated routed-weight container. Supported artifacts embed the checksummed ds4.expert_major.v2 store directly in GGUF.
• Narrow support, explicit failure. Unsupported model families, old stores, sidecars, and unqualified backend combinations fail closed instead of taking an untested fallback.
• Evidence before claims. Correctness and performance decisions are tied to exact commits, model hashes, hardware, prompts, memory state, and context frontiers.

Hebrus is not a general GGUF runner. Arbitrary community GGUFs are not expected to load, even when their model family name looks compatible.

Supported models

The current production contract is Apple Silicon + Metal + embedded ExpertMajor v2. AUTO is the normal startup mode.

Published artifact filenames carrying the legacy prefix are frozen compatibility identifiers tied to existing hashes and manifests. They do not represent the current product name; the runtime and user-facing project are Hebrus.

Model family	Minimum unified memory	Qualified execution	Artifact availability
DeepSeek V4 Flash	64 GiB	AUTO resolving to resident or SSD; explicit modes for qualification	Published ExpertMajor v2 artifact; download_model.sh deepseek-v2
GLM 5.2	64 GiB	AUTO resolving to SSD streaming only; resident requests are rejected	Published ExpertMajor v2 artifact; download_model.sh glm-v2
Qwen3.6-35B-A3B	16 GiB	AUTO resolving to resident or SSD; 16 GiB necessarily uses SSD	Published MLX affine4/group-64 artifact; download_model.sh qwen-v2

The canonical machine-readable Qwen release contract records the current artifact as published. The release is exactly Qwen3.6-35B-A3B-DS4-ExpertMajor-v2-MLX-Affine4-G64.gguf, 20,808,566,880 bytes, SHA-256 dd17266185833a9f05531ce366fd7284ddca1ed64aa3dcf06e321e8c72c9ea3d. It is pinned to immutable repository revision 7bf9c3f7f6136aeb2599d75ee61c0cc2f18e2b02 and requires runtime commit 73a332fef82a0bcdd567d17e0de17aa004cad85d or a compatible descendant. The older Qwen3.6-35B-A3B-DS4-ExpertMajor-v2-Q4_K_S.gguf store is incompatible with the current runtime and remains negative-only, never a download or inference fallback.

DeepSeek V4 PRO has no qualified runtime artifact in the current contract. Canonical model files are converter inputs, not inference fallbacks. CPU is a reference and build-isolation path; CUDA and ROCm are frozen with source absent; distributed inference is retired.

The authoritative details are in the runtime support contract.

Quick start

Requirements:

• Apple Silicon Mac;
• Xcode Command Line Tools;
• enough unified memory for the selected supported family;
• enough SSD space for the exact ExpertMajor v2 artifact.

Build and verify the engine without loading model weights:

xcode-select --install
git clone https://github.com/andreaborio/hebrus.git
cd hebrus

make -j
./hebrus --build-info
./hebrus --capabilities=json
make model-free-test

For a user-local installation that does not require sudo:

make install PREFIX="$HOME/.local"
export PATH="$HOME/.local/bin:$PATH"

hebrus --build-info

make install copies the five canonical hebrus* executables and creates the five ds4* compatibility names as relative symlinks to those files. Package builders can stage the same layout without touching the host filesystem:

make install DESTDIR="$PWD/package-root" PREFIX=/usr/local
make uninstall DESTDIR="$PWD/package-root" PREFIX=/usr/local

PREFIX defaults to /usr/local; BINDIR may override its bin directory. Uninstall removes only the ten explicit command paths and leaves the directory and every unrelated file intact. make install-test verifies the staged layout, aliases, capabilities, and uninstall boundary without a model or privileged access.

For a published DeepSeek V4 Flash artifact on a qualified 64 GiB-or-larger host:

./download_model.sh deepseek-v2

./hebrus \
  -m gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf \
  --ctx 8192 \
  -p "Explain why SSD streaming changes the memory limit for MoE inference."

Start the local HTTP server with the same artifact:

./hebrus-server \
  -m gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-DS4-ExpertMajor-v2.gguf \
  --ctx 8192

The canonical commands are hebrus, hebrus-server, hebrus-agent, hebrus-bench, and hebrus-eval. Their ds4* counterparts remain symlinks to the same binaries for compatibility; formats, model filenames, environment variables, and serialized DS4 identifiers are not renamed by this bridge.

How the runtime fits together

1. The engine mmaps the GGUF and validates the embedded ds4.expert_major.v2 manifest, tensor inventory, geometry, ranges, and digest before inference.
2. AUTO calculates fixed model and context memory, Metal headroom, live host pressure, and the model-specific residency policy.
3. When the complete admitted working set fits, Metal uses full mapped tensors. Otherwise, the runtime keeps non-routed state mapped and streams routed expert records through a bounded cache.
4. CLI, HTTP server, agent, benchmark, and evaluation frontends share the same engine and capability contract.

ExpertMajor v2 is a disk ABI, not a brand label. The tensor identifier ds4.expert_major.v2, storage wire values, published filenames and hashes, and disk-KV format remain unchanged.

Measured results

These are current durable measurements from one Apple M5 Pro with 64 GiB of unified memory. They are not cross-model rankings: artifacts, prompts, residency modes, and decode lanes differ. Follow the evidence links for exact commands, hashes, invalidations, and memory telemetry.

Path	Evidence lane	Measured result
Qwen3.6 MLX affine4/group-64	32K pure prefill; separate 128+16 decode, resident and forced SSD	Resident: 877.34 prefill t/s and 57.43 decode t/s. Forced SSD: 83.69 prefill t/s and 23.94 decode t/s.
DeepSeek V4 Flash ExpertMajor v2	32K prose + 128 decode, Metal SSD	164.43 prefill t/s and 7.27 decode t/s, with zero swapout.
GLM 5.2 ExpertMajor v2	32K prose + 128 decode, AUTO resolving to SSD	44.73 prefill t/s and 1.87 overall decode t/s, with zero swapout.

Sources:

• Qwen unified affine AUTO and SSD promotion
• DeepSeek and GLM long-context Metal stack
• Benchmark evidence index

Performance acceptance requires isolated short, medium, large, and 32K long-context lanes; memory-sensitive changes may require 65K and 100K as well. A short-context best is not substituted for the long-context gate.

Documentation

Start and operate

• Migration guide — install the bridge, validate both command identities, move integrations safely, and roll back without rewriting models or user data.
• Runtime support contract — supported families, hardware floors, backends, and fail-closed boundaries.
• Brand compatibility contract — canonical commands, legacy aliases, stable formats, and migration rules.
• Engine reference — CLI, server, agent, disk KV, tracing, evaluation, and operational details.
• Metal and SSD policy — build identity, AUTO residency, cache planning, and benchmark gates.

Models and formats

• ExpertMajor v2 roadmap
• DeepSeek ExpertMajor v2
• GLM 5.2 ExpertMajor v2
• Qwen ExpertMajor v2 MLX-affine store
• GGUF and quantization tools

Architecture and evidence

• Runtime flow diagram — accessible, versioned view of mmap GGUF, ExpertMajor validation, AUTO admission, and the resident Metal or SSD-streaming paths.
• Code map
• Hebrus naming and compatibility decision
• Benchmark methodology and records
• Release checklist
• Fork/upstream ledger

Project and community

• Cross-repository rebrand rollout plan — current engine, Studio, and website status; non-breaking rename order; CI, compatibility, repository, release, and launch gates.
• Hebrus bridge launch-candidate notes — implemented scope, preserved compatibility, release evidence, and external gates that remain pending.
• Contribution guide
• Security policy
• Code of conduct
• Governance
• Changelog
• Citation metadata
• Acknowledgments and third-party notices

Project status

The engine is beta software; hebrus-agent remains alpha. Large-model inference can create substantial memory pressure and I/O. Follow AUTO, use only exact qualified artifacts, monitor swap and memory pressure, and do not expose a local server to untrusted networks without an appropriate security boundary.

The companion desktop application is published as andreaborio/hebrus-studio. Its compatibility bridge preserves the legacy DSBox bundle ID, data locations, environment namespace, storage keys, and rollback path so the public rename does not split existing installations.

Contributing and security

Read CONTRIBUTING.md before opening a change. Contributions must identify realistic failure modes, run the applicable correctness and performance gates, and classify whether the work should also be proposed to antirez/ds4.

Do not publish vulnerability details in an ordinary issue. Until a dedicated private reporting channel is enabled, follow the zero-disclosure contact procedure in SECURITY.md. Enabling GitHub private vulnerability reporting remains a pre-launch administrative gate.

License and credit

The project-level license is MIT. Bundled or adapted third-party code retains its own notices and conditions; model weights and datasets are not granted rights by this repository's software license.

Read ACKNOWLEDGMENTS.md for engineering provenance and THIRD_PARTY_NOTICES.md for bundled-code notices.