MiniMax-H3-Inference

Local inference for MiniMax-H3 (33B joint video+audio DiT) on a pair of 48 GB GPUs, packaged three ways:

• Command line tool (mmh3 generate) — text, reference images/videos, and first/last-frame anchors in; an mp4 with a generated audio track out.
• HTTP queue API server (mmh3 serve) — the same generation code behind an async queue (modeled on oxen.ai's /ai/queue), with the ~42 GB of weights kept resident between jobs and history in SQLite. Optional bidirectional mirroring to an Oxen.ai hub.
• Web UI (served by mmh3 serve) — a Midjourney-style gallery: prompt bar with drag/paste references (images, video, audio), first/last-frame pinning, a frame-precise player, approve/reject curation, per-generation deep links, and live progress over SSE.

Model code is vendored from ostris/ai-toolkit@8502a84 ("MiniMax H3 T2V and I2V training") and loads the same Comfy-Org repack checkpoints that commit's trainer uses.

Usage

python -m mmh3 check                          # resolve checkpoints, report GPUs
python -m mmh3 generate (-p PROMPT | -f PROMPT_FILE) [-o OUT.mp4] [-i IMAGE ...] [options]
python -m mmh3 serve [--port 8000] [--preload]   # HTTP queue API + web UI

PY=/home/ox/.venv/bin/python
cd /home/ox/Code/AI/MiniMax-H3-Inference

# verify weights + GPUs before anything slow happens
$PY -m mmh3 check

# text -> video with generated audio
$PY -m mmh3 generate -p "a paper boat drifting down a rain gutter" -o out.mp4

# first-frame image-to-video; canvas derived from the image's aspect ratio
$PY -m mmh3 generate -p "she turns and smiles" -i portrait.png -o out.mp4

flag	default	what it does
-p, --prompt	required¹	text prompt
-f, --file PATH	required¹	read the text prompt from a file (- for stdin). ¹Pass exactly one of -p / -f.
-o, --out	auto	output .mp4 path. Defaults to outputs/<date>_<time>-<prompt preview>.mp4 (e.g. outputs/2026-08-05_143022-a-paper-boat-drifting.mp4); the outputs/ dir is created if needed.
-i, --image PATH	none	reference image or video; repeat for several. Videos expand to 4 evenly-spaced frames. The first image (from a video: its first sampled frame) is the i2v first frame and sets the canvas aspect.
--last-image PATH	none	still image pinned as the clip's last frame (fl2va only). Combine with -i for first→last interpolation, or use alone to generate a clip that ends on this image (it then sets the canvas aspect). Also fed to the text encoder as the final <Picture N>.
--frames N	124	frame count, snapped down to the VAE's 17n+5 grid (124 = 5.17 s)
-s, --seconds SEC	—	clip length in seconds; converted to frames at 24 fps and snapped up to the grid, so the clip is never shorter than asked. Mutually exclusive with --frames.
--steps N	28	denoising steps; runs N-1 model evaluations
--width PX, --height PX	from image, else 1280×704 (720p 16:9 on the 32-px grid)	output resolution in pixels. Multiples of 32; off-grid values are snapped with a warning. Give just one and the other follows the reference image's aspect ratio (16:9 without an image).
--size WxH	—	shorthand for the two above, e.g. --size 1344x768. A single number means square (--size 768).
--seed N	random	reproducible noise
--no-audio	off	skip audio decode, write a silent mp4
--partition {fl2va,ref2va}	fl2va	which DiT to load; fl2va covers t2v and first/last-frame i2v, ref2va is the reference-conditioned DiT (full latent reference conditioning is the server/UI path)
--device, --te-device	auto	override placement (e.g. cuda:0, cpu)
--offload	auto	free the text encoder after encoding the prompt
--dtype {bf16,fp16,fp32}	bf16	compute dtype
--frames-dir DIR	none	also dump raw PNG frames
--allow-download	off	fetch missing checkpoints from the hub (up to 42 GB)
-v, -q, --plain	—	more logging / warnings only / line logs instead of progress bars

--help on either subcommand lists everything, including the checkpoint-path overrides. See More examples, Resolution and Frames and steps for the canvas and frame-count rules.

Status: working, verified end-to-end on 2x RTX 6000 Ada (through 2026-08-06). 768x768/22f/16 steps ≈ 48 s wall clock; the longest verified clip is 15 s at 1280×704 (362 frames, ~3 min per denoising step — attention is quadratic in clip length). First+last-frame anchoring reproduces both anchor images at the clip ends. See Verification.

One caveat about this machine, not this code: direct GPU-to-GPU copies are silently corrupt here (they return zeros while can_device_access_peer() reports success). mmh3 detects this and stages cross-device transfers through host memory, so its output is correct — but any other multi-GPU tool here (DDP, NCCL, device_map="auto") will be silently wrong. Details and workarounds: CUDA-SETUP.md.

Server + web UI

mmh3 serve wraps the same generation code in an HTTP queue API (modeled on oxen.ai's /ai/queue) and serves a Midjourney-style gallery UI at the root URL.

cd web && npm install && npm run build && cd ..   # build the UI once
$PY -m mmh3 serve                                  # http://0.0.0.0:8000

Unlike the CLI, the server keeps the model resident: the ~42 GB of weights load once (on the first job, or at startup with --preload) and every later generation skips straight to prompt encoding. Jobs run one at a time from a persistent queue; history, parameters, seeds and timings live in outputs/mmh3.db (SQLite) next to the videos.

flag	default	what it does
--host	0.0.0.0	bind address (exposed on the LAN — there is no auth; use 127.0.0.1 for local-only)
--port	8000	port
--outputs DIR	outputs/	videos, reference images, posters and the history DB
--preload	off	load all weights at startup instead of on the first job
--oxen-remote URL, --oxen-token TOKEN	unset	mirror generations to an Oxen.ai hub (env: MMH3_OXEN_REMOTE / MMH3_OXEN_TOKEN). Optional — without these the server is fully local with zero Oxen dependencies.
--oxen-model NAME	minimax-h3	free-form model name generations carry in Oxen (no registration needed)
--oxen-poll SEC	10	also claim jobs enqueued on the Oxen side, polling every SEC seconds while the local queue is idle (0 disables claiming)
weights/devices	as in generate	--partition, --device, --te-device, --dtype, checkpoint paths, ...

When Oxen is configured, every local generation is mirrored: a pending record appears in the Oxen workbench (spinner and all) the moment the render starts, and on completion the mp4 uploads to the Hub, which stages it and appends the history-dataframe row with its own storage code — identical format to generations made in Oxen's UI. The claim loop makes it bidirectional: jobs enqueued in Oxen against your model name are pulled into the local queue and rendered here. Oxen failures are recorded on the generation (oxen_status) and never affect the local render; the local SQLite + outputs/ remain the source of truth for this UI. Design + Hub-side API: docs/OXEN-INTEGRATION.md.

The API (see /docs for the OpenAPI page):

endpoint	does
POST /api/ai/queue	enqueue: {prompt, image_ids?, last_image_id?, frames|seconds?, width?, height?, steps?, seed?, partition?, no_audio?, crf?} → {generations: [{generation_id, status: "queued", ...}]}. last_image_id (a still, fl2va only) pins the clip's final frame — the UI's "pin as last frame" on an image chip.
GET /api/ai/queue	list/search history: ?status=queued,processing&search=boat&limit=60&offset=0 → {count, generations}
GET /api/ai/queue/:id	poll one: status (queued→processing→succeeded|failed|cancelled), live progress (stage + denoise step), result_url, seed, timings
PATCH /api/ai/queue/:id	curation: {rating: "approved"|"rejected"|null} — approved shows a green check on the gallery thumbnail; null clears. Broadcast over SSE so all tabs stay in sync
DELETE /api/ai/queue/:id	active job: cancel (stops at the next denoise step). Finished: delete the record and its files
POST /api/images	multipart upload of reference images, videos and audio (videos get a poster) → image_ids to pass at enqueue. Under ref2va, videos/audio condition the DiT as pinned latent rows; under fl2va videos fall back to 4 frames as vision blocks and audio needs the Model toggle switched to ref2va.
GET /api/events	SSE stream of generation_update / generation_deleted events (what keeps the UI live)
GET /api/files/...	the mp4s, poster frames and uploaded references
GET /g/:id	deep link to one generation — serves the app shell, which opens that generation's lightbox (the URL the browser shows while a modal is open)
GET /api/status	engine residency/placement + queue counts
GET/PUT /api/settings	server-side settings (Oxen API key — stored in mmh3.db, never returned to the browser — rewrite toggle, model, system prompt)
POST /api/rewrite	streams a prompt rewrite (SSE token/done/error events), proxied to Oxen.ai chat completions with the configured key

The UI (built from web/, React + Vite; the only runtime additions to react/react-dom are media-chrome's headless player components): a dark gallery that plays clips on hover, click for a lightbox with a frame-precise player — the clips are exactly 24 fps, so the scrubber steps single frames (←/→, on-screen buttons, live frame / total readout via requestVideoFrameCallback), with play/pause (space), 1-second jumps (shift+←/→), loop (l), and mute/volume (m) — plus parameters, timings and actions (rerun with a new seed, edit & resubmit, download, copy, delete). Every open lightbox has a shareable URL (/g/<generation_id>; back/forward and hard refresh all work). Its rail is a source switcher: the result and each reference as labeled thumbnails (play/speaker badges mark video/audio) — click one and it replaces the result in the viewer, full-size or playing. Curation: approve (green check, also one click on the thumbnail hover) or reject from the lightbox; approved clips wear a check on their gallery tile, synced across tabs over SSE.

The bottom prompt bar takes drag/paste reference images, videos and audio (attaching audio auto-switches the model to ref2va), an image chip can be pinned as the clip's last frame (fl2va; the first chip is always frame 1), and a parameters popover covers length on the 17n+5 grid, size presets, steps, seed, audio and the fl2va/ref2va model toggle; prompt search (⌘K), status filters, and live progress on in-flight tiles over SSE — in-flight tiles keep the prompt and reference thumbnails visible, and click through to the lightbox's live progress view with all parameters and a cancel action.

Prompt rewriting (gear icon → Settings): store an Oxen.ai API key, pick a model (default claude-sonnet-4-6) and edit the rewriter's system prompt. With the toggle on, submitting detours through an approval sheet — the rewrite streams in token by token next to your original, and nothing generates until you pick one (Enter accepts the rewrite, Esc cancels). Rerun re-populates the prompt bar with a generation's prompt/refs/params and a fresh seed so you can tweak before running; Edit does the same but keeps the seed locked for exact reproduction. During development npm run dev proxies /api to a running server (MMH3_API=... npm run dev to point it elsewhere).

mmh3 serve --mock swaps in a fake engine (ffmpeg gradients, ~5 s per "job") for hacking on the server or UI without touching the GPUs.

What it does and does not take

MiniMax-H3 generates video and its audio track jointly from text plus optional references — keyframe images (first/last), reference images, videos and audio. It is guidance-distilled: one transformer forward per step, no negative prompt, no CFG scale.

input	supported	notes
text prompt	yes	-p/--prompt inline, or -f/--file to read it from a file
reference / keyframe image(s)	yes	-i/--image, repeatable. They enter the Qwen3-VL prompt as vision blocks; the first is the i2v first frame and sets the canvas aspect.
reference audio	yes (ref2va)	Upload audio in the UI (the Model toggle auto-switches to ref2va): the clip is audio-VAE encoded and conditions the DiT as pinned latent rows, labeled <Audio j> in the prompt. ≤ 15 s per clip. Mechanics: docs/REF2VA-CONDITIONING.md.
reference video(s)	yes	Under ref2va (server/UI path): full latent conditioning — the clip (17k+5-snapped frames on the released canvas) becomes pinned video rows, its soundtrack rides along as audio rows, and Qwen sees it at 2 fps as timestamped <Video k> blocks. Under fl2va (and the CLI): 4 evenly-spaced frames as vision blocks (the lighter path). Mixing kinds in one request works — images, videos and audio condition together in request order. base_video continuation/editing remains hosted-API-only.
last-frame image	yes (fl2va)	--last-image / last_image_id, or "pin as last frame" on an image chip in the UI. Pinned as condition rows at the clip's final frame.

Parameters: this port, the open release, the hosted API

Everything the model can be steered with, in one place: what this port exposes, what the open release supports beyond that, and what only exists behind MiniMax's hosted API.

Every knob in this port

Both partitions get the identical set — --partition swaps which 21 GB DiT is loaded (fl2va or ref2va), and every other parameter passes through the same sampler unchanged.

parameter	CLI (generate)	API (POST /api/ai/queue)	reaches the model as	default / range
prompt	-p / -f	prompt	Qwen3-VL text conditioning	required; ≤ 8000 chars here (hosted API caps at 7000)
reference images	-i, repeatable	image_ids (≤ 9, from POST /api/images)	vision blocks in the prompt; the first is also the fl2va first-frame condition and geometry anchor	none
last frame	--last-image	last_image_id (a still)	fl2va "last" keyframe anchor: pinned condition rows at the final frame's rotary position, plus the final vision block	none
frame count	--frames	frames	num_frames	124; snapped down to the VAE's 17n+5 grid
clip length	-s/--seconds	seconds	converted at 24 fps, snapped up	alternative to frames
canvas	--width/--height/--size	width, height	height, width	from image aspect, else 1280×704; multiples of 32
steps	--steps	steps	num_inference_steps	28 (N yields N−1 forwards)
seed	--seed	seed	CPU torch.Generator	random; the server draws and records one, so history is always reproducible
audio track	--no-audio	no_audio	with_audio (skips the audio VAE decode)	on; audio is denoised jointly either way
partition	--partition	partition	which DiT checkpoint	fl2va
H.264 quality	--crf, --preset	crf, preset	ffmpeg only — not a model input	18 / medium

Two sampler arguments are accepted and ignored by design: unconditional_embeds and guidance_scale. The model is guidance-distilled — no negative prompt, no CFG, one forward per step — so there is nothing to hook them to; they exist for harness compatibility.

In the open release but not wired up here

Gaps between this port's sampler (vendored from ai-toolkit's preview sampler) and what the released architecture supports — each is a natural extension point:

• Still images. The pipeline has a single-image path (num_frames <= 1 returns PIL stills); the CLI and server only expose the video path.
• Initial latents. pipeline(latents=...) accepts caller-supplied starting noise (the hook for inversion / variation tricks); not surfaced.

Only in the hosted API (not in the open weights)

• H3-Regenerate-2K — the 2K upscaling pass behind the API's resolution: "2K" (1440 px short edge). Not open-sourced; the open release is native 768 on the short edge.
• H3-Context-IR — the hosted API's multimodal prompt interpretation layer (it deeply reads the references and produces a structured, enriched prompt). Context-IR is NOT required for reference conditioning itself — the open weights fully support reference video/audio conditioning, and this port implements that layout (ref_packing.py / ref_pipeline.py, server/UI path; mechanics in docs/REF2VA-CONDITIONING.md). What is missing relative to the hosted API is only the prompt-enrichment layer; the optional Oxen prompt-rewrite feature is a lightweight stand-in.
• Request-level conveniences — duration (integer 4–15 s), resolution (768P | 2K), ratio (preset aspect ratios or adaptive), and content roles (first_frame, last_frame, reference_image, reference_video, reference_audio, base_video). This port's equivalents are the explicit frames/seconds + width/height knobs above; base_video (continuation/editing) has no open-release counterpart at all.

Install

Requires an existing PyTorch environment; /home/ox/.venv already has everything (torch 2.11.0+cu130, transformers 5.12.1, rich, soundfile, pynvml) plus ffmpeg on PATH.

cd /home/ox/Code/AI/MiniMax-H3-Inference
/home/ox/.venv/bin/python -m mmh3 check      # verify weights + GPUs

For a fresh environment: pip install -r requirements.txt and apt install ffmpeg. Or install it as a package (adds the mmh3 command):

pip install -e ".[server,dev]"
python -m pytest          # the test suite runs without GPUs, in seconds

More examples

PY=/home/ox/.venv/bin/python

# explicit widescreen canvas, fixed seed, more steps
$PY -m mmh3 generate -p "the cat leaps" -i ref.png \
    --size 1344x768 --frames 124 --steps 28 --seed 42 -o out.mp4

# only one axis: the other follows the reference image's aspect ratio
$PY -m mmh3 generate -p "the cat leaps" -i ref.png --width 1024 -o out.mp4

# first -> last interpolation: start on start.png, end on end.png
$PY -m mmh3 generate -p "the cat leaps onto the windowsill" \
    -i start.png --last-image end.png -o out.mp4

# ending only: the clip lands on this image (it sets the canvas aspect)
$PY -m mmh3 generate -p "a slow push-in on the trophy" --last-image end.png -o out.mp4

# the ref2va partition instead of fl2va
$PY -m mmh3 generate -p "..." -i ref.png --partition ref2va -o out.mp4

# silent output, and keep the raw frames
$PY -m mmh3 generate -p "..." --no-audio --frames-dir frames/ -o out.mp4

Run python -m mmh3 generate --help for the full flag list.

Resolution

Three equivalent ways to set the output size:

$PY -m mmh3 generate -p "..." --width 1344 --height 768 -o out.mp4
$PY -m mmh3 generate -p "..." --size 1344x768              -o out.mp4   # shorthand
$PY -m mmh3 generate -p "..." --size 768                   -o out.mp4   # square

Rules:

• Multiples of 32 on both axes — 16× VAE spatial compression × the 2×2 transformer patch. Off-grid values are snapped to the nearest multiple with a warning (--width 700 → 704) rather than rejected. Minimum 32.
• One axis is enough. Pass only --width (or only --height) and the other follows the first reference image's aspect ratio, or 16:9 if there is no image: --width 1024 with a 16:9 reference gives 1024×576.
• Pass nothing and the canvas comes from the reference image via the released recipe — short edge 768, area capped at 768×1344, rounded to 32 (a 1920×1080 reference gives 1344×768). With no image at all it is 1280×704 — 720p 16:9, with 720 snapped down to the 32-pixel grid.
• --size and --width/--height are mutually exclusive.

Verified working: 256×256, 512×512, 768×768, 1024×576 and 1280×704 (including a 15 s clip). The model's native short edge is 768; much below that costs visible quality.

Frames and steps

• --frames snaps down to the video VAE's 17n+5 grid: 5, 22, 39, 56, 73, 90, 107, 124 (the default, 5.17 s at 24 fps), 141, ...
• -s, --seconds is the same knob in time units: it multiplies by 24 fps and snaps up to that grid, so --seconds 5 gives 124 frames (5.17 s), never less than requested.
• --steps N performs N-1 model evaluations: the sigma grid includes the terminal 0 and collapses duplicates, so --steps 28 runs 27 forwards.
• Cost scales with pixels × frames: 768×768/22f/16 steps ≈ 0.7 s/step on one RTX 6000 Ada.

Timing report

Every run ends with a per-stage breakdown, rollups, and throughput — so it is obvious whether time went into loading weights or into actual inference:

Real output from --size 768 --frames 90 --steps 24:

 stage               ┃      time ┃  share
━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━
 resolve checkpoints │     0.00s │   0.0%
 text encoder load   │    15.17s │   8.3%
 prompt encode       │     0.69s │   0.4%
 VAE load            │    10.59s │   5.8%
 transformer load    │     3.16s │   1.7%
 denoise             │  2m 30.3s │  82.5%
 video decode        │    10.67s │   5.9%
 audio decode        │     0.15s │   0.1%
 mp4 encode          │     1.18s │   0.6%
─────────────────────┼───────────┼────────
 model loading       │    28.91s │  15.9%
 inference           │  2m 41.8s │  88.8%
 mp4 encode          │     1.18s │   0.6%
 other/overhead      │     0.00s │   0.0%
─────────────────────┼───────────┼────────
 total               │  3m 02.1s │ 100.0%

  per denoising step: 6.53s
  output: 768x768, 90 frames, 3.75s of video
  throughput: 0.6 frames/s of inference
  realtime factor: 48.6x slower than realtime (3.75s video in 3m 02.1s)
  megapixel-frames: 53.1 MP at 0.33 MP/s

Durations switch to Xm YY.Ys past a minute and Xh MMm SS.Ss past an hour; anything shorter stays in seconds with hundredths, so sub-second stages stay readable. other/overhead absorbs unattributed time, so the rows always add up to the real wall clock.

keyframe encode (only with -i), video decode and audio decode happen inside the vendored sampler, so they are measured in the model adapter and folded in at the end — STAGE_ORDER in ui.py keeps the table in execution order. --plain prints the same numbers as log lines instead of a table.

Watch the inference row rather than total when comparing settings: loading is a fixed ~29 s, so it dominates short clips (78% of a 22-frame run) and becomes noise on longer ones (16% here).

Under the hood

The parts of this implementation a profiler or debugger would eventually reveal, collected up front. Two 30B-class models (the 33B DiT and a 32B Qwen3-VL conditioner) plus two VAEs run on a pair of 48 GB cards; everything below is in service of that fit.

• The weights only fit because they ship quantized. The transformer is int8-ConvRot (21 GB instead of ~66 GB bf16) and the text encoder is nvfp4-AWQ (15.7 GB instead of ~64 GB). The loader attaches the pre-quantized layers as-is — 200 ConvRot layers in the DiT, 351 nvfp4/int8 layers in the conditioner — so nothing is ever materialized at full precision. Ada has no native fp4 GEMM, so these kernels dequantize per-matmul (see Performance note).

• The text encoder stops at layer 50 of 64. MiniMax-H3 conditions on hidden_states[50], so decoder layers 50–63 are never even allocated, the lm_head is dropped, and the final norm is replaced with Identity — transformers normalizes hidden_states[-1] in place, and the model wants the unnormalized layer-49 output.

• The DiT gets a card to itself. Activation transients scale with clip length: a 15 s 720p clip packs ~95k rows into one sequence and peaks at ~23 GB of forward transients on top of the 21 GB of weights (the fc1 output alone is 5.5 GB). So the auto-policy parks the text encoder and both VAEs on the second card — the VAEs only run at the ends of a job, so they cost nothing there — and keeps the DiT card clear. Free VRAM on that card is the binding constraint on clip length. Details and the single-GPU fallback: GPU placement.

• The allocator is told about the phase change. Denoising and decoding allocate multi-GiB buffers in different shapes; with the default CUDA caching allocator that fragments into "reserved but unallocated" gigabytes and a spurious OOM. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is set at import (your own value wins) so segments grow in place instead.

• Progress bars count transformer forwards. The model is guidance-distilled — exactly one forward per denoising step — so a transparent proxy around the transformer gives exact step progress (CLI bars, the server's SSE progress field) without touching a line of the vendored sampling loop.

• Load order is choreography, not chance. On a single card the text encoder loads first, encodes the prompt, and is freed before the 21 GB transformer loads — the two never coexist. The server's resident engine keeps that ordering on its first job, then skips straight to prompt encoding on every later one (~30 s of loading amortized away; a warm second job starts in ~2 s).

• Cross-GPU copies are never trusted — every transfer stages through host memory. On this machine direct peer copies silently corrupt while can_device_access_peer() reports success, and worse: which direction corrupts flips between processes, so even a passing integrity probe does not bind the copies made minutes later (first symptom: every prompt made the same unconditional video; second, subtler one: the server decoded noise or black frames because the denoised latents reached the VAE as stale VRAM contents while the startup probe had said the path was fine). devices.safe_to therefore stages all cuda↔cuda transfers via CPU unconditionally — one PCIe round trip per job, noise next to the render time. p2p_copy_ok survives as a diagnostic shown by mmh3 check. Any new cross-device path must go through safe_to (CUDA-SETUP.md has the full story).

• Triton launches on the "current" device, not the tensors' device. The quantized kernels die with a cryptic Pointer argument cannot be accessed from Triton (cpu tensor?) if anything — even a VRAM query — has switched the CUDA context when a cuda:1 forward starts. Every off-device call is wrapped in an on_device guard; upstream never hits this because it keeps everything on one card.

• Frames stream straight into ffmpeg. The encoder reads rawvideo from a pipe (no intermediate frame files), audio is muxed from a temp wav, and -shortest is deliberately not used — the 40 Hz audio-latent grid and the 24 fps video grid rarely end on the same millisecond, and -shortest would silently drop the final frame.

• Every result is reproducible. The server draws a seed when you don't give one and records it with the generation, so any clip in the history can be rerun exactly. Keyframe VAE encoding uses a fixed seed (42, from the released recipe) independent of the request seed — conditioning stays identical across seeds by design.

Checkpoints

Files are located without any configuration if the machine is already set up for ai-toolkit training. The search order mirrors upstream's _resolve_comfy_file:

1. an explicit --dit-path / --text-encoder-path / --video-vae-path / --audio-vae-path
2. $MODELS_PATH/<comfy-relative path> (default MODELS_PATH=/data/shared/models)
3. $MODELS_PATH/<bare filename>
4. the Hugging Face cache (HF_HOME, here /data/shared/hf_cache)
5. the hub — only with --allow-download

component	file	size
transformer (fl2va)	diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors	20.97 GB
transformer (ref2va)	diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors	20.97 GB
text encoder	text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors	15.69 GB
video VAE	vae/minimax_h3_video_vae_fp16.safetensors	5.21 GB
audio VAE	vae/minimax_h3_audio_vae_fp32.safetensors	0.61 GB

These specific variants are required. The loader infers the pruned checkpoint's factored timestep table (adaln_t_table) and raises on any key mismatch, so the _bf16 and non-pruned _int8_convrot files from the same repo will not load. Tokenizer, processor and text-encoder config come from MiniMaxAI/MiniMax-H3 under FL2VA/ (~24 MB, cached); point --original-repo at a local directory to avoid the hub entirely.

GPU placement

--device/--te-device/--offload override an automatic policy:

• 2+ GPUs — text encoder and VAEs on the second card, transformer alone on the first. Nothing is offloaded (this is what the 2× RTX 6000 Ada setup gets). The DiT card is kept clear because activation transients scale with clip length — a 15 s 720p clip packs ~95k rows into one sequence and peaks at ~23 GB of transients on top of the 21 GB of weights; with the VAEs also on that card it OOMs. If the second card can't hold text encoder + VAEs together, the VAEs fall back to the DiT card (capping workable clip length).
• 1 GPU — everything on it, with the text encoder freed after the prompt is encoded. All three resident is ~43 GB of weights plus activations, which does not fit a 48 GB card comfortably.
• --device cpu — runs, but a 33B DiT on CPU is impractically slow.

Each stage logs which device it loaded onto plus VRAM before/after, the denoising bar carries live SM/VRAM/temperature from NVML, and the final panel reports peak allocation per device.

Performance note

The text encoder is nvfp4 (AWQ) and the transformer is int8-ConvRot. Ada is sm89 and has no native fp4 GEMM, so the vendored kernels take a dequantize-then-matmul path; expect the text encoder to be slower per token than on Blackwell. The weights still occupy only their quantized footprint. Triton is optional — the kernels fall back without it, more slowly.

Verification

Run against the real checkpoints on this machine, without a GPU:

• checkpoint resolution and size checks for all five files
• tokenizer / processor / config load (Qwen2Tokenizer, Qwen3VLProcessor, Qwen3VLConfig, 64 layers available, 50 used)
• self-containment: all 27 modules import with nothing from ai-toolkit on sys.path, and the vendored toolkit still wins when a real ai-toolkit checkout is put first on PYTHONPATH
• load_text_encoder — 351 pre-quantized nvfp4/int8 layers attached, stack truncated to 50 decoder layers, final norm replaced with Identity, no lm_head
• load_transformer — 200 ConvRot int8 layers attached, 50 blocks, pruned adaln_t_table (1025, 8) detected, 0 leftover meta tensors
• load_vaes — strict load_state_dict for both VAEs (560/560 and 915/915 keys)
• real audio decode: 40 latents → (1, 1, 32000) = 1.00 s at 32 kHz
• real video decode: (1, 24, 2, 4, 4) → (1, 3, 5, 64, 64) in [-1, 1]
• full sampler integration with a stub transformer: keyframe encode → packed layout → denoise loop → VAE decode → mp4, confirming step accounting (--steps 4 → 3 forwards) and progress ticks
• mp4 muxing: h264 + AAC 32 kHz stereo, silent path, and ffmpeg failure surfacing
• placement policy across 2-GPU, 1-GPU, small-GPU, explicit-override, forced-offload and CPU inventories

On GPU, end-to-end:

• full generations at 256x256/5f, 512x512/39f, 768x768/22f and 15 s at 1280x704/362f, with the reported placement (text encoder + VAEs cuda:1, transformer alone on cuda:0) and no offloading
• prompt adherence: two unrelated prompts at the same seed differ by 86.9/255 mean pixel difference and each visibly matches its prompt
• image conditioning: with -i, frame 0 reproduces the reference image to 4.3/255 and subsequent frames develop coherent motion from it
• first+last anchoring (2026-08-06): with -i and --last-image set to two unrelated images, the output's first frame correlates 0.98 with the first anchor and its last frame 0.70 with the last anchor (cross-correlations ~0.2)
• audio and video durations agree (e.g. 1.62 s each for 39 frames)
• two multi-GPU bugs found this way and fixed: Triton kernels launching against the wrong CUDA device after a mem_get_info() call, and silently corrupt cross-device tensor copies (see the status note above)

python -m mmh3 check gates self-containment on every run: the vendored quantization backends import each other lazily inside get_ostris_quantizer, so a missing vendored module would otherwise surface only partway through a 15 GB model load. check resolves them in milliseconds and fails loudly instead.

Layout

mmh3/
  cli.py             argparse CLI: generate, check, serve
  generate.py        one-shot orchestration: resolve -> place -> load -> sample -> encode
  engine.py          resident engine for the server: load once, generate many
  loader.py          ports upstream's three load methods (transformer/TE/VAEs)
  model.py           the interface the vendored sampler expects + prompt encoders
  devices.py         GPU probing, placement policy, safe cross-GPU copies, VRAM/NVML
  weights.py         checkpoint resolution
  ref_conditioning.py  ref2va media loading, VAE encoding, Qwen presentation
  ref_packing.py     ref2va packed-sequence layout (references as pinned rows)
  ref_pipeline.py    ref2va sampler over the vendored transformer
  oxen.py            stdlib Oxen Hub client (optional mirroring)
  video_out.py       ffmpeg mp4 muxing
  ui.py              logging, spinners, progress bars
  server/            FastAPI queue API: db (SQLite), worker thread, SSE, Oxen bridge, mock engine
  vendor/            verbatim upstream copies - see vendor/PROVENANCE.md
web/                 React + Vite gallery UI; `npm run build` -> web/dist, served by `serve`
docs/                REF2VA-CONDITIONING.md, OXEN-INTEGRATION.md
tests/               pytest suite; runs without GPUs (`python -m pytest`)

vendor/ holds unmodified files so they stay diffable against upstream; a sys.path shim in vendor/__init__.py makes their from toolkit.util... imports resolve to the vendored copies. minimax_h3.py (the ai-toolkit BaseModel subclass) is deliberately not vendored — it depends on the training stack — so loader.py and model.py supply what the sampler needs instead.

Troubleshooting

No usable CUDA device found with NVIDIA cards listed on the PCI bus means the kernel driver is not loaded. On this machine (diagnosed 2026-08-04) the cause was missing kernel headers for the running kernel, which broke the nvidia module link step and left the driver packages half-configured. Fix:

sudo apt install -y linux-headers-$(uname -r) linux-headers-oem-24.04d
sudo dpkg --configure -a
sudo modprobe nvidia && nvidia-smi

Note that apt --fix-broken install alone does not fix it — the headers must be installed first. Full diagnosis, evidence and verification commands are in CUDA-SETUP.md.

transformer load mismatch: missing [...] means a wrong checkpoint variant — use the _pruned_int8_convrot file, not _bf16 or the non-pruned int8.

CUDA OOM during denoising — reduce --frames or the canvas, or pass --offload to free the text encoder before the transformer loads.

Frames but no audio — check for a soundfile/ffmpeg AAC error in the log; --no-audio skips audio decode entirely.