Generate video with native audio from MiniMax H3 locally, with a local LLM that writes H3's structured prompts for you.
H3 Studio is a ComfyUI custom node for text-to-video generation with MiniMax H3 (also called Hailuo 3.0) — the open-weights omni-modal model that produces up to 15 seconds of video at 24 fps with synchronised 32 kHz stereo audio.
It installs as a custom node and serves a single page — no second service, no CORS setup, no build step.
H3 does not take free text. MiniMax's pipeline runs an LLM called H3-Context-IR that rewrites a casual request into a structured schema, and only that schema reaches the weights. Context-IR is closed-source and hosted, so in ComfyUI there is no rewriter — whatever you type goes to the model raw.
The format the weights actually expect looks like this:
integrated_multimodal_description: [Shot 1] Live-action, cinematic, a medium-wide
shot frames a baker opening the shutters of a small street bakery before sunrise.
The camera pushes in with small amplitude at slow speed as the middle-aged baker
with a calm, slightly raspy voice (S1) places a fresh loaf on the wooden counter
and says: <d>[English] First batch of the morning.</d>
overall_soundscape: Wooden shutters scrape open over a quiet street as trays clink
softly inside the bakery. The doorbell rings once, followed by light footsteps.
non_diegetic_music: A soft acoustic-guitar pattern at a moderate tempo, joined by
sparse upright-bass notes and a gentle fade at the end.
Three labelled fields, a closed camera vocabulary, <d>[Language] …</d> dialogue markup, (S1) speaker IDs, and timestamps on every shot after the first. It is documented in two guides in MiniMax's HF repo that most third-party tutorials never mention.
H3 Studio is a local, open-weights replacement for Context-IR. You write a sentence; a local LLM expands it into the schema above, validates it, and hands it to H3.
- Prompt assistant — expands a plain brief into H3's three-field format, or refines an existing prompt in place. Few-shot prompted with MiniMax's own published examples.
- Keyframes — drop a first frame, a last frame, or both. The task follows automatically: T2VA → I2VA → L2VA → FL2VA, with the required picture-alignment line generated for you (it carries the clip duration to two decimals, which models get wrong). The canvas refits to your image's aspect ratio, because the first frame is stretched rather than cropped.
- Subject references — attach one or two photos defining who is in the clip rather than a literal frame of it. Switches to the Ref2VA checkpoint and the six-section reference schema, with
<Subject N>definitions and retention analysis written for you. - Refine & generate — applies the edit and queues the clip on one click. Chains only on success, so a failed edit can't silently render the old prompt.
- Keeps your keystones — named people, places, directors, films and styles are carried through verbatim rather than paraphrased away. Proper nouns are detected automatically; add more in the Must keep box. Anything dropped is reported.
- Edits are diffs, not rewrites — Refine emits a small set of exact find/replace operations applied locally, so untouched sentences stay byte-identical and an edit costs a couple of hundred tokens instead of a full regeneration.
- Refinement remembers — each applied edit is replayed as prior conversation turns, so a follow-up like "even more" has something to be more than. Only edits that actually landed are remembered; typing in the prompt box by hand clears the context, since hand-edits invalidate the stored find strings.
- Format validation and repair — catches missing or misordered fields,
<d>without a language tag, a timestamp on Shot 1, non-increasing cut times, cuts past the clip end, dialogue duplicated into the audio fields, and the 7,000-character cap. Deterministic slips are repaired automatically; the rest surface as warnings. - Live preview while sampling, via ComfyUI's latent preview stream.
- Progress that means something — real percentage and ETA for sampling (measured from your last eight step times), pulsing for phases with no measurable total, token counts while the LLM writes, and a live VRAM meter.
- Automatic VRAM handoff — the LLM and H3 rarely fit on one consumer card. Each unloads the other before it runs, in both directions.
- Ephemeral output — clips are wiped unless you download them. See Privacy.
- Endpoint editor — point it at any ComfyUI and any OpenAI-compatible LLM.
cd ComfyUI/custom_nodes
git clone https://github.com/CharlesMod/h3-studio.git h3_studio
cd h3_studio && ./scripts/fetch_corpus.sh # optional but recommendedRestart ComfyUI, then open:
http://127.0.0.1:8188/extensions/h3_studio/index.html
fetch_corpus.sh downloads MiniMax's prompt guides and worked examples from Hugging Face and builds the few-shot corpus. They are not redistributed here — H3 ships under a custom community licence. Without them the assistant still runs, it just has fewer examples to imitate.
- ComfyUI 0.30.0+ (native H3 support; earlier versions have no H3 nodes)
- H3 weights — diffusion model, Qwen3-VL text encoder, and both VAEs, from Comfy-Org/MiniMax-H3
- An OpenAI-compatible LLM endpoint for the prompt assistant — llama.cpp, llama-swap, Ollama, vLLM, LM Studio. Optional; everything else works without it.
Open the Endpoints panel at the bottom of the sidebar.
| Field | Default | Notes |
|---|---|---|
| H3 / ComfyUI | the server that served the page | Stored in your browser. A different ComfyUI needs h3_studio installed too. |
| LLM | http://127.0.0.1:8080 |
Base URL only — /v1/chat/completions is appended. Stored server-side. |
| Model | auto-detect | Blank picks the best of what the endpoint serves. |
Test checks both and reports what it found. The LLM base can also be set with the H3_LLM_BASE environment variable, which wins over the UI and greys the field out; H3_LLM_MODEL does the same for the model.
Ollama users: point at http://127.0.0.1:11434 — its OpenAI-compatible endpoint lives at the same base.
H3 needs roughly 15.5 GB with the NVFP4 build. A 12B LLM at Q8 needs about 14.7 GB. On a 16 GB card they cannot both be resident, so:
- Clicking Generate unloads the LLM first
- Clicking Write prompt unloads H3's weights first (skipped if a job is running or queued)
Both show as a phase in the progress bar. The VRAM meter turns amber past 75% and red past 92%.
A Mixture-of-Experts model is a good fit for the assistant. Gemma 4 26B-A4B activates only 4B parameters per token, so its experts sit in system RAM and it holds ~8 GB rather than ~14.7 GB — while writing roughly 3× faster than a dense 12B.
scripts/vram_watchdog.py optionally releases H3's VRAM after an idle period so games and other GPU work can have the card back. Wire it up with deploy/comfyui-vram.service.
Measured on an RTX 5080 (16 GB), 864×480, 124 frames, 20 steps, res_multistep + simple:
| First generation (cold) | ~150 s |
| Subsequent (weights cached) | ~100 s |
| Prompt written by Gemma 4 26B-A4B | ~45 s |
864×480 is the speed sweet spot, but not the quality one — see Faces and fine detail. 1344×768 peaked at 15.6 GB of 16.3 GB here and took over 5 minutes; the resolution dropdown warns you before you pick it.
torch.compile does not work with the quantised H3 builds: inductor fails on dynamic shapes in ComfyUI's weight-casting path. Don't waste time on it.
If eyes and other small features come out smeared or "swirly", two things are usually responsible, and neither is fixed by a LoRA — as of now no MiniMax H3 detail LoRA exists.
1. SageAttention corrupts H3's attention. ComfyUI's dispatcher defaults low_precision_attention to True, and H3's DiT does not pass it, so --use-sage-attention routes H3 through SageAttention's int8 QK path. H3 applies QK-RMSNorm plus a partial rope, leaving rotated and unrotated channels with very different statistics that int8 cannot carry. SageAttention 2 produces pure noise; SageAttention 1 survives it but pays in exactly this kind of high-frequency detail. Until that lands upstream, either drop --use-sage-attention or add the one-liner to comfy/ldm/minimax/model.py:
out = optimized_attention(q, k, v, self.heads, mask=None, skip_reshape=True,
transformer_options=transformer_options,
low_precision_attention=False) # H3 must opt outNine other model families in the same tree already opt out this way. Note this edits ComfyUI core, so a git pull will revert it.
2. The face has too few tokens to exist. The video VAE downsamples 16×, and the DiT patchifies 2×2 on top, so it reasons on a 32× grid. At 864×480 a medium-shot face spans about two tokens and an eye roughly 0.3 of one — there is no representation for it to live in. Two levers, both free:
| Canvas | Token grid | Eye ≈ |
|---|---|---|
| 864×480 | 27×15 | 0.28 tokens |
| 1024×576 | 32×18 | 0.34 tokens |
| 1344×768 | 42×24 | 0.45 tokens |
Generate at 1024×576 or higher, and frame closer — a close-up at 864×480 renders a better eye than a medium shot at 1344×768. The assistant is told to favour medium shots and close-ups when a subject reference is attached, for this reason.
Structurally, MiniMax did not release H3-Regenerate-2K, the stage that recovers fine facial detail in hosted Hailuo. Local H3-Base cannot fully match it.
Both VAE decodes must feed CreateVideo. H3's latent is a nested video+audio tensor pair, so the same sampler output goes to VAEDecode with the video VAE and VAEDecodeAudio with the audio VAE. Miss the second and you get a silent clip. The bundled workflows wire this correctly.
Three labelled fields — integrated_multimodal_description, overall_soundscape, non_diegetic_music — separated by blank lines, in that order. Free text works far less well because ComfyUI has no prompt rewriter. See Why this exists.
No. The released checkpoints are guidance-distilled: no negative_prompt, no guidance_scale, one forward pass per step. The ComfyUI node exposes a positive conditioning output only. Write exclusions as prose at the end of the description instead, e.g. "No text, subtitles, logos or watermarks of any kind."
Drop to 864×480 and 124 frames. On a 16 GB card, 1344×768 peaks around 15.6 GB and will OOM if anything else holds VRAM. Check nothing else is on the GPU — an LLM, a browser, a game — and note that H3 and a large local LLM cannot be resident together.
20 with res_multistep + simple is the community baseline. Quality degrades noticeably below about 15. Going to 25 helps slightly at proportional cost.
You need ComfyUI 0.30.0 or newer — H3 support landed day-0 in that release. Check for EmptyMiniMaxH3LatentAV, MiniMaxH3ImageToVideo, MiniMaxH3ReferenceToVideo, and MiniMaxH3SigmaShift.
Ref2VA uses a separate checkpoint — minimax_h3_ref2va_*, not minimax_h3_fl2va_*. H3 Studio swaps it for you when you attach a subject photo, but the file has to be in models/diffusion_models/ or the job fails at load. Keyframes and subject references cannot be combined: Ref2VA has no keyframe inputs, so filling one family clears the other.
The reference node's inputs are dot-namespaced in API-format graphs — ref_images.ref_image_0, not ref_image_0. A bare name is silently ignored: the graph validates, the job runs, and you get a perfectly clean video that references nothing. If you are hand-building graphs, this one costs an hour.
It shouldn't any more. Proper nouns in your brief are detected and pinned, and the model is told to name them directly rather than softening them into "resembling" or "inspired by" — the hedge discards the signal the name carried. H3's text encoder is Qwen3-VL 32B and carries real world knowledge, so a name is a stronger conditioning signal than a description of it. Add anything the detector misses to the Must keep box, and the assistant warns you if a term still gets dropped.
The job probably finished and the page missed the event — the usual cause is the
WebSocket dropping (server restart, laptop sleep, flaky network). The socket now
reconnects with backoff and the page polls /history every 5 seconds as a
fallback, so a completed clip appears even if the socket never recovers. Check
ComfyUI/output/video/ — if the file is there, the generation succeeded.
A reasoning model can spend its whole token budget thinking and never reach an answer — one edit here burned 9,264 characters of reasoning and emitted zero content. Requests now disable thinking (enable_thinking: false) and edits emit a small diff rather than a whole prompt, which keeps output in the hundreds of tokens. If it still happens you'll get an explicit message rather than a blank box; raise MAX_TOKENS in promptwriter.py.
Open the Endpoints panel and press Test. The base URL should not include /v1 — that is appended. If a generation just ran, H3's weights may still hold the card; the assistant frees them automatically, but only when no job is queued.
The first run after idle reloads weights, which is a one-time cost per load. Enable SageAttention (--use-sage-attention) if you have it. Don't bother with torch.compile — it fails on the quantised builds.
Generated clips are treated as disposable. A clip survives only if you press Download. Otherwise it is wiped when you close the tab, start another generation, press Discard now, restart the server, or after six hours unclaimed.
Keyframe and subject photos get the same treatment. They stay in your browser until you press Generate, upload under an h3_studio_kf_ name just before the run, and are wiped as soon as it ends — on completion, error, cancel, or tab close. Their time on disk is the run itself.
Wiping overwrites the file three times, renames it, unlinks it, and fsyncs the directory. Deletion is scoped strictly to files this tool created: for clips, names beginning h3_studio and ending .mp4 resolving directly inside output/video; for uploads, names beginning h3_studio_kf_ with an image extension resolving directly inside input/. Path traversal, symlink escape and lookalike names are all rejected — your own files in input/ are never candidates.
A run abandoned mid-flight (browser killed, machine slept) still finishes and writes a clip nothing is tracking. Those are swept too, once they are older than the six-hour threshold, so an orphaned job cannot leave a clip behind indefinitely. Reloading the page also interrupts the job it started, addressed by prompt id so a run queued by another client is never touched.
On an SSD this defeats undelete and file-carving, but it does not guarantee erasure. Wear-levelling means the physical flash pages rewritten are not necessarily the ones that held the original. For a hard guarantee use full-disk encryption or a RAM-backed output directory.
./scripts/verify_models.sh /path/to/ComfyUIThis exists because it caught something real. A community NVFP4 repo was serving .safetensors with a marker string appended past the end of the tensor data — every file in the repo, each with its own timestamp. The SHA matched Hugging Face's record, so it was in the upload, not the download. The script checks the safetensors header, compares the declared tensor extent against actual file size, and scans the tail.
Trailing bytes in a safetensors file are inert — the format executes no code — but they break ComfyUI's loader and mean the file was modified after it was built. Worth knowing before you load someone's quant.
__init__.py ComfyUI custom node: server routes
promptwriter.py LLM prompting, validation, repair
web/index.html the entire UI, no build step
corpus/ style rules (base + reference); fetch_corpus.sh adds the examples
workflows/ API-format and editor-format T2V graphs
scripts/ corpus builder, weight verifier, VRAM watchdog
deploy/ example systemd units and a llama-swap model entry
The prompt format, the camera vocabulary, and every worked example come from MiniMax's own documentation in the MiniMax-H3 repo. H3 itself is theirs, under the MiniMax H3 Community License. ComfyUI's day-0 H3 support made a local pipeline possible at all.
Code in this repo is MIT licensed. The model weights and MiniMax's documentation are not — see their licence.