MaskedMimic (SMPL) pretrained checkpoint incompatible with ProtoMotions3 (main)

[zyr-ai]

Context

I downloaded the pretrained MaskedMimic (SMPL humanoid) checkpoint from

https://huggingface.co/ctessler/MaskedMimic, which ships with a Hydra-style

config.yaml. The HuggingFace instructions say to run:

python protomotions/eval_agent.py +robot=smpl +simulator=isaaclab \

+motion_file= \

+checkpoint=data/pretrained_models/masked_mimic/smpl/last.ckpt

However, the current main branch (ProtoMotions3) has been refactored to use

Python experiment files + pickled resolved_configs, and no longer uses

eval_agent.py / Hydra. It now uses inference_agent.py and requires a

companion file resolved_configs_inference.pt next to the checkpoint — which

the old checkpoint does not include.

Running inference_agent.py with the old checkpoint fails with:

AssertionError: Could not find resolved configs at .../MaskedMimic/resolved_configs_inference.pt

Technical investigation

• The new resolved_configs_inference.pt is a dict with 7 keys: robot,

  simulator, terrain, scene_lib, motion_lib, env, agent — each

  value is a dataclass config with a target attr, saved via

  torch.save(..., weights_only=False).

• The old Hydra config references env class

  protomotions.envs.mimic.env.Mimic and agent class

  protomotions.agents.masked_mimic.agent.MaskedMimic — both classes no

  longer exist on current main.

• The old model architecture uses MultiHeadedMLP, MLP_WithNorm,

  VaeModule, VaeDeterministicOutputModel with state_dict prefixes like

  _encoder._trunk.input_models.* — entirely different from the current

  model's ModuleContainerConfig.models structure.

• Loading last.ckpt directly into the current MaskedMimicModel (default

  config): all 141/141 checkpoint keys are reported as "unexpected", 0

  params map. This confirms it is not a config issue but a fully changed

  network architecture — the old weights cannot be reused as-is.

Questions

1. Is there any MaskedMimic (SMPL) checkpoint already re-trained / re-exported

   compatible with the resolved_configs_inference.pt format of ProtoMotions3

   (current main)? If so, please share the download link.

2. If not, do you plan to release a v3-compatible MaskedMimic pretrained

   checkpoint, and is there an estimated timeline (if convenient to share)?

3. I confirmed the network architectures between the two versions are

   completely different (state_dict does not map at all — see evidence

   above). Is converting old weights to the new architecture considered

   infeasible, or is there a way (e.g. an internal migration script your team

   used during the refactor) to do the conversion?

4. If I need to run MaskedMimic right now, is the fastest path to checkout the

   last commit that still had eval_agent.py (the Hydra version)? If so, does

   that version support a MuJoCo (CPU-only) simulator, since I need to run on

   a machine without an NVIDIA GPU?

Thanks for your time and the great work on ProtoMotions!

Traceback

Traceback (most recent call last):

File "~/ProtoMotions/protomotions/inference_agent.py", line 468, in

main()

File "~/ProtoMotions/protomotions/inference_agent.py", line 245, in main

assert (

AssertionError: Could not find resolved configs at ~/ProtoMotions/MaskedMimic/resolved_configs_inference.pt

[zyr-ai]

Update 2: policy loads and produces non-zero actions, but character never actually moves on MuJoCo

Continuing investigation after my previous comment. Summary of what I verified so far, in case it's useful:

Confirmed working:

• The shipped checkpoint's resolved_configs_inference.pt (binary) does contain a valid agent key (SupervisedAgent / MaskedMimicModel, rollout_actor=STUDENT) — my earlier read of the human-readable .yaml sidecar missing an agent: section was a false alarm; the actual .pt has it.
• The policy produces non-zero actions (~0.5–0.7 range) and these are correctly applied to self.data.ctrl in the MuJoCo simulator (verified via debug prints, ctrl reaches ~2.17 immediately after apply).
• Reset behavior is expected eval-loop behavior (motion resampling via motion_manager, not a termination bug). Tested with both a 2.5s and a 116s packaged AMASS test motion — same result either way.

Not working:

• Despite non-zero actions reaching ctrl, the character's root_pos never changes meaningfully across 5000+ simulation steps — it stays pinned at reference-pose values from the motion, i.e. no visible physical motion on --simulator mujoco.

Given the model card explicitly states 'Simulator expectation: Training simulator only' and 'Cross-simulator transfer should not be assumed' for this checkpoint, my current best guess is that this is a known IsaacLab-only limitation rather than a bug — but I don't have an NVIDIA GPU available to verify on IsaacLab directly, so I can't confirm whether the same checkpoint moves correctly there.

Could you confirm whether this checkpoint is expected to produce visible motion on --simulator mujoco, or whether MuJoCo support for this specific SMPL MaskedMimic checkpoint is not yet validated/supported? If it's a known gap, updating the model card or Pre-trained Models table to flag it explicitly (similar to how the G1 deployment tracker is flagged as transfer-validated) would save others the same investigation time.

Happy to share debug logs / repro steps if helpful.

[tesslerc]

Huggingface is outdated. We will fix that.
Please use the pre-trained models supplied in this repo: https://github.com/NVlabs/ProtoMotions/blob/main/data/pretrained_models/masked_mimic/smpl/MODEL_CARD.md

SMPL/SOMA characters are currently only supported in IsaacLab.
G1 pre-trained models are supported in all simulators.