Released `exp_for_mtrans` Part_TMR checkpoint has a collapsed motion-embedding space: top-1 retrieval returns a single motion for 76% of all queries

[u7079256]

Summary

The Part_TMR retriever checkpoint that the released mask/residual transformer checkpoints are paired with (Part_TMR/checkpoints/exp_for_mtrans/HumanML3D/best_model.pt) maps almost all motions to nearly the same direction in embedding space: randomly sampled pairs of distinct motions have a mean cosine similarity of 0.93. As a direct consequence, when a full retrieval database is built with this checkpoint, top-1 retrieval degenerates into a near-constant lookup: across all 67,010 caption-level queries over 22,418 unique motions, only 97 distinct motions (0.43%) are ever returned at top-1, and a single motion (004128) alone receives 75.94% (50,890/67,010) of all top-1 hits.

This appears specific to this checkpoint rather than inherent to the architecture: the sibling checkpoint released in the same repository, Part_TMR/checkpoints/exp_pretrain/HumanML3D/best_model.pt (same architecture and embedding size; loads into the same model class with 0 missing / 0 unexpected keys), shows none of this under the identical protocol.

Affected assets

• https://huggingface.co/lycnight/ReMoMask/tree/main/Part_TMR/checkpoints/exp_for_mtrans/HumanML3D (the checkpoint selected by the default exp_name: exp_for_mtrans in

  ReMoMask/Part_TMR/conf/config.yaml

  Line 1 in 00499c3

  exp_name: exp_for_mtrans

  )
• Healthy comparison checkpoint: https://huggingface.co/lycnight/ReMoMask/tree/main/Part_TMR/checkpoints/exp_pretrain/HumanML3D

Steps to reproduce

1. Build the full retrieval database from the HumanML3D training split with build_rag_database.py and the exp_for_mtrans checkpoint (the script's L2-normalized encode_motion(...)['global'] / encode_text(...) outputs). This yields 67,010 caption-level rows covering 22,418 unique motions (unique = after stripping the per-caption suffix from motion_ids.npy entries).
2. Embedding-space collapse: sample 10,000 random pairs of distinct motions and compute the cosine similarity of their motion embeddings. Result: mean 0.9270, std 0.0574.
3. Retrieval concentration: for each of the 67,010 text embeddings, take the top-1 motion by cosine similarity against all motion embeddings, then aggregate hits per unique motion. Result: motion 004128 receives 50,890 hits (75.94%); 97 unique motions are ever hit; the Gini coefficient of the hit distribution is 0.9998; 2 motions cover 80% of all hits.
4. Control with the sibling checkpoint: load exp_pretrain into the same model class (loads cleanly, 0 missing / 0 unexpected keys), re-encode a fixed random sample of 1,500 (motion, caption) pairs drawn from the same underlying data (seed 3407), and repeat steps 2-3 on identical inputs.

Environment: single NVIDIA GPU, PyTorch, float32; all embeddings L2-normalized before cosine computation (as in build_rag_database.py).

Evidence

metric	exp_for_mtrans (paired with released transformers)	exp_pretrain (sibling, same repo)
random distinct-motion pair cosine, mean (std)	0.9270 (0.0574), n=10,000 pairs	0.0916 (0.1562), n=10,000 pairs
top-1 share of the single most-retrieved motion	75.94% (50,890 / 67,010)	0.47% (7 / 1,500)
unique motions ever retrieved at top-1	97 / 22,418 (0.43%)	1,134 / 1,468 sampled (77.2%)
Gini coefficient of the top-1 hit distribution	0.9998	0.3807
motions needed to cover 80% of top-1 hits	2	834 (56.8% of the pool)

The dominant-motion id (004128) and its hit count were additionally re-derived with an independent implementation (blocked matrix multiplication over the saved arrays) and matched exactly.

Impact

Retrieval built on the released exp_for_mtrans checkpoint returns essentially the same handful of motions regardless of the query text. Any retrieval-augmented behavior of the released transformer checkpoints — and any project building on them — is therefore conditioned on near-constant retrieval features rather than query-relevant motion exemplars.