- Datasets bridging computer vision and robotics by providing high-quality visual - annotations in manipulation-relevant scenes remain limited. - This work introduces the Multi-Object Photoreal Simulation (MOPS) - dataset, which provides comprehensive ground truth annotations for photorealistic - simulated environments. MOPS employs a zero-shot asset augmentation pipeline based on - Large Language Models (LLM) to automatically normalize 3D object scale and generate - part-level affordances. The dataset features pixel-level segmentations for tasks - crucial to robotic perception, including fine-grained part segmentation and affordance - prediction (e.g., “graspable” or “pushable”). - By combining detailed annotations with photorealistic simulation, MOPS generates a - vast, diverse collection of scenes to accelerate progress in robot perception and - manipulation. We validate MOPS through vision and robot learning benchmarks. + Datasets providing high-quality visual annotations in manipulation-relevant scenes + remain scarce. We introduce MOPS, a dataset generation framework that + combines 3D assets from PartNet-Mobility and RoboCasa with a zero-shot LLM-based + augmentation pipeline to automatically normalize object scale and generate part-level + affordance annotations, describing how an object part can be manipulated (e.g., a mug + handle is “graspable,” a drawer is + “pullable”). + Built on ManiSkill3, MOPS produces photorealistic indoor scenes with pixel-perfect + ground truth for class, part, and instance segmentation, multi-label affordances, depth, + surface normals, and 6D poses, spanning 54 affordance types across 137 object + categories. Human verification confirms 97.3% accuracy of the zero-shot + affordance labels. We validate MOPS on three vision benchmarks of increasing scene + complexity and show that ground-truth affordance masks improve imitation learning + success rates on 24 RoboCasa manipulation tasks by 7.9 percentage + points + over RGB-only baselines, with predicted affordances still yielding measurable gains. + The dataset and framework are publicly available.
Zero-shot asset augmentation using large language models for - automatic part-level labeling, scale normalization, and semantic understanding.
+ automatic part-level labeling, scale normalization, and semantic understanding + — 97.3% accurate against human verification.Kitchen environments, cluttered tabletops, and isolated object - scenarios spanning 137 object categories and 56 affordance labels.
+ scenarios spanning 137 object categories and 54 affordance labels.mops-data — Image generation in ManiSkill3
+ mops-repo — Generation framework & benchmarks
+ Available
+
+
+ 🤗
+ HuggingFace — MOPS dataset collection
Available
-
- 🤖
- mops-il — Robot trajectories in RoboCasa v0.1
- Coming Soon
-
- Prerequisites: Python 3.10 · - CUDA-compatible GPU · 16 GB+ RAM -
-conda create -n mops python=3.10
-conda activate mops
-
-pip install mani_skill
-git clone https://github.com/LiXiling/mops-data
-cd mops-data
-pip install -e .+ + 📖 Setup & Installation Guide → + +
diff --git a/index.md b/index.md deleted file mode 100644 index 3c8570f..0000000 --- a/index.md +++ /dev/null @@ -1,348 +0,0 @@ ---- -layout: project_page -permalink: / - -title: "MOPS: Multi-Object Photoreal Simulation Dataset for Computer Vision in Robot Manipulation" -authors: - Maximilian X. Li, Paul Mattes, Nils Blank, Rudolf Lioutikov -affiliations: - Intuitive Robots Lab, Karlsruhe Institute of Technology, Germany -paper: ./static/Li2026_MOPS.pdf -code: https://github.com/LiXiling/mops-data -#video: https://www.youtube.com/results?search_query=turing+machine -#data: https://huggingface.co/docs/datasets ---- - - -MOPS provides rich, multi-modal ground truth for every scene
-
- 
- 
- 
- High-quality visual rendering via ManiSkill3 and SAPIEN, optimized for computer vision tasks in robotic manipulation.
-Zero-shot asset augmentation pipeline using large language models for automatic part-level labeling and semantic understanding.
-Detailed ground truth for fine-grained part segmentation and affordance prediction (e.g., graspable, pushable).
-Rich indoor scenes including kitchen environments, cluttered tabletops, and isolated object scenarios at scale.
-Normalized asset management across multiple 3D libraries with automatic part-level annotation and semantic scene understanding.
-Comprehensive annotations including RGB, depth, surface normals, segmentation masks, affordance maps, and 6D pose information.
-Built on ManiSkill3 and SAPIEN for physics-accurate simulation with photorealistic rendering and programmable scene generation.
-MOPS provides significantly broader taxonomic coverage than existing datasets
-| Dataset | -Level | -Aff. Labels | -Obj. Cat. | -Objects | -
|---|---|---|---|---|
| RGB-D Part | -Part | -7 | -17 | -105 | -
| 3D-AffNet | -Part | -16 | -23 | -22,949 | -
| MOPS-Partnet | -Part | -24 | -46 | -2,345 | -
| MOPS-Robocasa | -Object | -44 | -101 | -1,008 | -
| MOPS (Total) | -Mixed | -56 | -137 | -3,353 | -
While 3D-AffNet has more instances, MOPS provides significantly higher taxonomic coverage across object categories and affordance types.
-Imitation learning on 24 RoboCasa tasks, evaluated over 10 environment seeds each
-| Policy Inputs | -Success Rate | -Gain | -
|---|---|---|
| RGB only | -13.33% | -— | -
| RGB + MOPS Affordances | -21.25% | -+7.92 | -
MOPS affordance annotations provide a consistent boost to imitation learning performance across 24 RoboCasa manipulation tasks.
-Prerequisites: Python 3.10 · CUDA-compatible GPU · 16 GB+ RAM
-conda create -n mops python=3.10
-conda activate mops
-
-pip install mani_skill
-git clone https://github.com/LiXiling/mops-data
-cd mops-data
-pip install -e .If you use MOPS in your research, please cite our work
-@article{li2026mops,
- title = {Multi-Objective Photoreal Simulation (MOPS) Dataset
- for Computer Vision in Robot Manipulation},
- author = {Maximilian Xiling Li and Paul Mattes and
- Nils Blank and Rudolf Lioutikov},
- year = {2026}
-}This work is supported by the Intuitive Robots Lab at Karlsruhe Institute of Technology, Germany.
-