WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes

Jichen Hu, Jiawei Guo, Jiazhong Cen, Chen Yang, Sikuang Li, Wei Shen

"WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes"

🔥🔥🔥News

• 2026-05-14: This repository is released.

---

Recent 3D world modeling systems based on generative scene synthesis, such as Marble, can create coherent and explorable 3D environments, yet their outputs are typically static monolithic assets with limited editability and physical interaction. This restricts their use in immersive content creation and embodied simulation, where generated worlds must be actively modified and manipulated. To tackle this challenge, we present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes. WorldAct uses a multimodal agent to guide scene decomposition, identify actionable objects, reconstruct geometrically aligned object-level meshes for interaction, and restore the residual background via 3D inpainting. The resulting scenes support object-level editing, collision-aware manipulation, and embodied task execution while preserving global scene coherence. Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.

---

Pipeline

---

TODO

• Release training code.
• Publish datasets.

Contents

1. Testing
2. Results
3. Citation
4. Acknowledgements

Testing

TBD

Results

We present the performance of our proposed WorldAct framework.

Quantitative Results (click to expand)

• Results in Tab. 1 and Tab. 2 of the main paper

  

Qualitative Results (click to expand)

• Results in Fig. 4 of the main paper

  

• Results in Fig. 5 of the main paper

  

Citation

If you find our model or code helpful in your research or work, please cite the following paper.

@article{hu2026worldact,
      title={WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes},
      author={Jichen Hu and Jiawei Guo and Jiazhong Cen and Chen Yang and Sikuang Li and Wei Shen},
      journal={arXiv preprint arXiv:2605.15843},
      year={2026}
}

Acknowledgements

This project builds on numerous model repositories. We thank Marble and SAM3D for the research ideas that inspired this work.