Sim2Sim2Sim

Official implementation of Dynamics Distillation for Efficient and Transferable Control Learning.

A progressive framework for learning driving policies through multi-stage simulation transfer. This project bridges high-fidelity vehicle simulation and scalable reinforcement learning by distilling simulator dynamics into a highly parallelizable learned dynamics model, enabling efficient policy optimization with reliable transfer to realistic environments.

Key Contributions

Robust control policy learning for autonomous driving requires training environments to be both physically realistic and computationally scalable, properties that existing simulators provide only in isolation. This work demonstrates:

• Efficient policy optimization through training purely within distilled simulation environments.
• Reliable transfer under challenging dynamics when deploying policies to high-fidelity simulators.
• Novel evaluation methodology showing that predictive accuracy alone does not fully characterize a learned dynamics model's suitability as an RL training environment — the quality of enabled policies matters more.

Overview

Sim2Sim2Sim implements a three-stage pipeline that progressively transfers learned behaviors across simulation environments:

Stage 1: Dynamics Model Distillation

Learn compact and efficient dynamics models from high-fidelity simulations.

• Data Collection: Gather diverse driving scenarios and vehicle dynamics data from BeamNG using a racing wheel
• Training & Evaluation: Train and evaluate a distilled dynamics model on collected data

Stage 2: RL Policy Learning

Train reinforcement learning policies in the distilled simulation environment.

• Setup & Training: Configure and run PPO training with different dynamics backends
• Evaluation & Visualization: Assess policy performance with detailed metrics, visualize rollouts as GIFs, and filter evaluation by specific scenario types (e.g., hard braking, intersections)

Stage 3: BeamNG Validation

Transfer learned policies to a high-fidelity driving simulator for real-world validation.

• Policy Evaluation: Evaluate trained policies on the Putnam Park circuit against human expert reference trajectories
  • Compute detailed metrics: cross-track error, heading error, speed tracking, action smoothness
  • Test robustness with configurable surface friction (asphalt, ice, snow, gravel, etc.)
  • Optional ice patches scattered around the track to challenge low-friction handling
  • Ice-aware trajectory preprocessing with adaptive braking/acceleration profiles

---

Results

Pretrained checkpoints for all dynamics models and control policies are available.

Checkpoint type	Description
dynamics_model/beamng_bicycle_*	Bicycle model baselines
dynamics_model/beamng_ddm_*	Deep Dynamics Model (DDM)
dynamics_model/beamng_trans_*	Transformer-based dynamics
dynamics_model/beamng_dytr_*	DYTR models
control_policies/PPO____*_bicycle	Policies trained on bicycle dynamics
control_policies/PPO____*_ddm	Policies trained on DDM
control_policies/PPO____*_trans	Policies trained on Transformer
control_policies/PPO____*_dytr_ddm	Policies trained on DYTR dynamics
control_policies/PPO____*_oracle	Oracle (privileged obs) policies

---

Citation

If you found this repository useful in your research, please consider citing our work:

@article{GuChittaEtAl2026,
  author    = {Gu, Xunjiang and Chitta, Kashyap and Golchoubian, Mahsa and Suplin, Vladimir and Gilitschenski, Igor},
  title     = {Dynamics Distillation for Efficient and Transferable Control Learning},
  journal   = {arXiv preprint arXiv:2605.01516},
  year      = {2026}
}

---

License

This repository is licensed under Apache 2.0. See the LICENSE file for details.