This is the repository for Self Trained Decision Transformer (STDT), a course project at University of Waterloo. Refer to link for packages installation and run an example here. The report could be found in the link.
We acknowledge the work of Decision Transformer upon which much of the codes in this repository are built.
Experiments have been conducted on the Racetrack environment.
Please install Stable-Baselines3, HighwayEnv, and Minari according to their instructions.
You can execute the following codes to collect Offine dataset and self-train a DT.
We use Proximal Policy Optimization (PPO) as provided in Stable-Baselines3 to train our Offline agent. You could train the agent using the following code:
python racetrack/train_ppo.py
We could then use the PPO policy to generate our Offline dataset (via Minari) by executing the code below:
python racetrack/collect_data.py
The dataset we collected is named as racetrack/test-v2 and is stored locally. To load the dataset, run the code:
import minari
dataset = minari.load_dataset("racetrack/test-v2")
We could use the following code to test STDT:
python experiment.py --num_eval_episodes 10 --max_iters 4 --num_steps_per_iter 500 --self_train True --num_gen_episodes 15 --num_episode_updates 5 --log_to_wandb True
Note that you should get your wandb account ready to see the results.
You can also train and evaluate DT only by setting --self_train False --num_episode_updates 1.