Reproducibility and Setup Questions for TokenBench

[Wang-Ji-Zhe]

I’m trying to reproduce the TokenBench evaluation and have a few questions:

1. StyleGAN checkpoint download

   • The doc suggests:

     snapshot_download(
         repo_id="LanguageBind/Open-Sora-Plan-v1.0.0", 
         local_dir=local_dir
     )

   • In practice, I need:

     snapshot_download(
         repo_id="LanguageBind/Open-Sora-Plan-v1.0.0",
         repo_type="space",
         local_dir=local_dir
     )

   • In fact, the folder structure I get is pretrained_ckpts/LanguageBind/Open-Sora-Plan-v1.0.0/opensora/..., not the pretrained_ckpts/Open-Sora-Plan-v1.0.0/opensora/... stated in the docs, nor the path expected by token_bench/fvd.py:

     _STYLEGAN_TORCHSCRIPT_CKPT = (
         "./pretrained_ckpts/opensora/eval/fvd/styleganv/i3d_torchscript.pt"
     )

2. Hardcoded absolute path

   • In token_bench/video/preprocessing_script.py, the code uses:

     raw_video_dir = "/root/dataset"

   • This will break on systems without /root/dataset.

3. Missing data extraction script

   • The repo provides token_bench/video/list.txt (the selected videos), but I can’t find the script that extracts these from the full dataset. Could you please add or point to the extraction script?

---

Main question:
Could you confirm whether this repository contains all the necessary code to fully reproduce the TokenBench evaluation? If so, these issues are all minor.

Thank you for your work and guidance!