Releases: BitMind-AI/gasbench
Release 0.6.6
| Released dataset name | Obfuscated holdout name | Modality | Media type |
|---|---|---|---|
v13-e4s |
synthetic-video-holdout-90cc07d9 |
video |
synthetic |
v13-echonet-synthetic-v1 |
synthetic-video-holdout-589f8477 |
video |
synthetic |
v13-lemonade |
real-video-holdout-0ea2b302 |
video |
real |
v13-live-whisperx-526k |
real-video-holdout-40577fd5 |
video |
real |
v14-real-hallo3-training-data |
real-video-holdout-e99a6cd3 |
video |
real |
v14-real-moments-in-time-raw |
real-video-holdout-c627820a |
video |
real |
Released v15 video holdouts — Human
73 datasets.
| Released dataset name | Obfuscated holdout name | Modality | Media type |
|---|---|---|---|
DH-FaceVid-1K-0003-part_3 |
real-video-holdout-bb599c24 |
video |
real |
DH-FaceVid-1K-0003-part_4 |
real-video-holdout-0c5992ea |
video |
real |
DH-FaceVid-1K-0003-part_5 |
real-video-holdout-a778c46c |
video |
real |
MAVOS-DD-english_sonic |
synthetic-video-holdout-30ca29f6 |
video |
synthetic |
MAVOS-DD-freevc |
synthetic-video-holdout-476bb844 |
video |
synthetic |
MAVOS-DD-german_echomimic |
synthetic-video-holdout-4cda34f8 |
video |
synthetic |
MAVOS-DD-german_freevc |
synthetic-video-holdout-0ddd910c |
video |
synthetic |
MAVOS-DD-german_hififace |
synthetic-video-holdout-2d68a443 |
video |
synthetic |
MAVOS-DD-german_inswapper |
synthetic-video-holdout-ad7af038 |
video |
synthetic |
MAVOS-DD-german_knnvc |
synthetic-video-holdout-319f0b94 |
video |
synthetic |
MAVOS-DD-german_liveportrait |
synthetic-video-holdout-cd2b3dba |
video |
synthetic |
MAVOS-DD-german_memo |
synthetic-video-holdout-5f57b6ea |
video |
synthetic |
MAVOS-DD-german_roop |
synthetic-video-holdout-0f4d071b |
video |
synthetic |
MAVOS-DD-german_sonic |
synthetic-video-holdout-567ebf9e |
video |
synthetic |
MAVOS-DD-hindi_echomimic |
synthetic-video-holdout-ca8ba835 |
video |
synthetic |
MAVOS-DD-hindi_hififace |
synthetic-video-holdout-b83ad09b |
video |
synthetic |
MAVOS-DD-hindi_inswapper |
synthetic-video-holdout-8ae3fcab |
video |
synthetic |
MAVOS-DD-hindi_knnvc |
synthetic-video-holdout-5f319c69 |
video |
synthetic |
MAVOS-DD-hindi_liveportrait |
synthetic-video-holdout-28922fda |
video |
synthetic |
MAVOS-DD-hindi_memo |
synthetic-video-holdout-d36cb85b |
video |
synthetic |
MAVOS-DD-hindi_roop |
synthetic-video-holdout-1359e798 |
video |
synthetic |
MAVOS-DD-hindi_sonic |
synthetic-video-holdout-34b113dc |
video |
synthetic |
MAVOS-DD-knnvc |
synthetic-video-holdout-71c4ccd8 |
video |
synthetic |
MAVOS-DD-mandarin_echomimic |
synthetic-video-holdout-60de4b3a |
video |
synthetic |
MAVOS-DD-mandarin_freevc |
synthetic-video-holdout-421732bd |
video |
synthetic |
MAVOS-DD-mandarin_hififace |
synthetic-video-holdout-ef35d61b |
video |
synthetic |
MAVOS-DD-mandarin_inswapper |
synthetic-video-holdout-f4873f15 |
video |
synthetic |
MAVOS-DD-mandarin_knnvc |
synthetic-video-holdout-488f6e35 |
video |
synthetic |
MAVOS-DD-mandarin_liveportrait |
synthetic-video-holdout-1f619331 |
video |
synthetic |
MAVOS-DD-mandarin_memo |
synthetic-video-holdout-1f087f85 |
video |
synthetic |
MAVOS-DD-mandarin_roop |
synthetic-video-holdout-e44fa7cd |
video |
synthetic |
MAVOS-DD-mandarin_sonic |
synthetic-video-holdout-b4f9beec |
video |
synthetic |
MAVOS-DD-romanian_echomimic |
synthetic-video-holdout-8ea00713 |
video |
synthetic |
MAVOS-DD-romanian_freevc |
synthetic-video-holdout-91ec85a3 |
video |
synthetic |
MAVOS-DD-romanian_hififace |
synthetic-video-holdout-343d88d7 |
video |
synthetic |
MAVOS-DD-romanian_inswapper |
synthetic-video-holdout-1bc66d5d |
video |
synthetic |
MAVOS-DD-romanian_knnvc |
synthetic-video-holdout-79b89c16 |
video |
synthetic |
MAVOS-DD-romanian_liveportrait |
synthetic-video-holdout-ffff1093 |
video |
synthetic |
MAVOS-DD-romanian_memo |
synthetic-video-holdout-23ccac96 |
video |
synthetic |
MAVOS-DD-romanian_roop |
synthetic-video-holdout-b2ab16d9 |
video |
synthetic |
MAVOS-DD-romanian_sonic |
synthetic-video-holdout-97a8e023 |
video |
synthetic |
MAVOS-DD-russian_echomimic |
synthetic-video-holdout-41a03d06 |
video |
synthetic |
MAVOS-DD-russian_freevc |
synthetic-video-holdout-1a2148f3 |
video |
synthetic |
MAVOS-DD-russian_hififace |
synthetic-video-holdout-213d65fa |
video |
synthetic |
v15-human-vid-celebv-hq |
real-video-holdout-f10a679d |
video |
real |
v15-human-vid-dfdm_cfr23-dfaker |
synthetic-video-holdout-032b8470 |
video |
synthetic |
v15-human-vid-dfdm_cfr23-dfl-h128 |
synthetic-video-holdout-c92645dc |
video |
synthetic |
v15-human-vid-dfdm_cfr23-iae |
synthetic-video-holdout-9eee885c |
video |
synthetic |
v15-human-vid-dfdm_cfr23-lightweight |
synthetic-video-holdout-d127ad71 |
video |
synthetic |
v15-human-vid-dfdm_cfr23-real |
real-video-holdout-bd7b715b |
video |
real |
v15-human-vid-dh-facevid-1k-0002-part_1 |
real-video-holdout-a39f716a |
video |
real |
v15-human-vid-dh-facevid-1k-0002-part_2 |
real-video-holdout-055386d3 |
video |
real |
v15-human-vid-dh-facevid-1k-0002-part_3 |
real-video-holdout-3ac018ea |
video |
real |
v15-human-vid-dh-facevid-1k-0002-part_4 |
real-video-holdout-62772f42 |
video |
real |
v15-human-vid-dh-facevid-1k-0002-part_5 |
real-video-holdout-f4dab550 |
video |
real |
v15-human-vid-dh-facevid-1k-0003-part_1 |
real-video-holdout-8b493ecc |
video |
real |
v15-human-vid-dh-facevid-1k-0003-part_2 |
real-video-holdout-1cf5d2f9 |
video |
real |
v15-human-vid-digifakeav_echomimic_21501_22000 |
synthetic-video-holdout-00a91199 |
video |
synthetic |
v15-human-vid-digifakeavfvfa_with_audio |
synthetic-video-holdout-18c2b9ce |
video |
synthetic |
v15-human-vid-mavos-dd-arabic-echomimic |
synthetic-video-holdout-ec3d3cd0 |
video |
synthetic |
v15-human-vid-mavos-dd-arabic-hififace |
synthetic-video-holdout-0920c4a7 |
video |
synthetic |
v15-human-vid-mavos-dd-arabic-inswapper |
synthetic-video-holdout-af05e52b |
video |
synthetic |
v15-human-vid-mavos-dd-arabic-liveportrait |
synthetic-video-holdout-f61b042c |
video |
synthetic |
v15-human-vid-mavos-dd-arabic-roop |
synthetic-video-holdout-982fab07 |
video |
synthetic |
v15-human-vid-mavos-dd-arabic-sonic |
synthetic-video-holdout-2ebd76d7 |
video |
synthetic |
v15-human-vid-mavos-dd-english_echomimic |
synthetic-video-holdout-118ac16f |
video |
synthetic |
v15-human-vid-mavos-dd-english_freevc |
synthetic-video-holdout-4b760ab9 |
video |
synthetic |
v15-human-vid-mavos-dd-english_hififace |
synthetic-video-holdout-51cd64cb |
video |
synthetic |
v15-human-vid-mavos-dd-english_inswapper |
synthetic-video-holdout-9451219f |
video |
synthetic |
v15-human-vid-mavos-dd-english_knnvc |
synthetic-video-holdout-909d326b |
video |
synthetic |
v15-human-vid-mavos-dd-english_liveportrait |
synthetic-video-holdout-372db1bb |
video |
synthetic |
v15-human-vid-mavos-dd-english_memo |
synthetic-video-holdout-19d06111 |
video |
synthetic |
v15-human-vid-mavos-dd-english_roop |
synthetic-video-holdout-9f9485b2 |
video |
synthetic |
Release 0.6.5
- Removed
shooter-fakeimage dataset, previously incorrectly marked synthetic - Fixing label for
cosyvoice-instruct
Release 0.6.4
| Released dataset name | Obfuscated holdout name | Modality | Media type |
|---|---|---|---|
v14-real-vcapcv-vggsound-test-15446-audio-cut |
real-audio-holdout-ba647a50 |
audio |
real |
v14-real-vggsound-test-15446-video-cut |
real-audio-holdout-4e846c0f |
audio |
real |
v14-real-kallaama |
real-audio-holdout-574a0401 |
audio |
real |
v14-real-chichewa-dataset |
real-audio-holdout-dcb3cc7b |
audio |
real |
v14-real-vivos |
real-audio-holdout-74d82871 |
audio |
real |
v14-real-nisqa-corpus-dataset |
real-audio-holdout-2c36fa90 |
audio |
real |
v14-real-natural-odss |
real-audio-holdout-b8338bca |
audio |
real |
v14-real-fastpitch-hifigan |
real-audio-holdout-930542a5 |
audio |
real |
v14-real-daps |
real-audio-holdout-b20f4647 |
audio |
real |
v14-real-bci-datasets |
real-audio-holdout-aff9e8fe |
audio |
real |
v14-real-ravdess-speech-16k |
real-audio-holdout-ff483ea4 |
audio |
real |
v14-fake-somos |
synthetic-audio-holdout-222cd5cc |
audio |
synthetic |
v14-fake-diffgan-tts-aux |
synthetic-audio-holdout-19638fa3 |
audio |
synthetic |
v14-fake-grad-tts |
synthetic-audio-holdout-847ccfa5 |
audio |
synthetic |
v14-fake-tacotron2-dca-diffwave |
synthetic-audio-holdout-7d903c0d |
audio |
synthetic |
v14-fake-wavegrad2 |
synthetic-audio-holdout-4c09a9a9 |
audio |
synthetic |
v14-fake-diffgan-tts-naive |
synthetic-audio-holdout-435ce3e0 |
audio |
synthetic |
v14-fake-natspeech-diffspeech |
synthetic-audio-holdout-e6a66c72 |
audio |
synthetic |
v14-fake-fast-pitch |
synthetic-audio-holdout-b9a6a0d3 |
audio |
synthetic |
v14-fake-tacotron2-dca |
synthetic-audio-holdout-527cb6c8 |
audio |
synthetic |
v14-fake-tacotron2-dca-wavegrad |
synthetic-audio-holdout-3d18d856 |
audio |
synthetic |
v14-fake-diffgan-tts-shallow |
synthetic-audio-holdout-65bf3c65 |
audio |
synthetic |
v14-fake-prodiff |
synthetic-audio-holdout-97af50ad |
audio |
synthetic |
v14-fake-glow-tts |
synthetic-audio-holdout-3051cd6a |
audio |
synthetic |
v14-fake-tacotron2-dca-bddm |
synthetic-audio-holdout-6c72fa6a |
audio |
synthetic |
v14-fake-vits-1 |
synthetic-audio-holdout-0a918a26 |
audio |
synthetic |
v14-fake-vits |
synthetic-audio-holdout-11335f29 |
audio |
synthetic |
v14-fake-vcapv-t2a |
synthetic-audio-holdout-b2340f5f |
audio |
synthetic |
v14-MLAAD-Fake-part_01 |
fake-audio-holdout-4aeede7d |
audio |
fake |
v14-MLAAD-Fake-part_02 |
fake-audio-holdout-182aa55f |
audio |
fake |
v14-MLAAD-Fake-part_03 |
fake-audio-holdout-823cfdef |
audio |
fake |
v14-MLAAD-Fake-part_04 |
fake-audio-holdout-0a2f31b0 |
audio |
fake |
v14-MLAAD-Fake-part_05 |
fake-audio-holdout-065f3770 |
audio |
fake |
v14-MLAAD-Fake-part_06 |
fake-audio-holdout-df10edeb |
audio |
fake |
v14-MLAAD-Fake-part_07 |
fake-audio-holdout-9b51a670 |
audio |
fake |
v14-MLAAD-Fake-part_08 |
fake-audio-holdout-0c25b759 |
audio |
fake |
v14-MLAAD-Fake-part_09 |
fake-audio-holdout-7ce01af9 |
audio |
fake |
v14-MLAAD-Fake-part_10 |
fake-audio-holdout-418c4a90 |
audio |
fake |
v14-dag-asr-audio |
real-audio-holdout-c9491bb5 |
audio |
real |
v14-WaxalNLP-TTS-part_01 |
real-audio-holdout-de4859f5 |
audio |
real |
v14-WaxalNLP-TTS-part_02 |
real-audio-holdout-c1c223f4 |
audio |
real |
v14-WaxalNLP-TTS-part_03 |
real-audio-holdout-e53dfd16 |
audio |
real |
v14-WaxalNLP-TTS-part_04 |
real-audio-holdout-7f031cf1 |
audio |
real |
v14-WaxalNLP-TTS-part_05 |
real-audio-holdout-fcdd0715 |
audio |
real |
v14-WaxalNLP-TTS-part_06 |
real-audio-holdout-a6702abe |
audio |
real |
v14-WaxalNLP-TTS-part_07 |
real-audio-holdout-c462f462 |
audio |
real |
v14-WaxalNLP-TTS-part_08 |
real-audio-holdout-a80cd5b3 |
audio |
real |
v14-WaxalNLP-TTS-part_09 |
real-audio-holdout-a0791dd4 |
audio |
real |
v14-WaxalNLP-TTS-part_10 |
real-audio-holdout-14d066a5 |
audio |
real |
v14-real-mmhu-h-videos |
real-video-holdout-d4f56405 |
video |
real |
v14-real-mmhu-t-videos |
real-video-holdout-14cc82ed |
video |
real |
v14-real-mmhu-v-videos |
real-video-holdout-b7174cb7 |
video |
real |
v14-real-vivid |
real-video-holdout-8534b595 |
video |
real |
v14-real-soccernet-10s-5class |
real-video-holdout-74616031 |
video |
real |
v14-real-ofdvdnet |
real-video-holdout-b1b60d9c |
video |
real |
v14-real-or-video-mov |
real-video-holdout-aacc44e4 |
video |
real |
v14-real-poultry-videos |
real-video-holdout-acaccb84 |
video |
real |
v14-real-spatialvid-group-001 |
real-video-holdout-a5173e55 |
video |
real |
v14-real-spatialvid-group-002 |
real-video-holdout-b4180215 |
video |
real |
v14-real-spatialvid-group-003 |
real-video-holdout-908daf61 |
video |
real |
v14-real-spatialvid-group-004 |
real-video-holdout-52ee57f0 |
video |
real |
v14-real-spatialvid-group-005 |
real-video-holdout-58bed19f |
video |
real |
v14-real-videoespresso-train-video-01 |
real-video-holdout-2546c150 |
video |
real |
v14-real-videoespresso-train-video-02 |
real-video-holdout-e444b81e |
video |
real |
v14-real-open-o3-video |
real-video-holdout-73a9d20e |
video |
real |
v14-real-panflow-1 |
real-video-holdout-51fd7e37 |
video |
real |
v14-real-panflow-2 |
real-video-holdout-2ed3f604 |
video |
real |
v14-real-panflow-3 |
real-video-holdout-4d085ece |
video |
real |
v14-real-panflow-4 |
real-video-holdout-e4fe8a2c |
video |
real |
v14-real-dh-facevid-1k-0001 |
real-video-holdout-6fc0b313 |
video |
real |
v14-real-tracking-any-granularity-videos |
real-video-holdout-ccf53e73 |
video |
real |
v14-real-wild-animal-recognition-video-dataset |
real-video-holdout-d2b5f026 |
video |
real |
v14-real-wlasl-videos |
real-video-holdout-2e1dc2db |
video |
real |
v14-real-wlasl-videos-1 |
real-video-holdout-1c782169 |
video |
real |
v14-real-wlasl-raw-videos-mp4 |
real-video-holdout-1122cfce |
video |
real |
v14-real-youtubeclips |
real-video-holdout-b055743a |
video |
real |
v14-fake-allegro |
synthetic-video-holdout-089c6870 |
video |
synthetic |
v14-fake-animatediffturbo |
synthetic-video-holdout-d7c4ecc2 |
video |
synthetic |
v14-fake-ltxvideo |
synthetic-video-holdout-3785e46b |
video |
synthetic |
v14-fake-mochi1 |
synthetic-video-holdout-c4ccd94d |
video |
synthetic |
v14-fake-pyramidflow |
synthetic-video-holdout-38ebcccf |
video |
synthetic |
v14-fake-videocrafter2 |
synthetic-video-holdout-0b71f5df |
video |
synthetic |
v14-fake-animatediff |
synthetic-video-holdout-b4755652 |
video |
synthetic |
v14-fake-cogvideox |
synthetic-video-holdout-22f24e88 |
video |
synthetic |
v14-fake-fastsvd |
synthetic-video-holdout-58631be3 |
video |
synthetic |
v14-fake-lavie |
synthetic-video-holdout-e14b7524 |
video |
synthetic |
v14-fake-modelscope |
synthetic-video-holdout-4df93a32 |
video |
synthetic |
v14-fake-opensora12 |
synthetic-video-holdout-8ab9a23d |
video |
synthetic |
v14-fake-opensora |
synthetic-video-holdout-59a9b3c9 |
video |
synthetic |
v14-fake-t2vturbo |
synthetic-video-holdout-dd2a8901 |
video |
synthetic |
v14-fake-vcapav-t2v |
synthetic-video-holdout-e31c3965 |
video |
synthetic |
v14-fake-cameraclone-0316 |
synthetic-video-holdout-82bcb0bf |
video |
synthetic |
v14-fake-cameraclone-0317 |
synthetic-video-holdout-5a0cbed0 |
video |
synthetic |
v14-fake-cameraclone-0401 |
synthetic-video-holdout-d4b3ea97 |
video |
synthetic |
v14-fake-cameraclone-0402 |
synthetic-video-holdout-e194c1a8 |
video |
synthetic |
v14-fake-cameraclone-0404 |
synthetic-video-holdout-dc7cf915 |
video |
synthetic |
v14-fake-cameraclone-0407 |
synthetic-video-holdout-8e30a656 |
video |
synthetic |
v14-fake-cameraclone-0410 |
synthetic-video-holdout-decdec92 |
video |
synthetic |
v14-real-chinese-mp4-in-audio |
real-video-holdout-6e020e98 |
video |
real |
Synthetic-Images-Fire-Scenario |
synthetic-image-holdout-8fae48b4 |
image |
synthetic |
synthetic-dataset |
synthetic-image-holdout-e4b4fa03 |
image |
synthetic |
Midjourneyv5-5K |
synthetic-image-holdout-8dc0c9de |
image |
synthetic |
fake_sdxl_12k-part-1 |
synthetic-image-holdout-edd5dd1b |
image |
synthetic |
fake_sdxl_12k-part-2 |
synthetic-image-holdout-0f3be6af |
image |
synthetic |
fake_sdxl_12k-part-3 |
synthetic-image-holdout-f291d1b7 |
image |
synthetic |
fake_sdxl_12k-part-4 |
synthetic-image-holdout-64663d9f |
image |
synthetic |
Synthetic-Dog-Images |
synthetic-image-holdout-c11a8170 |
image |
synthetic |
synthetic_data_0.1 |
synthetic-image-holdout-67a8e625 |
image |
synthetic |
syntheticdata_0.15 |
synthetic-image-holdout-af60fa23 |
image |
synthetic |
ptd-synthetic |
synthetic-image-holdout-061fbf94 |
image |
synthetic |
image_patches_raw |
synthetic-image-holdout-6405b7db |
image |
synthetic |
stable-imagenet1k-flat |
synthetic-image-holdout-3b5b6e0b |
image |
synthetic |
Shooter-fake |
synthetic-image-holdout-29eb8247 |
image |
synthetic |
SDv15R-dpmsolver-25-15K-part0 |
synthetic-image-holdout-1d5da5bc |
image |
synthetic |
SDv15R-dpmsolver-25-15K-part1 |
`synthetic-image-hol... |
Release 0.6.3
Deprecating old cache policy logic, previously used to determine what samples to keep in the gasstation cache when fool rates were more dynamic.
Release 0.6.2
Parallelize gasbench data loading & fix memory leaks
Problem
Image and video benchmarks run unacceptably slowly when data was coming from NAS (not noticeable on local setups), and also occasionally OOM deep into runs. T
Root causes identified:
-
Sequential disk I/O from network volumes — The
DatasetIteratorreads each image/video file one-by-one in the producer thread. Each read from a NAS incurs network latency, paid serially N times. -
"Drain-all-futures" stall —
PrefetchPipelineaccumulatednum_workers * 2futures then blocked on ALL of them (for future in futures: future.result()). The pipeline stalls on the slowest task even when other workers are idle. -
Only 3 worker threads — With I/O-bound work (network volume reads + PIL decode), 3 threads underutilize available concurrency.
-
Memory leak from large images — Datasets with very large source images (100+ megapixels observed in logs) cause multi-GB memory spikes because image bytes are held in multiple places simultaneously: the sample dict, the result dict, and the batch queue. No explicit cleanup of PIL Image objects in multi-threaded workers.
Changes
gasbench/src/gasbench/dataset/iterator.py
- Added
lazy_read: boolparameter toDatasetIterator - When
True, image samples yield{"image_path": ...}instead of reading file bytes; video samples yield{"video_path": ...}for file-based videos (frame directories are already lazy) - Iterating the dataset becomes near-instant (path collection only, no I/O)
gasbench/src/gasbench/benchmarks/image_bench.py
- Rewrote
PrefetchPipelinewith three fixes:- Parallel I/O: New
_read_and_preprocess()does file read + PIL decode + augmentation as a single unit inside worker threads — 8 threads read from the network volume concurrently - Bounded sliding window: Uses
wait(FIRST_COMPLETED)withmax_in_flight = num_workers * 4 = 32instead of submit-all. Prevents unbounded memory growth from completed-but-unconsumed futures - Sample metadata stripping: Drops heavy keys (
image,image_bytes,image_path) from result dicts immediately after preprocessing — tracker only needs metadata fields
- Parallel I/O: New
- Default
num_workersincreased from 3 → 8 DatasetIteratorcreated withlazy_read=Trueexecutor.shutdown()now usescancel_futures=Truefor clean teardown
gasbench/src/gasbench/benchmarks/video_bench.py
- Same rewrite applied to
VideoPrefetchPipeline - Default
num_workersincreased from 3 → 4 (fewer than image due to heavier per-sample memory) max_in_flight = num_workers * 3 = 12(tighter bound for video frames)- Strips
video_bytesandvideo_pathfrom result dicts
gasbench/src/gasbench/processing/media.py
- Added explicit
image.close()inprocess_image_sample()after extracting the numpy array — prevents PIL Image objects from lingering in multi-threaded workers
Expected impact
| Metric | Before | After |
|---|---|---|
| Image I/O concurrency | 1 (serial) | 8 threads |
| Video I/O concurrency | 1 (serial) | 4 threads |
| Pipeline stall pattern | Drain all 6, block on slowest | FIRST_COMPLETED, no stalls |
| Peak in-flight samples (image) | 6 | 32 (bounded) |
| Peak in-flight samples (video) | 6 | 12 (bounded) |
| Image bytes in result dict | Held until tracker consumes | Stripped immediately |
| PIL Image cleanup | GC-dependent | Explicit .close() |
| Est. image benchmark time | ~5 hours (52 datasets) | ~1-2 hours |
Release 0.6.1
Replaces libtorchcodec in audio benchmark with ffmpeg decoding to avoid issues with missing/incorrect FFmpeg libavutil SONAMEs
Release 0.6.0
Release 0.6.0
Released v13 holdouts
These datasets were previously reserved as private evaluation holdouts and are now publicly available in the gasbench benchmark pool.
Audio
- asvspoof2021-df-eval — synthetic (ASVspoof 2021 DeepFake track evaluation set)
- asvspoof2021-la-eval — synthetic (ASVspoof 2021 Logical Access track evaluation set)
- asvspoof2021-pa-eval — synthetic (ASVspoof 2021 Physical Access track evaluation set)
- audiospoofing-mini-fake — synthetic (AudioSpoofing mini — synthetic speech subset)
- audiospoofing-mini-real — real (AudioSpoofing mini — genuine speech subset)
- cfad-fake — synthetic (CFAD Chinese Fake Audio Detection — fake audio)
- cfad-real — real (CFAD Chinese Fake Audio Detection — genuine audio)
- cvoice-small-fake — synthetic (Common Voice small subset — TTS / voice-converted fake speech)
- cvoice-small-real — real (Common Voice small subset — genuine human speech)
- fakesound2-fake — synthetic (FakeSound 2 — AI-generated environmental audio)
- fakesound2-real — real (FakeSound 2 — genuine environmental audio)
- scdf-en — synthetic (SCDF English subset — deepfake speech detection corpus)
- vcc2020-fake — synthetic (Voice Conversion Challenge 2020 — converted speech)
- vcc2020-real — real (Voice Conversion Challenge 2020 — source speech)
- audioset-sl — real (AudioSet Sound Level subset — environmental audio, FLAC)
Image
- digi2real — synthetic (Digi2Real synthetic-to-real image translation)
- fake-animals-images — synthetic (AI-generated fake animal images)
- animalweb — real (AnimalWeb animal face detection and alignment)
- labelme-12-50k — real (LabelMe 12-class 50k images)
- uagd — real (UAGD urban AI-generated detection — real images)
- imdb-crop — real (IMDB-WIKI face crops)
Video
- fmfcc-v-fake — semisynthetic (FMFCC-V — manipulated videos)
- fmfcc-v-real — real (FMFCC-V — original videos)
- polyglotfake-fake — semisynthetic (PolyGlotFake — synthetic videos)
- polyglotfake-real — real (PolyGlotFake — real source videos)
- blendface — semisynthetic (BlendFace face-swap deepfakes)
- dagan — semisynthetic (DAGAN GAN-generated face animation deepfakes)
- wav2lip — semisynthetic (Wav2Lip lip-sync deepfakes)
- rareact — real (RareAct — rare compound actions, YouTube)
- vidor — real (ViDor video relation detection)
Image human vertical (image_human_datasets.yaml)
New dataset config for human faces and related sources. Run with
gasbench run --image-model model.zip --dataset-config image_human_datasets.yaml.
Image (human)
- celeb-a-hq — real (CelebA-HQ celebrity faces)
- ffhq-256 — real (Flickr-Faces-HQ 256px)
- lfw — real (Labeled Faces in the Wild)
- imdb-crop — real (IMDB-WIKI face crops; same source as general image pool)
- fairface — real (FairFace balanced demographics, WACV 2021)
- idoc-mugshots — real (IDOC mugshot faces)
- fakeclue-real-ffpp — real (FaceForensics++ real faces)
bitmind/FakeClue(data/real/ff++/)
- SFHQ-part1 — synthetic (Synthetic Faces HQ — part 1)
bitmind/SyntheticFacesHQ(SFHQ-part1.zip)
- SFHQ-part2 — synthetic (Synthetic Faces HQ — part 2)
bitmind/SyntheticFacesHQ(SFHQ-part2.zip)
- SFHQ-part3 — synthetic (Synthetic Faces HQ — part 3)
bitmind/SyntheticFacesHQ(SFHQ-part3.zip)
- SFHQ-part4 — synthetic (Synthetic Faces HQ — part 4)
bitmind/SyntheticFacesHQ(SFHQ-part4.zip)
- face-swap — semisynthetic (face-swap deepfakes)
- fakeclue-fake-ffpp — synthetic (FaceForensics++ deepfakes)
bitmind/FakeClue(data/fake/ff++/)
Release 0.5.14
Include open-clip-torch dependency
Release 0.5.13
Fixed configurations for oxford-flower-102, xd-violence, and vpdata
Release 0.5.12
Add row-based filtering using filter_column and filter_value fields in dataset configs.