Hi, thank you for releasing SADA.
I am trying to reproduce the EPIC-Kitchens S6 result in Table 1 (new → old):
- tIoU 0.1–0.5:
23.94 / 22.95 / 21.47 / 19.16 / 15.24
- Average mAP:
20.55
I used repository commit:
a96a15b0d0c905a553c7a2b14f607c9485ebe758
However, I could not find an author-released S6 (new_vs_old) configuration or checkpoint. The released checkpoint package appears to contain S1, S2, S4, S5, and CharadesEgo, but not S3 or S6. The repository contains configs/old_vs_new/version_final_ckpt.yaml, but no corresponding new_vs_old configuration.
I also found that the configuration embedded in the official S5 checkpoint:
best_ckpt_old_vs_new/saved_ckpt/best_ckpt/checkpoint-epoch10.pt
differs substantially from the released:
configs/old_vs_new/version_final_ckpt.yaml
| Parameter |
S5 checkpoint config |
Released YAML |
dim_proj_adv |
128 |
128 |
pl_threshold |
0.55 |
0.3 |
conditional_adv_drop_rate |
0.5 |
0.1 |
loss_weights.weight_adv |
3 |
2 |
loss_weights.weight_downstream |
3 |
2 |
disc_learning_rate |
0.02 |
0.004 |
beta |
1 |
0.5 |
weight_source_cond_adv |
0.6 |
0.75 |
weight_target_cond_adv |
0.6 |
0.75 |
| Level weights |
[0.3, 0.3, 0.2, 0.4, 0.3, 0.9] |
[0.6, 0.1, 1.0, 0.6, 0.2, 0.4] |
The tensor shapes in the official S5 checkpoint also confirm that its discriminator width is 128:
- Standard discriminator:
(128, 512) → (128, 128) → (1, 128)
- Conditional discriminator:
(128, 1024) → (128, 128) → (1, 128)
For S6, I reconstructed the direction {EK100 \ EK55} → EK55 and used the supplementary Table A parameters:
- Level weights:
[0.6, 0.1, 1.0, 0.6, 0.2, 0.4]
alpha / pl_threshold = 0.3
weight_adv = 2.0
- Seed 6
- Checkpoint selection based only on source-validation downstream loss
Results so far:
- Width 128: target-training Avg mAP was 18.30 at epoch 5 and 17.91 at the source-validation-selected epoch 10 checkpoint. Later epochs declined.
- Width 512 controlled run: target-training Avg mAP was 18.19 at epoch 5.
I also observe differences between the released annotation package and the S6 split counts reported in the paper appendix:
- Paper source: train 150 / validation 44
- Released package source: train 157 / validation 37
- Paper target: train 342 / validation 90
- Released package target: train 345 / validation 87
Could you please clarify or share:
- The exact original S6
new_vs_old YAML/configuration used for the paper.
- The official S6 checkpoint, if available.
- The annotation/split version used to obtain the S6 train/validation counts reported in the paper.
- Whether
configs/old_vs_new/version_final_ckpt.yaml is intended to reproduce S5, given that it differs from the configuration embedded in the official S5 checkpoint.
- The exact
dim_proj_adv used for S6. The paper architecture description mentions width 512, while the official S5 checkpoint uses 128.
- Whether additional scenario-specific parameters not listed in Table A are required to reproduce S6.
Any guidance or released artifacts would be greatly appreciated. Thank you!
Hi, thank you for releasing SADA.
I am trying to reproduce the EPIC-Kitchens S6 result in Table 1 (
new → old):23.94 / 22.95 / 21.47 / 19.16 / 15.2420.55I used repository commit:
a96a15b0d0c905a553c7a2b14f607c9485ebe758However, I could not find an author-released S6 (
new_vs_old) configuration or checkpoint. The released checkpoint package appears to contain S1, S2, S4, S5, and CharadesEgo, but not S3 or S6. The repository containsconfigs/old_vs_new/version_final_ckpt.yaml, but no correspondingnew_vs_oldconfiguration.I also found that the configuration embedded in the official S5 checkpoint:
best_ckpt_old_vs_new/saved_ckpt/best_ckpt/checkpoint-epoch10.ptdiffers substantially from the released:
configs/old_vs_new/version_final_ckpt.yamldim_proj_advpl_thresholdconditional_adv_drop_rateloss_weights.weight_advloss_weights.weight_downstreamdisc_learning_ratebetaweight_source_cond_advweight_target_cond_adv[0.3, 0.3, 0.2, 0.4, 0.3, 0.9][0.6, 0.1, 1.0, 0.6, 0.2, 0.4]The tensor shapes in the official S5 checkpoint also confirm that its discriminator width is 128:
(128, 512) → (128, 128) → (1, 128)(128, 1024) → (128, 128) → (1, 128)For S6, I reconstructed the direction
{EK100 \ EK55} → EK55and used the supplementary Table A parameters:[0.6, 0.1, 1.0, 0.6, 0.2, 0.4]alpha / pl_threshold = 0.3weight_adv = 2.0Results so far:
I also observe differences between the released annotation package and the S6 split counts reported in the paper appendix:
Could you please clarify or share:
new_vs_oldYAML/configuration used for the paper.configs/old_vs_new/version_final_ckpt.yamlis intended to reproduce S5, given that it differs from the configuration embedded in the official S5 checkpoint.dim_proj_advused for S6. The paper architecture description mentions width 512, while the official S5 checkpoint uses 128.Any guidance or released artifacts would be greatly appreciated. Thank you!