Questions about baseline reproducibility and experiment pipeline

[abattada]

Hi, thank you for releasing the TimeDART code. I am trying to reproduce the experiments reported in the paper, especially the comparisons described in Appendix B.1 and B.2, and I have a few questions about the current repository implementation.

According to the paper, the experiments compare TimeDART with both self-supervised and supervised baselines:

• Self-supervised baselines: SimMTM, PatchTST-SSL, TimeMAE, CoST
• Supervised baselines: supervised PatchTST, DLinear, FormerTime
• Random initialization baseline: the same representation encoder as TimeDART, but directly fine-tuned without pre-training

Appendix B.2 also states that self-supervised methods use a unified encoder as the representation network, are pre-trained for 50 epochs, and are then fine-tuned under consistent downstream settings.

While checking the released code, I found several points that I am not sure how to interpret.

Only TimeDART, TimeDART_v2, and SimMTM appear to be connected in run.py

In run.py, the experiment map currently contains:

Exp_map = {
    "TimeDART": Exp_TimeDART,
    "TimeDART_v2": Exp_TimeDART_v2,
    "SimMTM": Exp_SimMTM,
}

I could not find runnable experiment entries for:

PatchTST-SSL
TimeMAE
CoST
DLinear
FormerTime
Random Init

Could you clarify whether these baselines are expected to be reproduced from this repository, or whether their results were obtained using separate original repositories or scripts?

Whether the released repository is intended to fully reproduce the paper's baseline comparisons, or only the TimeDART and partial SimMTM experiments?

PatchTST.py exists, but does not seem to be connected to the current pipeline

There is a models/PatchTST.py file, but PatchTST is not included in Exp_map.

Also, the model appears to expect task names such as:

long_term_forecast
short_term_forecast
imputation
anomaly_detection
classification

whereas run.py uses:

pretrain
finetune

Additionally, run.py always expects a pre-trained checkpoint path during finetune.
This seems incompatible with the supervised PatchTST baseline, which should be trained directly without self-supervised pre-training.

Could you clarify how the supervised PatchTST baseline and PatchTST-SSL baseline in the paper were trained?

SimMTM fine-tuning seems not to load the pre-trained checkpoint

In exp_simmtm.py, the checkpoint loading logic appears to be commented out:

# if self.args.load_checkpoints:
#     print("Loading ckpt: {}".format(self.args.load_checkpoints))
#     model = transfer_weights(self.args.load_checkpoints, model, device=self.device)

At the same time, run.py still constructs args.load_checkpoints for fine-tuning.

Does this mean that SimMTM fine-tuning currently starts from random initialization instead of the pre-trained encoder? If not, could you point me to where the pre-trained SimMTM encoder is loaded?

Random Init. baseline does not seem to have a dedicated entry

The paper reports a Random Init. baseline, which I understand as using the same representation encoder as TimeDART but skipping pre-training and directly fine-tuning on the downstream task.

However, I could not find a clear command-line option or script for this setting. Since run.py --task_name finetune automatically sets args.load_checkpoints, it is not obvious how to run the Random Init. baseline cleanly.

Could you provide the intended command or script for reproducing Random Init.?

Thanks again for releasing the code. I would appreciate any clarification, since I am trying to make sure the reproduced results match the experimental protocol described in the paper.