Hi!
Hi, thanks for the great work and for releasing the code!
I have a question about the SKEMPI v2 preprocessing. In prepare_data.ipynb, the final split_0/1/2 columns are produced by two consecutive shuffles:
-
A seeded shuffle inside SkempiDataset (random.Random(split_seed=2023).shuffle(complex_list)), which determines the order in which entries are appended to the dataframe.
-
An unseeded shuffle in the cell that assigns folds:
unique_complexes = df['complex'].unique()
np.random.shuffle(unique_complexes)
Because the second shuffle uses NumPy's global RNG without np.random.seed(...), every fresh kernel produces a different fold partition, even with all upstream code unchanged. As a result, the exact processed_data.csv used for the paper's SKEMPI numbers can't be regenerated from the repo alone.
Would it be possible to share the processed_data.csv you used for the reported results, or to add a seed before the np.random.shuffle call? Either would make the SKEMPI benchmark reproducible.
Thanks!
Hi!
Hi, thanks for the great work and for releasing the code!
I have a question about the SKEMPI v2 preprocessing. In prepare_data.ipynb, the final split_0/1/2 columns are produced by two consecutive shuffles:
A seeded shuffle inside SkempiDataset (random.Random(split_seed=2023).shuffle(complex_list)), which determines the order in which entries are appended to the dataframe.
An unseeded shuffle in the cell that assigns folds:
Because the second shuffle uses NumPy's global RNG without np.random.seed(...), every fresh kernel produces a different fold partition, even with all upstream code unchanged. As a result, the exact processed_data.csv used for the paper's SKEMPI numbers can't be regenerated from the repo alone.
Would it be possible to share the processed_data.csv you used for the reported results, or to add a seed before the np.random.shuffle call? Either would make the SKEMPI benchmark reproducible.
Thanks!