Evaluation differences with the paper's report

[lvtuan98]

Hi team,

First of all, thank you for your contribution — your approach to the molecule generation problem is both interesting and innovative.

I’ve been reviewing the public code and pre-trained weights. After running the sampling process to generate 100 samples and evaluating them using the Vina-family metrics, I noticed that my results differ significantly from those reported in the paper.

Could you please help me verify this or let me know if I might have made a mistake in my process?
Thanks!

• This is my config for sampling:

model:
  checkpoint: ./pretrained_models/pretrained-IRDiff.pt

sample:
  seed: 2021
  num_samples: 100
  num_steps: 1000
  pos_only: False
  center_pos_mode: protein
  sample_num_atoms: prior

• Sampling script:

python inference.py \
    --config ./configs/sampling.yml \
    --train_config ./configs/training.yml \
    --device 'cuda:0' \
    --batch_size 25 \
    --result_path ./experiments/sampled_results \
    --test_prompt_indices_path ./src/test_prompt_ligand_indices_top3.pt \
    --start_index 0 \
    --end_index 99

• Evaluating script:

python eval_split.py \
    --sample_path ./experiments/sampled_results \
    --verbose False \
    --eval_step -1 \
    --eval_start_index 0 \
    --eval_end_index 26 \
    --save True \
    --protein_root ./data/crossdocked_v1.1_rmsd1.0 \
    --atom_enc_mode add_aromatic \
    --docking_mode vina_dock \
    --exhaustiveness 16

#option: vina_dock|qvina
    
python cal_metrics_from_pt.py \
    --eval_path ./experiments/sampled_results