sanity checking on new target

[linminhtoo]

hi Chao Shen,

i am creating another issue as this is different from the other one.

in your paper you mentioned that the performance varies depending on the protein. in real-world use-cases, we want to screen ligands against a specific target. so, there's uncertainty about whether the method will do well.

do you have any recommendations for how we can sanity check whether RTMScore can be used reliably for a new protein?

thanks

[sc8668]

I recommend you to assess our method on your own target in advance, e,g., docking power (redocking, cross-docking), screening power (testing whether our approach could distingush active compounds from inactive componds for your own target).

[Zhuyanjuan]

hi,I would like to ask you a question, Can I use it to screen large molecular databases?
Thanks!

[sc8668]

Of course you can have a try.

[Zhuyanjuan]

The database given in the literature cannot be downloaded. Is there any way to solve it？
thanks！

[sc8668]

I can access all the links provided in the litertature (http://www.pdbbind.org.cn/; https://
zenodo.org/record/5525936; https://zenodo.org/record/6859325). Perhaps there is something wrong with your network.

[Zhuyanjuan]

Ok, thank you for your answer.

[Zhuyanjuan]

Does the code need to change a lot if it is used to screen large molecule databases? After reading your paper, I don't understand one thing. Do I need to conduct docking for screening?Just need to feed the protein and the ligands ? thanks! | | yanjvanzhu | | ***@***.*** | ---- Replied Message ---- | From | ***@***.***> | | Date | 12/13/2022 11:14 | | To | ***@***.***> | | Cc | ***@***.***> , ***@***.***> | | Subject | Re: [sc8668/RTMScore] sanity checking on new target (Issue #4) | Of course you can have a try. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[sc8668]

You can directly use the code provided as the example for rescoring, without any changes. Just see our method as a generic scoring function like X-Score or AutoDock Vina. However, it should be noticed that our model in this version just serves as a scoring function rather than a docking program, so you need to generate the binding poses first. Additionally, it is better for you to generate multiple poses (e.g., 10) for a certain ligand for our model to select the most reasonable pose for rescoring, which may achieve better performance.

[Zhuyanjuan]

Need I to generate the binding poses between the each ligand in the database and a protein? Can I target a specific protein for screening large databases? Thank you very much! | | yanjvanzhu | | ***@***.*** | ---- Replied Message ---- | From | ***@***.***> | | Date | 12/13/2022 14:56 | | To | ***@***.***> | | Cc | ***@***.***> , ***@***.***> | | Subject | Re: [sc8668/RTMScore] sanity checking on new target (Issue #4) | You can directly use the code provided as the example for rescoring, without any changes. Just see our method as a generic scoring function like X-Score or AutoDock Vina. However, it should be noticed that our model in this version just serves as a scoring function rather than a docking program, so you need to generate the binding poses first. Additionally, it is better for you to generate multiple poses (e.g., 10) for a certain ligand for our model to select the most reasonable pose for rescoring, which may achieve better performance. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[sc8668]

Yes, you need to generate the binding poses between the each ligand in the database and a protein.
Of course you can, but it is better for you to assess our model on your own target first, since the performance of our method may be highly target-specific.

[Zhuyanjuan]

What does mean “since the performance of our method may be highly target-specific.”? Is it targeted protein 1qkt? Thank you very much for so many answers! | | yanjvanzhu | | ***@***.*** | ---- Replied Message ---- | From | ***@***.***> | | Date | 12/13/2022 16:14 | | To | ***@***.***> | | Cc | ***@***.***> , ***@***.***> | | Subject | Re: [sc8668/RTMScore] sanity checking on new target (Issue #4) | Yes, you need to generate the binding poses between the each ligand in the database and a protein. Of course you can, but it is better for you to assess our model on your own target first, since the performance of our method may be highly target-specific. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[sc8668]

As shown in Figure 4 in our paper, the virtual screening performance of our method may vary a lot when different targets are focused on. Hence of you try our method on your own target, I'm unclear how will it actually perform and thus I suggest you to conduct an initial assessment first.

[Zhuyanjuan]

Thank you very much for your answers! | | yanjvanzhu | | ***@***.*** | ---- Replied Message ---- | From | ***@***.***> | | Date | 12/13/2022 20:40 | | To | ***@***.***> | | Cc | ***@***.***> , ***@***.***> | | Subject | Re: [sc8668/RTMScore] sanity checking on new target (Issue #4) | As shown in Figure 4 in our paper, the virtual screening performance of our method may vary a lot when different targets are focused on. Hence of you try our method on your own target, I'm unclear how will it actually perform and thus I suggest you to conduct an initial assessment first. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[Zhuyanjuan]

Hi,I would also like to ask you a question, what does the EF 1% value mean? Is the bigger the better?The value of EF 1% indicates that the model is suitable for the target? Thank you! | | yanjvanzhu | | ***@***.*** | ---- Replied Message ---- | From | ***@***.***> | | Date | 12/13/2022 20:40 | | To | ***@***.***> | | Cc | ***@***.***> , ***@***.***> | | Subject | Re: [sc8668/RTMScore] sanity checking on new target (Issue #4) | As shown in Figure 4 in our paper, the virtual screening performance of our method may vary a lot when different targets are focused on. Hence of you try our method on your own target, I'm unclear how will it actually perform and thus I suggest you to conduct an initial assessment first. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[sc8668]

It is defined as the percentage of true binders observed among all of the true binders for 1% of the top-ranked candidates of
a chemical library.
The bigger means the better.
Not necessarily. It is just a metric for retrospective study, but not necessarily suitable for your own system in real-world applications since both the size of the screening library and the number of compounds you tend to select for purchase/synthesis are unknown. The simplest method to judge the performance of our model for your target may just see whether the model can make the active compounds lie in the front rank.