Pytorch model

[kfarivar]

Hi
I wanted to know how I can calculate the clever score for a pytorch model ?

[pinyuchen]

Hello @kfarivar
You can use the general wrapper from IBM Adversarial Robustness Toolbox: https://adversarial-robustness-toolbox.readthedocs.io/en/stable/modules/metrics.html

[huanzhang12]

Thank you @pinyuchen for the answer. Let us know if you have any further questions.

[kfarivar]

I have seen the wrapper. It has 2 major problems. 1. The implementation is not efficient. The original implementation uses multitasking library while theirs is a serial implementation.
2. The implementation is incomplete. The original code estimates the c_init while that's not implemented in the wrapper.

[huanzhang12]

@kfarivar I agree with you that the original implementation is more efficient. I did a lot of multiprocessing optimizations to fully utilize the GPU. The new implementation was done by an independent team which focuses more on functionality rather than performance.

It is possible to change the original implementation to use a PyTorch model. CLEVER is a sampling based method so as long as you can integrate pytorch model inference into the code, everything else should work. More precisely, you will need to replace this tensorflow sess.run with a pytorch model inference:

https://github.com/huanzhang12/CLEVER/blob/master/estimate_gradient_norm.py#L461

Most other things can be kept unchanged, including the multitasking part. However it does need some efforts if you plan to take this route.

[liuyiming199721]

Hello, is there an easy way to use your pytorch models?