Have you tired to train the baseline and grafting models both with cosine learning rate?

[junleiz]

Hello:
I noticed that the acc would drop after the decreasing epoch. I trained your baseline model and grafting model with the same cosine learning rate. The acc for the baseline (mobilenetv2) is 72.83, and that for the grafting model(2 models) is 72.80. The acc decreased! I noticed that you never tried the cosine lr. I wonder that is there any hyperparameters wrong (I changed nothing in your grafting.py)?

[fxmeng]

Thanks for your attention,
We compared the effect of grafting and baseline using mobilenetv2 model on cifar100 dataset, and found that grafting works for both steplr and coslr(The initial learning rate we used is 0.1).
In addition, coslr and grafting are two different ways of optimizing training, and do not necessarily have the effect of 1 + 1> 2.

[fxmeng]