Mentor: Prof. R.M. Hegde, Department of Electrical Engineering, IITK
– Learned deep learning models using PyTorch with both fully connected and convolutional architectures.
– Evaluated student-teacher model performance in terms of accuracy and parameter efficiency.
– Implemented and compared three major KD strategies — response-based, feature-based, and relation-based distillation — on the FashionMNIST dataset.