Sofia Lundborg defended her Master Thesis on June 4, 2020. Congrats!

Sofia Lundborg defended her Master Thesis in Complex Adaptive Systems at Chalmers University of Technology on 4 June 2020. Congrats!

Title: Training Binary Deep Neural Networks Using Knowledge Distillation

Binary networks can be used to speed up inference time and make image analysis possible on less powerful devices. When binarizing a network the accuracy drops.
The thesis aimed to investigate how the accuracy of a binary network can be improved by using knowledge distillation.
Three different knowledge distillation methods were tested for various network types. Additionally, different architectures of a residual block in ResNet were suggested and tested. Test on CIFAR10 showed an 1.5% increase in accuracy when using knowledge distillation and an increase of 1.1% when testing on ImageNet dataset. The results indicate that the suggested knowledge distillation method can improve the accuracy of a binary network. Further testing needs to be done to verify the results, especially longer training. However, there is great potential that knowledge distillation can be used to boost the accuracy of binary networks.

Master programme: MPCAS – Complex Adaptive Systems
Supervisor: Giovanni Volpe
Supervisors @ Bit Addict: Karl Svensson, Fredrik Ring and Niclas Wikström
Examiner: Giovanni Volpe
Opponent: Viktor Olsson, Wilhelm Tranheden

Time: June 4, 2020 at 15:00
Place: Online via Zoom