Abstract:Existing lightweight networks for classifying thoracic diseases have a large number of parameters and require significant hardware resources. This paper proposes a lightweight algorithm for classifying thoracic diseases based on mixed knowledge distillation training strategy. The algorithm incorporates an optimized residual shrinkage module into the MobileViT base network and employs soft thresholding to filter background noise in x-ray images. A mixed knowledge distillation training strategy is proposed, utilizing multi-level attention maps and similarity activation matrices as supervisory signals to enhance the ability of lightweight networks to recognize thoracic diseases. The focal loss function is employed to address the imbalance between positive and negative samples in the dataset. Experimental results on the ChestX-Ray14 dataset demonstrate that the average AUC value for the RMSNet student model trained with distilled knowledge to recognize thoracic diseases is 0.836. The number of parameters and computational complexity are only 0.96M and 0.27G, respectively. These results indicate that the proposed algorithm improves classification accuracy while maintaining a low number of parameters and FLOPs, enabling the network to run with less hardware.