基于混合知识蒸馏的轻量级胸部疾病分类算法
DOI:
作者:
作者单位:

1.天津大学;2.天津市胸科医院

作者简介:

通讯作者:

中图分类号:

TP.391.4

基金项目:

国家自然科学基金项目(面上项目,重点项目,重大项目)


Lightweight thoracic disease classification algorithm based on mixed knowledge distillation
Author:
Affiliation:

1.Tianjin university;2.Tianjin Chest Hospital

Fund Project:

The National Natural Science Foundation of China (General Program, Key Program, Major Research Plan)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对现有胸部疾病分类算法参数量较大、对运行设备的硬件资源要求较高的问题,本文基于混合知识蒸馏的训练策略提出一种轻量级胸部疾病分类算法RMSNet。首先,该算法将优化后的残差收缩模块加入到基础网络MobileViT中,利用软阈值化的方式过滤X光片中的背景噪声;其次,提出混合知识蒸馏训练策略,利用多层级注意力图和相似性激活矩阵作为监督信号,提升轻量级模型的分类性能;最后,使用焦点损失函数缓解数据集正负样本不均衡的问题。在ChestX-ray14数据集上展开验证,蒸馏训练后的RMSNet学生模型识别14类胸部疾病的平均AUC值为0.836,而参数量和浮点计算量分别为0.96M和0.27G。实验结果表明,本文算法在保持轻量化的同时分类精度更高,能有效降低算法运行时的硬件要求

    Abstract:

    Existing lightweight networks for classifying thoracic diseases have a large number of parameters and require significant hardware resources. This paper proposes a lightweight algorithm for classifying thoracic diseases based on mixed knowledge distillation training strategy. The algorithm incorporates an optimized residual shrinkage module into the MobileViT base network and employs soft thresholding to filter background noise in x-ray images. A mixed knowledge distillation training strategy is proposed, utilizing multi-level attention maps and similarity activation matrices as supervisory signals to enhance the ability of lightweight networks to recognize thoracic diseases. The focal loss function is employed to address the imbalance between positive and negative samples in the dataset. Experimental results on the ChestX-Ray14 dataset demonstrate that the average AUC value for the RMSNet student model trained with distilled knowledge to recognize thoracic diseases is 0.836. The number of parameters and computational complexity are only 0.96M and 0.27G, respectively. These results indicate that the proposed algorithm improves classification accuracy while maintaining a low number of parameters and FLOPs, enabling the network to run with less hardware.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2023-02-20
  • 最后修改日期:2023-05-31
  • 录用日期:2023-06-08
  • 在线发布日期:
  • 出版日期: