基于混合知识蒸馏的轻量级胸部疾病分类算法
CSTR:
作者:
作者单位:

(1.天津大学 微电子学院,天津 300072;2.天津大学 电气自动化与信息工程学院,天津 300072;3.天津市胸科医院 心血管外科,天津 300222)

作者简介:

李 锵 (1974-),男,博士,教授,博士生导师,主要从事医学图像处理、数字系统与微系统设计方面的研究 。

通讯作者:

中图分类号:

TP391.4

基金项目:

国家自然科学基金(61471263, 62272337)、 天津市自然科学基金(16JCZDJC31100)和天津大学自主创新基金(2021XZC-0024) 资助项目


Lightweight thoracic disease classification algorithm based on mixed knowledge distillation
Author:
Affiliation:

(1.School of Microelectronics, Tianjin University, Tianjin 300072, China;2.School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China;3.Department of Cardiovascular Surgery, Tianjin Chest Hospital, Tianjin 300222, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对现有胸部疾病分类算法参数量较大、对运行设备的硬件资源要求较高的问题,本文基于混合知识蒸馏(knowledge distillation,KD) 的训练策略提出一种轻量级胸部疾病分类算法RMSNet。首先,该算法将优化后的残差收缩模块加入到基础网络MobileViT中,利用软阈值化的方式过滤X光片中的背景噪声;其次,提出混合知识蒸馏训练策略,利用多层级注意力图和相似性激活矩阵作为监督信号,提升轻量级模型的分类性能;最后,使用焦点损失函数(focal loss,FL) 缓解数据集正负样本不均衡的问题。在ChestX-ray14数据集上展开验证,蒸馏训练后的RMSNet学生模型识别14类胸部疾病的平均AUC值为0.836,而参数量和浮点计算量分别为0.96 M和0.27 G。实验结果表明,本文算法在保持轻量化的同时分类精度更高,能有效降低算法运行时的硬件要求。

    Abstract:

    Existing lightweight networks for classifying thoracic diseases have a large number of parameters and require significant hardware resources.This paper proposes a lightweight algorithm for classifying thoracic diseases based on mixed knowledge distillation (KD) training strategy.Firstly, the algorithm incorporates an optimized residual shrinkage module into the MobileViT base network and employs soft thresholding to filter background noise in X-ray images.Then a mixed knowledge distillation training strategy is proposed,utilizing multi-level attention maps and similarity activation matrices as supervisory signals to enhance the ability of lightweight networks to recognize thoracic diseases.Finally, the focal loss function is employed to address the imbalance between positive and negative samples in the dataset.Experimental results on the ChestX-Ray14 dataset demonstrate that the average AUC value for the RMSNet student model trained with distilled knowledge to recognize 14 types of thoracic diseases is 0.836.The number of parameters and FLOPs are only 0.96 M and 0.27 G,respectively.These results indicate that the proposed algorithm improves classification accuracy while maintaining lightweight,enabling the network to run with less hardware.

    参考文献
    相似文献
    引证文献
引用本文

赖裕,李锵,聂为之,白云鹏,赵丰.基于混合知识蒸馏的轻量级胸部疾病分类算法[J].光电子激光,2024,35(9):993~1000

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2023-02-20
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2024-08-19
  • 出版日期:
文章二维码