基于时空图卷积网络的学生在线课堂行为识别
DOI:
CSTR:
作者:
作者单位:

(西北师范大学 计算机科学与工程学院,甘肃 兰州 730070)

作者简介:

齐永锋(1972-),男,博士,教授,硕士生导师,主要从事图像处理与模式识别方面的研究.

通讯作者:

中图分类号:

基金项目:

甘肃省科技计划项目(18JR3RA097)资助项目 (西北师范大学 计算机科学与工程学院,甘肃 兰州 730070)


Recognition of students′ online classroom action based on spatio- temporal graph convolutional network
Author:
Affiliation:

(College of Computer Science and Engineering,Northwest Normal University, Lanzhou, Gansu 730070, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    为了有效地识别学生在线课堂行为,提出了一种融合全局注意力机制和时空图卷积 网络的人体骨架行为识别模型。首先在时空图卷积网络的空间图卷积网络和时间卷积网络之 间加入全局注意力模块,空间图卷积网络输出的空间特征图作为注意力模块的输入。其次引 入按时间维度的平均池化和最大池化操作,以增加模型学习全局特征信息的能力。最后用三 个加入注意力机制的时空图卷积神经网络和类激活图(class activation map,CAM),构造对遮挡数据识别能力更强 的丰富激活图卷积网络(RA-GCNv2-A)模型,并通过迁移学习实现学生在线课堂行为识别功 能。 在NTU-RGB+D和NTU-RGB+D120数据集上进行实验验证,与RA-GCNv2模型相比,在NTU-RGB +D 和NTU-RGB+D120数据集上的识别准确率分别提高了(cross-subject,CS)1.3%、(cross-view,CV)1.2%和(cross-subject,CSub)1.6%、 (cross-setup,CSet)1.4%。实验结果表明,提出的方法是一种有效的学生在线课堂 行为识别方法。

    Abstract:

    In order to effectively identify students′ online classroom action,a h uman skeleton action recognition model integrating global attention mechanism and spatiotempor al convolution network is proposed.Firstly,a global attention module is added bet ween the spatial graph convolutional network and the temporal convolutional network of the Spatio temporal graph convolutional neural network,and the spatial feature map output by the sp atial graph convolutional network is used as the input of the attention module;Secondly,av erage pooling and maximum pooling operations according to the time dimension are introduced to increase the ability of the model to learn global feature information.Finally,three spa tiotemporal graph convolutional neural networks and class activation map (CAM) added to the atten tion mechanism are used to construct a rich activation map convolutional network with stronger ability to recognize occlusion data (RA-GCNv2-A) model,and realize student onl ine classroom action recognition function through transfer learning.Experimental ve rification is performed on the NTU-RGB+D and NTU-RGB+D120two datasets.Compared with the RA - GCNv2model,the recognition accuracy on the NTU-RGB+D dataset is increased by 1.3% (cross-subject,CS),1.2% (cross-view,CV),the recognition accuracy on the NTU-RGB+D120dataset is increas ed by 1.6% (cross-subject,CSub),1.4% (cross-setup,CSet) respectively.The experimental results show that the pr oposed method is an effective way to recognize students′ online classroom action.

    参考文献
    相似文献
    引证文献
引用本文

胡锦林,齐永锋,王佳颖.基于时空图卷积网络的学生在线课堂行为识别[J].光电子激光,2022,33(2):149~156

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-06-03
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2022-03-24
  • 出版日期:
文章二维码