基于ECA-Net的双信息流图像字幕生成方法研究
DOI:
作者:
作者单位:

兰州理工大学

作者简介:

通讯作者:

中图分类号:

TP391

基金项目:

国家自然科学基金项目


Research on Image Captioning Generation Method of Double Information Flow Based on ECA-Net
Author:
Affiliation:

Lanzhou University of Technology

Fund Project:

The National Natural Science Foundation of China (General Program, Key Program, Major Research Plan)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对图像字幕生成中由于视觉信息不足使生成的描述语句与图像内容不匹配的问题,提出一种基于ECA-Net的双信息流图像字幕生成方法。首先,该方法将图像分割特征作为另一个视觉信息源,采用迭代独立层归一化模块融合分割特征和网格特征,以双信息流网络提取图像特征;其次,在编码器中添加高效通道注意力模块,通过跨通道交互学习图像特征之间的相关性,使预测结果更加关注视觉内容。最后,解码器根据提供的视觉信息和部分生成的字幕预测下一个词组,从而生成准确的字幕。在MSCOCO数据集上进行实验证明,该方法可以增强图像视觉信息之间的依赖性,使生成字幕相关度更高,语法更准确。

    Abstract:

    In order to solve the problem of mismatch between description statements and image content due to insufficient visual information in image captioning generation, the image captioning generation method is proposed based on ECA-Net. Firstly, the image segmentation feature is used as another visual information source, and an iterative independent layer normalization module is used to fuse segmentation feature and grid features, extracting image feature using a dual information flow network. Secondly, an efficient channel attention module is added to the encoder to learn the correlation between image features through cross-channel interaction, so that the prediction results are more focused on visual content. Finally, the decoder predicts the next phrase based on the provided visual information and the partially generated captions, thus generating accurate captions. Experimental results on MSCOCO dataset have shown that the proposed method can enhance the dependency between the visual information of images, resulting in higher correlation and more accurate grammar in generating subtitles.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2023-06-18
  • 最后修改日期:2023-09-07
  • 录用日期:2023-11-08
  • 在线发布日期:
  • 出版日期: