Image Semantic Segmentation Based on Multi-layer Feature Information Fusion and Dual Convolutional Attention Mechanism

Lin Teng1, Yulong Qiao1, Jinfeng Wang2, Mirjana Ivanović3 and Shoulin Yin1, 4

  1. School of Information and Communication Engineering, Harbin Engineering University
    145 Nantong Street, Nangang District, Harbin, 150001, China
    qiaoyulong@hrbeu.edu.cn
  2. Weifang Vocational College
    Weifang, 261041 China
    1057417325@qq.com
  3. Faculty of Sciences, University of Novi Sad, Serbia
    mira@dmi.uns.ac.rs
  4. Software College, Shenyang Normal University
    Shenyang, 110034 China
    yslin@hit.edu.cn

Abstract

Traditional semantic segmentation methods have problems such as poor multi-scale feature extraction ability, weak lightweight backbone network feature extraction ability, lack of effective fusion of context information, resulting in edge segmentation errors and feature discontinuity. In this paper, a novel semantic segmentation model based on multi-layer information fusion and dual convolutional attention mechanism is proposed. In this method, SegFormer network is used as the backbone network, and multi-scale features of encoder output are fused with overlapping features. The feature extraction subnetwork is optimized by constructing the object region enhancement module, and the intermediate feature map is refined adaptively in each convolutional block of the deep network, so as to strengthen the fine extraction of multi-dimensional feature information of complex images. Dual convolutional attention module is used to fusion high-level semantic information to avoid the loss of feature information caused by up-sampling operation and the influence of introducing noise, and refine the effect of target edge segmentation. At the same time, the feature pyramid grid is proposed to process the overlapping features, obtain the context information of different scales, and enhance the semantic expression of features. Finally, the features processed by the feature pyramid grid module are combined to improve the segmentation effect. The experimental results on the public data set show that the proposed method has better performance than the existing methods, and has better segmentation effect on the object edge in the scene.

Key words

Semantic segmentation, multi-layer information fusion, dual convolutional attention mechanism, feature pyramid grid

How to cite

Teng, L., Qiao, Y., Wang, J., Ivanović, M., Yin, S.: Image Semantic Segmentation Based on Multi-layer Feature Information Fusion and Dual Convolutional Attention Mechanism. Computer Science and Information Systems