LAUSR.org creates dashboard-style pages of related content for over 1.5 million academic articles. Sign Up to like articles & get recommendations!

Global–Local 3-D Convolutional Transformer Network for Hyperspectral Image Classification

Photo by ldxcreative from unsplash

Benefiting from powerful feature extraction capabilities, convolutional neural networks (CNNs) have gained prominence in hyperspectral image (HSI) classification. Nevertheless, with restricted receptive fields of convolution kernels, CNN-based methods fail to… Click to show full abstract

Benefiting from powerful feature extraction capabilities, convolutional neural networks (CNNs) have gained prominence in hyperspectral image (HSI) classification. Nevertheless, with restricted receptive fields of convolution kernels, CNN-based methods fail to learn complex characteristics of long-range sequences. Meanwhile, the vision transformer allows us to learn long-range dependencies in a global view, but local region features are ignored. To overcome these limitations, we propose a novel method entitled global–local 3-D convolutional transformer network (GTCT), where 3-D convolution is embedded in a dual-branch transformer to simultaneously capture global–local associations in both the spectral and spatial domains. In particular, the global–local spectral convolutional transformer (GECT) is designed to exploit global spectral sequence signatures and local spectral relationships between bands. Symmetrically, the global–local spatial convolutional transformer (GACT) is devised to exploit local spatial context features and global interactions among different pixels. In addition, multiscale global–local spectral–spatial information is adaptively fused with trainable weights by the weighted multiscale spectral–spatial feature interaction (WMSFI) module. It is worth noting that a spectral–spatial global attention mechanism (SSGAM) is incorporated into multihead convolutional attention to further integrate discriminative spectral–spatial information. Extensive experiments on four HSI datasets, including the GF-5 and ZY1-02D satellite HSIs, demonstrate the superiority of the proposed GTCT method over other state-of-the-art algorithms with fewer parameters and lower floating-point operations (FLOPs) in practical applications.

Keywords: convolutional transformer; spectral spatial; local convolutional; global local; hyperspectral image; transformer

Journal Title: IEEE Transactions on Geoscience and Remote Sensing
Year Published: 2023

Link to full text (if available)


Share on Social Media:                               Sign Up to like & get
recommendations!

Related content

More Information              News              Social Media              Video              Recommended



                Click one of the above tabs to view related content.