LAUSR.org creates dashboard-style pages of related content for over 1.5 million academic articles. Sign Up to like articles & get recommendations!

Effective Pan-Sharpening by Multiscale Invertible Neural Network and Heterogeneous Task Distilling

Photo by diggity_dog from unsplash

As recognized, the ground-truth multispectral (MS) images possess the complementary information (e.g., high-frequency components) of low-resolution (LR) MS images, which can be considered as privileged information to alleviate the spectral… Click to show full abstract

As recognized, the ground-truth multispectral (MS) images possess the complementary information (e.g., high-frequency components) of low-resolution (LR) MS images, which can be considered as privileged information to alleviate the spectral distortion and insufficient spatial texture enhancement. Since existing supervised pan-sharpening methods only utilize the ground-truth MS image to supervise the network training, its potential value has not been fully explored. To accomplish this, we propose a heterogeneous knowledge-distilling pan-sharpening framework that distills pan-sharpening by imitating the ground-truth reconstruction task in both the feature space and network output. In our work, the teacher network performs as a variational autoencoder to extract effective features of the ground-truth MS. The student network, acting as pan-sharpening, is trained by the assistance of the teacher network with the process-oriented feature imitation learning. Moreover, we design a customized information-lossless multiscale invertible neural module to effectively fuse LRMS and panchromatic (PAN) images, producing expected pan-sharpened results. To reduce the artifacts generated by the knowledge distillation process, a knowledge-driven refinement subnetwork is further devised according to the pan-sharpening imaging model. Extensive experimental results on different satellite datasets validate that the proposed network outperforms the state-of-the-art methods both visually and quantitatively. The source code will be released at https://github.com/manman1995/pansharpening.

Keywords: network; pan; ground truth; pan sharpening; invertible neural; multiscale invertible

Journal Title: IEEE Transactions on Geoscience and Remote Sensing
Year Published: 2022

Link to full text (if available)


Share on Social Media:                               Sign Up to like & get
recommendations!

Related content

More Information              News              Social Media              Video              Recommended



                Click one of the above tabs to view related content.