LAUSR

Deep learning methods are popular for hyperspectral and multispectral image (HSI-MSI) fusion to obtain a high-resolution HSI. However, most of them are unsatisfactory due to limited generalization ability and poor interpretability. This article proposes a highly interpretable deep HSI-MSI fusion method based on probabilistic matrix factorization (PMF) under the Bayesian framework. In the proposed method, an HSI is factorized into two matrices, namely, the Gaussian-prior-regularized spectral matrix and the deep-prior-regularized abundance matrix. Then, we split the optimization process into two meaningful iterative updating steps: updating the spectral matrix based on least-squares estimation and updating the abundance matrix based on a convolutional neural network (CNN)-based Gaussian denoiser for 2-D gray images. To improve the generalization ability, we provide solutions for selections of hyperparameters, CNN-based denoiser architecture, and training strategy. Using the given solutions, the proposed fusion method can be trained with 2-D images once and then used to fuse different types of HSI and MSI excellently. Experiments on three datasets demonstrate that the proposed fusion method has good fusion performance and high generalization ability compared with other state-of-the-art methods. The source code will be available at https://github.com/KevinBHLin/.