LAUSR

Spatiotemporal fusion is a topical framework for solving the mutual restricted problem between the spatial and temporal resolution of satellite images. We pioneer an approach to replace similarity measurement steps in spatiotemporal fusion algorithms with convolutional neural networks, building a bridge between weight function-based models and the learning-based models. Specifically, we propose a non-local form that separates the relational computation part from the value representation part, and construct the convolutional neural network-based similarity weights learning block for learning normalized weights. The block can be inserted into STARFM to replace the manually designed weights calculation rules common in weight function-based methods, or into the convolutional neural network model StfNet to better utilize neighboring high-resolution images. The trained model output a high-resolution prediction from each base date image pair. The final result is a combination of the two predictions. In this regard, we propose the standard deviation-based weights to combine two prediction results. Four experiments are performed on Landsat-MODIS image pairs to determine the following: 1) the performance of the model at the target training date, 2) the generalization of the model in the target training time period, 3) the generalization of the model at different dates and different geographical locations, each considering the different case of giving one and two pairs of known images. Experimental results demonstrate the superiority of the similarity weights learning block and standard deviation-based weights. Among them, STARFM with the similarity weights learning block exhibits strong generalization, which testifies to the practical value of our model.