LAUSR.org creates dashboard-style pages of related content for over 1.5 million academic articles. Sign Up to like articles & get recommendations!

Fine-Grained Motion Estimation for Video Frame Interpolation

Photo by angelekamp from unsplash

Recent advances in video frame interpolation have shown that convolutional neural networks combined with optical flow are capable of producing a high-quality intermediate frame between two consecutive input frames in… Click to show full abstract

Recent advances in video frame interpolation have shown that convolutional neural networks combined with optical flow are capable of producing a high-quality intermediate frame between two consecutive input frames in most scenes. However, existing methods have difficulties dealing with large and non-uniform motions that widely exist in real-world scenes because they often adopt the same strategy to deal with different motions, which easily results in unsatisfactory results. In this article, we propose a novel fine-grained motion estimation approach (FGME) for video frame interpolation. It mainly contains two strategies: multi-scale coarse-to-fine optimization and multiple motion features estimation. The first strategy is to gradually refine optical flows and weight maps, both of which are used to synthesize the target frame. The second strategy aims to provide fine-grained motion features by generating multiple optical flows and weight maps. To demonstrate its effectiveness, we propose a fully convolutional neural network with three refinement scales and four motion features. Surprisingly, this simple network produces state-of-the-art results on three standard benchmark datasets and real-world examples, with advantages in terms of effectiveness, simplicity, and network size over other existing approaches. Furthermore, we demonstrate that the FGME approach has good generality and can significantly improve the synthesis quality of other methods.

Keywords: video frame; grained motion; frame interpolation; fine grained; motion

Journal Title: IEEE Transactions on Broadcasting
Year Published: 2021

Link to full text (if available)


Share on Social Media:                               Sign Up to like & get
recommendations!

Related content

More Information              News              Social Media              Video              Recommended



                Click one of the above tabs to view related content.