Photo by manish_mk100 from unsplash
Multi-modal neural networks have become increasingly pervasive in many machine learning application domains due to their superior accuracy by fusing various modalities. However, they present many unique characteristics such as… Click to show full abstract
Multi-modal neural networks have become increasingly pervasive in many machine learning application domains due to their superior accuracy by fusing various modalities. However, they present many unique characteristics such as multi-stage execution, frequent synchronization and high heterogeneity, which are not well understood in the system and architecture community. In this article, we first present and characterize a set of multi-modal neural network workloads of different sizes at inference stage. We then explore their important implications from system and architecture aspects. We hope that our work can help guide future software/hardware design and optimization for efficient inference of multi-modal DNN applications.
Journal Title: IEEE Computer Architecture Letters
Year Published: 2022
Link to full text (if available)
Share on Social Media: Sign Up to like & get
recommendations!