Understanding and Characterizing Communication Characteristics for Distributed Transformer Models
Sign Up to like & getrecommendations! Published in 2025 at "IEEE Micro"
DOI: 10.1109/mm.2025.3531323
Abstract: The transformer architecture has revolutionized many applications, such as large language models. This progress has been largely enabled by distributed training, yet communication remains a significant bottleneck. This article examines the communication behavior of transformer… read more here.
Keywords: understanding characterizing; transformer models; language; characterizing communication ... See more keywords