US-Byte: An Efficient Communication Framework for Scheduling Unequal-Sized Tensor Blocks in Distributed Deep Learning
Sign Up to like & getrecommendations! Published in 2024 at "IEEE Transactions on Parallel and Distributed Systems"
DOI: 10.1109/tpds.2023.3331372
Abstract: The communication bottleneck severely constrains the scalability of distributed deep learning, and efficient communication scheduling accelerates distributed DNN training by overlapping computation and communication tasks. However, existing approaches based on tensor partitioning are not efficient… read more here.
Keywords: tensor; tensor blocks; communication framework; communication ... See more keywords