DropDim: A Regularization Method for Transformer Networks
Sign Up to like & getrecommendations! Published in 2023 at "IEEE Signal Processing Letters"
DOI: 10.1109/lsp.2022.3140693
Abstract: We introduce DropDim, a structured dropout method designed for regularizing the self-attention mechanism, which is a key component of the transformer. In contrast to the general dropout method, which randomly drops neurons, DropDim drops part… read more here.
Keywords: method; regularization method; embedding dimensions; dropdim regularization ... See more keywords