LAUSR

DropDim: A Regularization Method for Transformer Networks

Sign Up to like & get
recommendations!
1 Published in 2023 at "IEEE Signal Processing Letters"

DOI: 10.1109/lsp.2022.3140693

Abstract: We introduce DropDim, a structured dropout method designed for regularizing the self-attention mechanism, which is a key component of the transformer. In contrast to the general dropout method, which randomly drops neurons, DropDim drops part… read more here.

Keywords: method; regularization method; embedding dimensions; dropdim regularization ... See more keywords