On the diversity of multi-head attention
Sign Up to like & getrecommendations! Published in 2021 at "Neurocomputing"
DOI: 10.1016/j.neucom.2021.04.038
Abstract: Abstract Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions. In this work, we propose two approaches to better exploit such diversity for multi-head attention,… read more here.
Keywords: attention; diversity multi; head; head attention ... See more keywords
Intelligent Bearing Fault Diagnosis Using Multi-Head Attention-Based CNN
Sign Up to like & getrecommendations! Published in 2020 at "Procedia Manufacturing"
DOI: 10.1016/j.promfg.2020.07.005
Abstract: Abstract Aiming at automatic feature extraction and fault recognition of rolling bearings, a new data-driven intelligent fault diagnosis approach using multi-head attention and convolutional neural network (CNN) is proposed. Firstly, a simple signal-to-image spatial transform… read more here.
Keywords: fault; multi head; diagnosis; bearing fault ... See more keywords
Classification of ECG using ensemble of residual CNNs with or without attention mechanism
Sign Up to like & getrecommendations! Published in 2022 at "Physiological Measurement"
DOI: 10.1088/1361-6579/ac647c
Abstract: Objective. This paper introduces a winning solution (team ISIBrno-AIMT) to the official round of PhysioNet Challenge 2021. The main goal of the challenge was a classification of ECG recordings into 26 multi-label pathological classes with… read more here.
Keywords: attention mechanism; classification; multi head; head attention ... See more keywords
Multiscaled Multi-Head Attention-Based Video Transformer Network for Hand Gesture Recognition
Sign Up to like & getrecommendations! Published in 2023 at "IEEE Signal Processing Letters"
DOI: 10.1109/lsp.2023.3241857
Abstract: Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand… read more here.
Keywords: recognition; gesture recognition; hand; head attention ... See more keywords