LAUSR

Sign language (SL) is used by hearing impaired and deaf people. The WiFi-based sign language recognition (SLR) technology has attracted much attention due to its contactless nature and wide applications. Most previous SLR works are designed to recognize a single isolated sign word in data samples. However, such assumption is not realistic in practical applications, such as, in daily communication, deaf people usually express their mind through a phrase (a.k.a. a group of words) rather than an isolated word. Compared with the previous works, the sign words in a phrase have variety of length, sequential patterns, and combinations in realistic communications between deaf people. Therefore, it is challenging in exploiting the WiFi signals to accurately capture the unique patterns of SL in the previous natural setting. In this article, we propose Wi-Phrase, a multigesture context-awareness SLR system. Wi-Phrase exploits WiFi signals to translate SL to the English phrase. To achieve this, Wi-Phrase employs principal component analysis (PCA) projection to filter out the noise and convert cleaned WiFi signals to spectrogram. Then, we propose a novel Residual-MultiHead model that exploit residual learn structure to obtain local patterns of phrases and adopt multihead block to capture the global context information of phrase. To prove the advanced nature of our model, we design a WiFi-based SL phrase data set of 40 categories for experiments. Our comprehensive evaluation shows that Wi-Phrase achieves an accurate phrase recognition accuracy of 95.03%. In future, we envision Wi-Phrase could be widely used as the phrase command control system for deaf people in IoT devices.