Transformer-Based Optimization for Text-to-Gloss in Low-Resource Neural Machine Translation

Abstract

Sign Language is the primary means of communication for the Deaf and Hard of Hearing community. These gesture-based languages combine hand signs with face and body gestures for effective communication. However, despite the recent advancements in Signal Processing and Neural Machine Translation, more studies overlook speech-to-sign language translation in favor of sign language recognition and sign language to text translation. This study addresses this critical research gap by presenting a novel transformer-based Neural Machine Translation model specifically tailored for real-time text-to-GLOSS translation. First, we conduct trials to determine the best optimizer for our task. The trials involve optimizing a minimal model, and our complex model with different optimizers; The findings from these trials show that both Adaptive Gradient (AdaGrad) and Adaptive Momentum (Adam) offer significantly better performance than Stochastic Gradient Descent (SGD) and Adaptive Delta (AdaDelta) in the minimal model scenario, however, Adam offers significantly better performance in the complex model optimization task. To optimize our transformer-based model and obtain the optimal hyper-parameter set, we propose a consecutive hyper-parameter exploration technique. With a 55.18 Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score, and a 63.6 BiLingual Evaluation Understudy 1 (BLEU1) score, our proposed model not only outperforms state-of-the-art models on the Phoenix14T dataset but also outperforms some of the best alternative architectures, specifically Convolutional Neural Network (CNN), Long Short Term Memory (LSTM), and Gated Recurrent Unit (GRU). Additionally, we benchmark our model with real-time inference tests on both CPU and GPU, providing insights into its practical efficiency and deployment feasibility.