Necati Cihan Camgoz

TL;DR: This work formalizes SLT in the framework of Neural Machine Translation (NMT) for both end-to-end and pretrained settings (using expert knowledge) and allows to jointly learn the spatial representations, the underlying language model, and the mapping between sign and spoken language.

TL;DR: A novel deep learning approach to solve simultaneous alignment and recognition problems (referred to as “Sequence-to-sequence” learning) is proposed, which decompose the problem into a series of specialised expert systems referred to as SubUNets, and serves to significantly improve the performance of the overarching recognition system.

TL;DR: This work applies the approach to the domain of sign language recognition exploiting the sequential parallelism to learn sign language, mouth shape and hand shape classifiers and clearly outperform the state-of-the-art on all data sets and observe significantly faster convergence using the parallel alignment approach.

TL;DR: A novel transformer based architecture that jointly learns Continuous Sign Language Recognition and Translation while being trainable in an end-to-end manner is introduced by using a Connectionist Temporal Classification (CTC) loss to bind the recognition and translation problems into a single unified architecture.

TL;DR: Sign Language Transformers as mentioned in this paper use a Connectionist Temporal Classification (CTC) loss to bind the recognition and translation problems into a single unified architecture, which leads to significant performance gains.