U of A researchers develop new translation tool for quicker, more meaningful machine-human interactions

The rapid adoption and integration of online chatbots and other AI powered virtual assistants like Siri and Alexa are changing how we do business. Human-machine interactions face many challenges, however, including the quality and speed of AI systems — and U of A research expertise in natural language processing may hold the solutions for overcoming these obstacles.

The field of natural language processing encompasses the myriad ways that humans use language to interface with computers. In NLP, the trend is to build language models that use multi-layered deep learning networks to produce human-like text. It is a relatively slow process, with implications for both users and industry who rely on NLP to meet customers’ needs. 

Current text generation models are typically autoregressive — predicting one word at a time until the entire text is generated. The advantage of autoregressive translation is that it can take into account the impact of the previous words, thus producing high quality translation. 

Non-autoregressive text generation, which decodes target words in parallel, non-sequentially, is much faster but the quality of the translation is significantly reduced. This is because it does not take advantage of the interplay between the words and the narrative, says Department of Computing Science professor Osmar Zaïane and assistant professor Lili Mou, who propose a novel DPSL, which achieves both high efficiency and high quality. Zaïane and Mou, both Fellow and Canada CIFAR AI Chairs at Amii, validated their approach on four machine translation tasks, achieving 10x speed-up without loss of translation quality.

Mou says the key insight with DPSL is its capacity to train a non-autoregressive model using layer-wise prediction and deep supervision to improve the quality of translation layer by layer.

There are wide industrial applications for improvements to these techniques, explains Mou, citing Google Translate as an example which can take several seconds to translate text. 

“At the end of the day,” Mou says, “DPSL generates outputs that are consistent, better calibrated, and more meaningful than the predicted ones.” 

In NLP research, this is just the beginning, says Zaïane, adding that there is still a lot to be done. 

“This gate is opening,” he says, “and there are many dreams in front of us.”

The paper, Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision, will be presented at the 36th AAAI Conference on Artificial Intelligence, February 22 to March 1, 2022.

Related: 

• University of Alberta computing scientists develop machine learning tool to harness the power of Twitter for understanding health and wellness.