Google unveils SL2T, an AI that translates sign language into text

Google has unveiled sign-language-to-text, or SL2T, an AI system capable of translating sign language in video into text almost in real time. The new tool currently supports only American Sign Language, although Google says other sign languages will be added soon.

Yesterday, Google introduced its latest Google Pixel smartphones. Alongside the Pixel 11, its variants and new accessories, the company also announced several notable software additions. These include Tap to share, an Android feature that lets two people exchange contact details by bringing their smartphones close together.

Google also presented new health technologies, including an algorithm that identifies respiratory emergencies and automatically contacts emergency services when the user can no longer respond. For accessibility, the company is launching its sign-language-to-text, or SL2T, AI model, which can turn sign language into text with near-real-time results.

A sign language translator

Put simply, this AI receives sign language through a video feed and produces the matching text. Google created the feature to allow deaf and hard-of-hearing people to interact more naturally with products such as Gemini, rather than having to rely on a keyboard all the time.

The feature can also act as an interpreter between a deaf or hard-of-hearing person and someone who does not understand sign language. Its main limitation, however, is that availability of this new AI remains highly restricted for now.

Limited access to Google SL2T for now

“SL2T enables sign-to-text dictation in Gboard and Live Transcribe on Pixel 11, starting with American Sign Language (ASL) to English,” Google says. However, the company also promises to support more devices and more sign languages “soon”. Regarding languages, Google says it has already worked with 100,000 hours of data covering more than 50 different sign languages, with one quarter of that data involving American Sign Language.

Device support may be more complicated, as the feature could require a certain level of on-device computing power. SL2T partly operates locally: an on-device model transforms the signer’s video into geometric coordinates, before those geometric points are converted into text on Google’s servers. The benefit of this approach is improved privacy, since the video feeds themselves do not need to be processed in the cloud.

70 million people affected

An estimated 70 million people worldwide are deaf or hard of hearing. According to Google, while AI’s ability to process spoken language has advanced rapidly, this “revolution” has not yet benefited people who need to use sign language.

“Just as hearing users can use voice dictation to speak instead of typing text, this feature lets deaf users communicate in sign language on their phone wherever they would normally type text. You can use sign language to search the web, write messages or documents, and ask Gemini to answer questions or carry out tasks,” explains the Mountain View company.

Google recently also shared data on how people use Gemini, which has just passed one billion monthly active users. The company highlighted the growing popularity of voice interactions with AI: 63% of Gemini users speak directly to the AI, and some no longer use a keyboard at all to interact with it.