Tencent and China Unicom have jointly developed speech-to-text converter that is changing the life of approximately 28 million people who are hard of hearing in China.

The mobile app is powered by Tencent’s AI speech-enhanced algorithm to assist listening. This technology is developed by Tencent Meeting’s Ethereal Audio Lab, which shares its AI audio technology for free to improve life quality for people who are hard of hearing. Since 2020, the Ethereal Audio Lab and Tencent Charity Foundation worked with partners including the China Association of the Deaf to offer hearing aid solutions. 

Zhihui Chen, a soccer player with hearing loss, is on a video chat with her coach through the app powered by the Ethereal Audio Lab’s AI technology. 

Incoming calls are switched into a speech-to-text interface, starting with a customized message. It informs the caller about the feature’s activation and understanding for the delay in response.

When the call is connected, every word coming from is automatically converted into text and appears on the screen on the recipient’s end. 

Users can type text in the box and convert it into a voice message. 

Vice versa, the receiver of the call can select pre-programmed messages or type them manually to convert them into audio.

The same logic is used for video calls. On top of the usual video call interface, the caller’s speech is converted into texts in real-time on top of their screen.

The single-byte speech recognition rate and the real-time caption recognition accuracy rate increased by 66 percent and 6 percent to 10 percent, respectively.

The feature’s announcement comes shortly after Tencent’s barrier-free emergency feature launched in December, a function to help users call the ambulance or police without having to communicate using voice.

Through the “Barrier-Free First Aid" or “Penguin Emergency” Mini Programs, the hearing-impaired person can call the emergency medical center by text. The function translates the text into voice and sends it to the nearest local emergency center. The dispatcher receives a special alert and the caller’s location information is transmitted to the emergency center simultaneously.

Users can press the SOS button on the Penguin Emergency Mini Program to get help in emergencies.

Tencent said this feature is backed by Tencent Cloud's voice recognition technology, which recognizes 24 dialects and received 98 percent accuracy.

Tencent is cooperating with the first aid centers across China, starting with 24 cities including Beijing, Nanjing, Wuhan, and Jiujiang. 

Tencent said it will continue to build a more inclusive platform by adding cloud computing capability for the emergency service, AED equipment, first aid volunteers, and others.