MANILA, Philippines – Google’s releasing cheaper Pixel phones, but far more intriguing than that is a new Android feature that can provide real-time captions to what appears to be all the video and audio content on your phone.

Whether you’re playing a YouTube video or playing a video you yourself recorded, Google’s Live Caption is supposed to be able to provide subtitles to those automatically. It’s part of the upcoming Android Q mobile operating system – the successor to Android Pie – and it is expected to roll out around August.

Google says it’s both a necessary accessibility feature and one of convenience.

It will make more content available to the “466 million deaf and hard of hearing people around the world,” said Google in its blog. It’s also a very appealing feature for those who watch videos on the go, when they’re in a public, crowded place, or when they don’t want to disturb others – even without a pair of earphones.

“Live Caption brings real-time captions to media on your phone – videos, podcasts, and audio messages, across any app – even stuff you record yourself,” said Google.

The feature also won’t require a network connection as it relies on on-device machine learning as opposed to fetching and sending data over the cloud. It’s an evolution of the automatic captions Google introduced to YouTube videos in 2009. CNET says it uses technology similar to Smart Reply, which provides users reply suggestions when composing an email.

With Live Caption, “everything happens on the device, thanks to a breakthrough in speech recognition that we made earlier this year,” said the company. For better privacy, Google also assured that “no audio stream ever leaves your device.”

The Verge, who has had hands-on time with the feature, called it “impressive” and that the results are “extremely accurate.” 

To use the feature, one needs to press one of the volume rocker buttons to bring up the traditional volume indicator. An icon appears along with the volume indicator, which can be tapped to activate it. The user then has to play the video, and the captions will appear in a black box that can be moved around depending on where the user wants it. – Rappler.com