CALIFORNIA, USA – Samsung on Wednesday, January 17 (January 18, Philippine time), announced its newest S-series flagship device, the S24, coming in its usual 3 variants, the base version, the Plus, and the Ultra. 

The biggest change this year is on the software side. The S24 brings a host of familiar AI-powered tools.

Top among these is its “AI Live Translate Call” that allows a user to call and converse with another person speaking in a different language. As a user speaks, the phone provides a text translation that the person being spoken to would be able to understand. The phone also reads aloud the translation.

The words of the person on the other end are also translated for the user. 

This is perhaps one of the best examples of AI integration into a phone, onto one of its most elementary functions, calling. 

The processing for AI call translation is done on-device. This means that the voice data doesn’t have to be sent to the cloud, where it can be translated, and then sent back to the device. The S24 processes the data right on the device, which makes the process faster – a key element as slow translations would make for a potentially awkward, time-consuming conversation. 

On-device processing is also theoretically better for data privacy as the data remains on the phone, and is not transported to a cloud service, whether that might be on Samsung’s servers or a third-party service. 

For the feature to function, language packs are downloaded onto the device. Filipino isn’t yet one of those options, but Samsung said that work is ongoing to offer more languages. 

Call translation works whatever type of device the user is calling, even landline phones, Samsung said. 

Similar to AI call translation is an interpreter feature that Samsung is gearing towards travelers. With the feature, users can use the phone as a translation device that doesn’t require a data signal to use as processing is also done on device. 

Generative AI imaging

Not all AI functions will be on device, however. For example, its generative AI-powered image editing features require cloud processing, and thus, a data connection, presumably given the large datasets that power generative AI. These features include the ability to select an object in a photo and move it to another spot, or to completely delete a person or an object in a photo.

The generative AI technology is able to fill in the blank spaces left by the deletion or movement of selected elements. 

An AI-powered slow motion feature allows users to choose segments in standard videos to be shown in slow motion. Regular videos are shot typically at a rate of 30 or 60 image frames per second – not enough for an effective slow motion effect. But the phone’s technology allows it to generate the extra image frames needed to create the slow motion effect. 

Other features include the ability to remove blur, add high dynamic range effects, erase mirror reflections, erase shadows, and remove moire or the lines that appear when taking a photo of a screen. 

Transcription, chat assist features, Circle to Search

One feature that certain professionals – journalists for example  – will like is the phone’s transcription feature. Often, there are paid AI-powered services that allow for the transcription of a voice recording. But the S24 is able to do that on-device, even without an internet connection. 

The phone can also summarize the transcription, with users having the option to get either a standard or a more detailed summary. It can also summarize documents including PDF files, and articles on news websites via Samsung’s own browser. Unlike transcription, summarization features require cloud processing.

Users can also get suggestions for how they want their chat message to sound, whether they want it to sound professional or casual, via the phone’s chat assist features.

Chat messages, documents, and transcriptions can also be translated into different languages.

Lastly, Circle to Search, accessible by long-pressing the home button, allows users to encircle an image on the browser or a photo, and get search results about the encircled item. For example, in a photo showing several food dishes, a user can encircle a particular dish to know what it is, and the phone, via Google, will show search results. 

Another example is a user can encircle an article of clothing on a person – a shoe, for example – or maybe a building in a cityscape, and the phone will be able to tell what the shoe is.

Many of these features such as translating voice or text, transcribing audio, or the ability to be an interpreter between people speaking different languages do not sound novel. But Samsung’s focus on integrating all these functions with a phone is, and that’s the special thing here.

Whereas the past years have seen phone makers up one another with their camera hardware, and imaging techniques (many of which are also AI-powered to an extent by the way), with the arrival of the S24, Samsung appears to usher in a new race.

And it’s a race that’s not just among chipmakers as to who can do the best on-device processing. For the phone brands, it’s about bringing about the smoothest ways they can integrate AI-enabled functions, both the familiar and the new and innovative, into their units. – Rappler.com 

Samsung sponsored the trip to the US where its Unpacked event was held.