Stolen profile pictures and images generated by artificial intelligence were used by a Chinese network to hide the nature of its fake accounts, a report from Graphika revealed on Tuesday, September 22.

Graphika’s findings are part of a report meant to explain how this cluster of Facebook and Instagram accounts operated.

The Chinese network was one of two networks Facebook took down on Tuesday. The other was an inauthentic Philippine network linked to the Philippine military.

Must Read

Facebook removes fake network linked to AFP, PNP

The report explained two methods by which specific fake accounts were able to hide in plain sight.

The first method was the reuse of stolen photographs taken from a number of sources online. These included photographs from Instagram, posts on Chinese social media platforms, and available footage from photo shoots.

For example, the network appeared to have used the same images above for accounts named Ann Perdue and Wendy Bowen.

Artificially-made profile pictures

In some cases, the network took advantage of a form of artificial intelligence known as Generative Adversarial Networks (GAN) to make fake profile pictures for use on their accounts.

Graphika noted the “AI is readily available online, and its use (or abuse) by covert operations has exploded in the last year.”

GAN-made images can’t be readily reverse-searched, but there are still telltale signs of fakery that can be detected by human sight.

Said Graphika, “The technology struggles with peripheral features, especially ears, glasses, and hair, and it struggles still more with photo backgrounds – since there is far more variation in the range of possible backgrounds a person can have than there is variation in the layout of the human face.”

Graphika added the Chinese operation “used a dozen such pictures among its accounts.” Stickers may have also been added to mask deficiencies in the GAN image generation.

Must Read

Chinese fake account network’s focus on Imee Marcos ‘particularly striking’ – Graphika

At least one of the accounts taken down used an AI-generated image to pretend to be an American and was used for influence operations in the US elections.

Mixed success

Graphika also noted the fake account operations’ success was mixed.

Said the report, “Two pages that focused on the Philippines attracted around 57,000 and 40,000 followers, respectively. A page that mainly posted about security in the South China Sea attracted 16,000. None of the other pages had more than 10,000 followers, and one, focused on criticizing Donald Trump, had none at all.”

Meanwhile, a group targeted towards the Philippines had over 51,000 members, but none of the operation’s other groups had more than 2,000. A group dedicated to US presidential hopeful Pete Buttigieg had only two members – both of which were run by the operation.

The full report is downloadable and available here. – Rappler.com