MANILA, Philippines – One question often asked of and by fact-checkers is this: how do you know if the fact-checking program works? 

The fact-check group Science Feedback attempted to answer this question by analyzing Facebook data captured through CrowdTangle and BuzzSomo. They presented their findings at During Global Fact 9, a conference of fact check organizations held in Oslo, Norway from June 21 to 25. 

Science Feedback is a French fact check organization which focuses on science-related information. In a nutshell, the group found that Meta’s enforcement of its rules against repeat offender accounts, defined as those pages or groups that have posted at least two of its articles as false within a 90-day period, results in reduced engagement on these posts during the enforcement period. However, they also found that this did not change the behavior of repeat offenders in the long run, according to Emmanuel Vincent, director of Science Feedback.

“For the ones who are doing it unintentionally, there are anecdotes (that) they do corrections,” Vincent said during a breakout session at the recently held Global Fact 9. But this is not the case with groups and pages spreading misinformation on a massive scale. “They do it repeatedly. They really know what they are doing… Those pages and groups have no intention of changing their behavior in any way.” 

Because of this finding, Vincent recommended that Meta, the parent company of Facebook, take more information into account when enforcing rules against perennial purveyors of disinformation. “Ninety days is extremely short. We have some actors we know have published more than 100 over the last 2 years. This is valuable information.” 

He also recommended looking into other signals of disinformation other than fact-checking. For instance, what could be analyzed are the narratives that are being pushed in some groups, he said. “A lot of groups have become good at not explicitly saying that vaccines are going to kill your children. Still they are just saying that somebody died and he took a vaccine before.”  

Precious data source

In conducting its research, Science Feedback used a number of tools for gathering data. One of these is CrowdTangle, a public insights tool owned by Meta that makes it easy to follow, analyze, and report on what’s happening with public content on social media.

Acquired by Facebook’s parent company in 2016, CrowdTangle has been used by newsrooms to monitor and compare the performance of public assets and content on Facebook, Instagram. When Facebook clamped down on access to its APIs in 2018 following the Cambridge Analytica scandal, researchers, journalists, and fact checkers had to rely on CrowdTangle to monitor the spread of disinformation and misinformation in these platforms.

“We used CrowdTangle to identify all the pages, all the groups that publish misinformation and looked at statistics of posting behavior and whether they are gaining or losing audience,” Vincent told Rappler. 

It is a question of transparency, he said. “In our research we have been able to show how much Facebook is reducing the visibility of posts in these repeat offender accounts. We were able to transparently verify that Facebook is implementing this policy. Otherwise there is no way of knowing that. “We really need this kind of data if we want to be able to measure what is happening on the platform.” 

Faktisk, a fact check organization based in Norway has also been using CrowdTangle data to map the spread of false claims. In an article published in 2020, the group published a story showing how a contro­ver­sial COVID-19 article published by Norway’s public broadcasting network spread disin­for­mation to millions. The same article identified so-called “superspreaders” on Facebook and Twitter. 

Rappler has also been using CrowdTangle data extensively for investigating disinformation peddlers as well as those attacking journalists on Facebook. Some of these investigations have led to takedowns. 

Access to CrowdTangle is one of the main benefits of the fact-checking partnership with Meta, Kristoffer Egeberg of Faktisk, told Rappler. 

If reports that Meta is closing down CrowdTangle are true, then this benefit may not be available for long.  

As early as April 2021, Meta reportedly reassigned Crowdtangle staff to other divisions. More recently, Bloomberg reported that shutdown plans are still on track and some Facebook engineers have been tasked to killing the tool. The official internal process to shut down CrowdTangle which was initiated in February was initially paused due to deliberations over the Digital Services Act, according to Bloomberg. 

Commitment to transparency

Vincent said this reported move in fact goes against the company’s commitments under the new Code of Practice against Disinformation, which asks for more transparency about what the platforms are doing against disinformation. “This move would clearly make it less transparent for outside researchers to see what is happening on the platforms.” 

In particular, the Code requires that safe design practices are put in place to limit the spread of disinformation and ensure more transparency of their recommender systems, adapting them to limit the propagation of disinformation. 

The code also provides for the creation of a Transparency Center that is accessible to all citizens, and which will “allow for an easy overview of the implementation of the Code’s measures, providing transparency and regular updates of relevant data.” 

Signatories to the code include Meta, Google, Tiktok, Twitter and other tech giants. The strengthened Code of Practice, along with the Digital Services Act and the upcoming legislation on transparency and targeting of political advertising, is an essential part of the European Commission’s toolbox for fighting the spread of disinformation.

For now, CrowdTangle is still accessible.

“If it is true that this tool (CrowdTangle) will be discontinued, that would make it much more difficult or maybe even impossible for external researchers to verify if Facebook is actually doing something and how much of an impact it has on the circulation of disinformation,” Science Feedback’s Vincent tells Rappler. 

Faktisk’s Egebert added that this would hurt their work. “It will also make our fact-checking partnership with Meta less relevant, as access to, and our ability to influence the development of, CrowdTangle is one of the main benefits.”

He goes further: “If the reasoning for this is what these rumors claim (too much transparency), it shows how little those in power at Meta has learned or valued from our partnerships. That their own transparency through this tool also enables critical press, should be welcomed.” – Rappler.com