Menu Close

What happened when Twitter and other social media platforms cracked down on extremists

Add to my bookmarks

Share This Article:

Picture: Reuters/Kacper Pempel/Illustration – ‘I think we’ve entered a new phase in which social media has altered and warped how we encounter information, how we process it, how we internalise what counts as the truth. It’s having significant impacts on our democracy,’ the writer says.

By AC Thompson

Since Donald Trump’s election in 2016, an entire ecosystem of right-wing social media platforms has come into existence — from Gab (where the alleged Pittsburgh synagogue shooter posted hateful screeds) to Parler (a hotspot for insurrectionary activities in the run-up to January 6) to the former president’s own Truth Social (that was frequented by a fan of his who was recently shot dead after attacking a Cincinnati FBI office). This new wave of apps and sites follows in the footsteps of 4chan and 8kun, older internet message boards that continue to attract a sizable audience of conspiracy theorists and violent racists.

Welton Chang knows this corner of the digital world well. A former Army intelligence officer and human rights activist, Chang runs Pyrra, a small tech startup dedicated to identifying and tracking the extremist ideas circulating in these spaces. Pyrra, which launched in early 2022 with $1.3 million in funding, monitors more than 20 alternative social media sites and online forums, scanning some 100 million messages per week.

Chang, a data scientist, says increased content moderation at major social media platforms — including the ouster of figures ranging from Trump to Alex Jones — has driven a sizable contingent of users to the spaces Pyrra tracks, which tend toward an absolutist view of free speech.

This conversation has been edited for length and clarity.

Can you tell me in simple terms what Pyrra does?

Pyrra is a threat intelligence company. We do three things: We collect content — publicly available information — from alternative social media sites. We use machine learning and advanced algorithms to detect violent threats, hate speech and disinformation that are popping up on these platforms. And then we display that information for our clients, either through reports or through a platform that we have.

Look at the Pew Research polls that are out there about how many people believe the core tenets of QAnon. I think we’ve entered a new phase in which social media has altered and warped how we encounter information, how we process it, how we internalise what counts as the truth. It’s having significant impacts on our democracy.

I really do believe that social media is an accelerator. …

An accelerator of societal disintegration?

Yes, yes, exactly.

You had an interesting Twitter thread about the disinformation you’re seeing around the January 6 committee. Can you tell me about that?

On these alternative social media platforms, the narrative about January 6 started getting pushed on January 7. People started by saying it was antifa that was responsible. That got amplified by more mainstream characters, even Tucker Carlson talked about antifa maybe having a role in January 6.

Right off the bat they were trying to deflect blame. You had card-carrying members of the MAGA community like [January 6 protester] Ray Epps getting falsely accused of being FBI informants and being responsible for pushing people into the Capitol. He came out and said, “I was one of them [the pro-Trump movement], and they just kind of turned on me.”

All it takes is a single user on one of these platforms to write something outlandish without any factual basis or evidence. They’re not citing anything, they’re not looking at any hardcore piece of information or they’re taking things out of context. And that just gets endlessly amplified by other users. People who are not sophisticated consumers of information see that on these platforms, and they go: “I agree with that. That sounds plausible. It’s now the truth for me.”

If you ask people, “Who was responsible for January 6?” significant numbers of people will tell you antifa had a role in January 6. Multiple credible investigations have shown that antifa had no role in January 6. … Yet this maintains a consistent narrative, and that narrative started spinning basically as soon as people were cleared from the Capitol building.

In the past that’s the kind of thing that would’ve happened on Twitter. But now it starts on the smaller platforms. It may eventually migrate to Twitter. But Twitter and the larger platforms actually do some content moderation, making it harder for this stuff to gain traction or get picked up.

These smaller places either don’t have the resources to do content moderation or don’t have the will to do it. They are allowing these narratives to fester and gain traction and eventually jump hosts.

Out of all the alternative social media apps and sites, which seem to be the most successful? Where is the energy?

It’s still 4chan. … One secret about 4chan is they actually have to do a significant amount of content moderation now — where they remove posts because of how bad and violent they are. There’s a massive amount of people on 4chan on a regular basis, who are frequent flyers on the boards. It’s still crazy there.

More than Telegram, an instant messenger service?

Telegram is also huge. Right now we track thousands of Telegram channels, but that’s just a drop in the bucket.

AC Thompson is a reporter for ProPublica and covers hate crimes and racial extremists

This article was first published in www.propublica.org