Connect with top gaming leaders in Los Angeles at GamesBeat Summit 2023 this May 22-23. Register here.
Online gaming has boomed over the last decade and gaming has become more social. Voice chat is the preferred way to communicate in social games, but it’s also the most toxic medium.
That’s one of the conclusions in a new report from Speechly, a Finnish startup that uses machine learning to help stop toxic speech in gaming voice chat.
Speechly commissioned Voicebot research to do a survey of over 1,000 U.S. online gamers, and it found that voice chat is widely used today and liked by gamers. But they also say it is the worst channel for toxicity in terms of reach, frequency, and severity.
And it doesn’t seem like it will end soon, considering about two-thirds of toxic behavior victims in online game voice chat have never reported an incident, and those that have didn’t report every incident. The problem is much larger than previously reported, said Otto Söderlund, CEO of Speechly in an interview with GamesBeat.
GamesBeat Summit 2023
Join the GamesBeat community in Los Angeles this May 22-23. You’ll hear from the brightest minds within the gaming industry to share their updates on the latest developments.
“According to this research, two-thirds of gamers actually use voice chat for games. I thought that voice chat was a bit earlier in the adoption curve. But it seems it actually is actually pretty far along in the adoption curve,” Söderlund said.
“It wasn’t surprising that there are massive amounts of toxicity,” said Söderlund. “About 70% of voice chat users say that they have experienced a toxic event. That represents an industry-wide massive challenge.”
Söderlund said the toxicity profiles are continuous and repetitive, with gamers saying the average number they have experienced is over three incidents. That means it’s not random and is more structural.
“Those who file reports don’t actually file every incident,” Söderlund said.
The survey was done in December and it has a margin of error of about 5%. Overall, the work took about three months to complete.
Bringing AI and machine learning to the table
Companies like Modulate, Speechly and Spectrum Labs are all using AI to attack the problem through automation. But their solutions have to be tuned for the type of game and player, as children’s games have a much lower tolerance for toxicity than mature-rated games like Call of Duty.
In addition, there are insidious problems that make the detection of toxic behavior difficult. False accusations of toxic behavior in voice chat are a known problem in games where the victims get harassed again, and few game companies have a way to verify who is telling the truth.
The types of toxicity
Sexual harassment is among the highest in terms of types of toxicity. About 15.9% of users have experienced it. On the other hand, offensive name-calling was experienced by 39.9%, and trolling by 38.4%. Bullying was about 29.9%.
As for using AI effectively to enforce different rules for different games, Söderlund said, “AI can absolutely deal with that. Every single platform has its own rules that need to be enforced, there are differences, and every different title has a different culture. You can process all the conversations via AI and you will get real-time visibility into everything that happens.”
Gamers favor solutions ranging from proactive monitoring and recording audio for verifying incidents to simple one-click reporting.
Solving toxicity is a big challenge as more companies plan to move us all online with ideas like the metaverse. It’s not a trivial amount of data to sift through. There are millions of players playing a game each day for hours at a time. That adds up to tens of millions of audio data generated each day.
Söderlund agreed that one of the common approaches to dealing with toxicity is good. Rather than ban people for every report, the moderators can keep track of those who have reports filed about their behavior and then give them a reputation score. That score can be used over time whether to take action against them, such as banning them or simply telling them what isn’t acceptable.
“It’s also pretty clear that most of the toxic behavior comes from a very small percentage of the gamers,” Söderlund said. “It’s a small population that creates the biggest problems on the volume. Using reputation scores helps to detect really problematic people.”
Söderlund’s company has assembled a team of machine learning experts to use speech recognition AI to detect toxic speech and deal with it. The team has 14 people, and its investors include Y Combinator.
“We’ve pretty much been working on natural language processing problems for the last decade, solving some of the toughest problems,” he said.
The startup got going in Helsinki a couple of years ago, figured out how to do speech recognition cost-effectively, focused on gaming more recently, and now its tech is being used in some of the world’s largest companies in pilot tests for moderating speech.
But I can’t say I’m as optimistic as Söderlund that technology will solve this human problem.
GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it. Discover our Briefings.