Please ensure Javascript is enabled for purposes of website accessibility

Google Is Using AI to Combat Internet Trolls

By Danny Vena - Updated Apr 11, 2017 at 2:15PM

You’re reading a free article with opinions that may differ from The Motley Fool’s Premium Investing Services. Become a Motley Fool member today to get instant access to our top analyst recommendations, in-depth research, investing resources, and more. Learn More

Social media and other sites are frequently under pressure to curb abusive members. AI just might provide the key.

One of the biggest challenges facing websites, news organizations, and social media is online abuse and harassment. Since the dawn of the internet, sites have battled against abusive, toxic, and inappropriate comments posted in their online forums. With the millions of comments posted, news organization quickly became overwhelmed by the magnitude of data produced.

Websites and publishers have become so concerned by the volume of vitriolic comments and fear of lawsuits that many have removed the ability to post comments on their sites. Consider that 72% of internet users in America have witnessed online harassment, and nearly half have experienced it themselves. Social media sites have been under fire to control hate speech and online abuse by their members. Technological advancements in artificial intelligence (AI) may finally provide the answer.

Hacker in hoodie with stylized binary code.

Google uses AI to battle abusive content. Image source: Pixabay. 

An important piece of the puzzle

Jigsaw, a technology incubator of Google parent Alphabet (GOOGL -1.81%) (GOOG -1.62%), has developed an artificial neural network -- an AI system that replicates the structure and learning capacity of the human brain using algorithms and software models. Using this technology, it aims to identify and control abusive online comments. Google and Jigsaw are making the program available free of charge, and it's being added to the Google's TensorFlow library and Cloud Machine Learning Platform. The product, dubbed Perspective, uses deep learning to sift through reams of data to detect harassment, insults, and abusive speech in online forums in real time. Jigsaw explains how the program works:

Perspective scores comments based on the perceived impact a comment might have on a conversation, which publishers can use to give real-time feedback to commenters, help moderators sort comments more effectively, or allow readers to more easily find relevant information. We'll be releasing more machine learning models later in the year, but our first model identifies whether a comment could be perceived as "toxic" to a discussion. 

Do you kiss your mother with that mouth?

Google and Jigsaw used comment data from The New York Times Company (NYT -0.34%), Wikipedia, and several unnamed partners. They then showed that data to panels of people and had them rate whether the comments were toxic. They used these human responses as training data for the AI system, which will provide ratings of a phrase based on its toxicity on a scale of 100 and allows you to test the system. Type in the phrase "you are ignoring important information" rates a 10%, "your mother wears combat boots" garnered 55%, while "you're a jerk" gets a response of 86% toxicity.

The New York Times reports that it only has the resources to allow comments on 10% of its articles. The company provided the archives of comments in hopes of expanding its comments section and to "increase the speed at which comments are reviewed." With only 14 moderators to manually review every comment, the task became overwhelming, reviewing on average 11,000 comments daily.

The New York Times building.

The New York Times joins Google to curb online abuse. Image source: Pixabay.

Twitter tries to clean up its act

Google is not the only company seeking to curb online bullying using AI. At IBM's (IBM 0.52%) InterConnect conference last month, Twitter, Inc.'s (TWTR -0.56%) vice-president of data strategy Chris Moody announced that the popular social network had partnered with IBM's Watson, the AI-based cognitive computer, to address online abuse. He stated:

We're starting just now to partner with the Watson team. Watson is really good at understanding nuances in language and intention. What we want to do is be able to identify abuse patterns early and stop this behavior before it starts.

This comes at a crucial time for Twitter, which has been under fire for not policing its users. The company announced last month that it was working to make the site a safer place by limiting abusive users' ability to create new accounts and updating how users can report abusive tweets. 

That didn't end well

AI systems have been tested on social media before, though the results have been less than stellar. In early 2016, the folks at Microsoft (MSFT -1.05%) research had used Twitter as a testing ground for an AI-based chatbot, @tayandyou, aka TayTweets, to learn the speech patterns of millennials by interacting with them on the site. Unfortunately, within 24 hours and 96,000 tweets, the experiment was suspended when the fledgling AI began spewing venomous vitriol. Microsoft later said:

The AI chatbot Tay is a machine learning project, designed for human engagement. As it learns, some of its responses are inappropriate and indicative of the types of interactions some people are having with it. We're making some adjustments to Tay. 

Final thoughts

Each new technological innovation brings benefits and challenges. The dawn of the internet age brought with it internet trolls, who sought to control the conversation and silence dissenting voices. Programs like Perspective and Watson seek to return voices to those vulnerable speakers, which benefits us all.

Invest Smarter with The Motley Fool

Join Over 1 Million Premium Members Receiving…

  • New Stock Picks Each Month
  • Detailed Analysis of Companies
  • Model Portfolios
  • Live Streaming During Market Hours
  • And Much More
Get Started Now

Stocks Mentioned

Alphabet Inc. Stock Quote
Alphabet Inc.
$2,316.67 (-1.81%) $-42.83
Alphabet Inc. Stock Quote
Alphabet Inc.
$2,332.45 (-1.62%) $-38.31
International Business Machines Corporation Stock Quote
International Business Machines Corporation
$142.80 (0.52%) $0.74
Microsoft Corporation Stock Quote
Microsoft Corporation
$264.89 (-1.05%) $-2.81
The New York Times Company Stock Quote
The New York Times Company
$29.56 (-0.34%) $0.10
Twitter, Inc. Stock Quote
Twitter, Inc.
$39.19 (-0.56%) $0.22

*Average returns of all recommendations since inception. Cost basis and return based on previous market day close.

Related Articles

Motley Fool Returns

Motley Fool Stock Advisor

Market-beating stocks from our award-winning analyst team.

Stock Advisor Returns
S&P 500 Returns

Calculated by average return of all stock recommendations since inception of the Stock Advisor service in February of 2002. Returns as of 06/28/2022.

Discounted offers are only available to new members. Stock Advisor list price is $199 per year.

Premium Investing Services

Invest better with The Motley Fool. Get stock recommendations, portfolio guidance, and more from The Motley Fool's premium services.