INSUBCONTINENT EXCLUSIVE:
Image search engine Giphy bills itself as providing a &fun and safe way& to search and create animated GIFs
But despite its ban on illicit content, the site is littered with self-harm and child sex abuse imagery, TechCrunch has learned.
A new
report from Israeli online child protection startup L1ght — previously AntiToxin Technologies — has uncovered a host of toxic content
hiding within the popular GIF-sharing community, including illegal child abuse content, depictions of rape and other toxic imagery
associated with topics like white supremacy and hate speech
The report, shared exclusively with TechCrunch, also showed content encouraging viewers into unhealthy weight loss and glamorizing eating
disorders.
TechCrunch verified some of the company findings by searching the site using certain keywords
(We did not search for terms that may have returned child sex abuse content, as doing so would be illegal.) Although Giphy blocks many
hashtags and search terms from returning results, search engines like Google and Bing still cache images with certain keywords.
When we
tested using several words associated with illicit content, Giphy sometimes showed content from its own results
When it didn&t return any banned materials, search engines often returned a stream of would-be banned results.
L1ght develops advanced
solutions to combat online toxicity
Through its tests, one search of illicit material returned 195 pictures on the first search page alone
L1ght team then followed tags from one item to the next, uncovering networks of illegal or toxic content along the way
The tags themselves were often innocuous in order to help users escape detection, but they served as a gateway to the toxic
material.
Despite a ban on self-harm content, researchers found numerous keywords and search terms to find the banned content
We have blurred this graphic image
(Image: TechCrunch)
Many of the more extreme content — including images of child sex abuse — are said to have been tagged using keywords
associated with known child exploitation sites.
We are not publishing the hashtags, search terms or sites used to access the content, but we
passed on the information to the National Center for Missing and Exploited Children, a national nonprofit established by Congress to fight
child exploitation.
Simon Gibson, Giphy head of audience, told TechCrunch that content safety was of the &utmost importance& to the company
and that it employs &extensive moderation protocols.& He said that when illegal content is identified, the company works with the
authorities to report and remove it.
He also expressed frustration that L1ght had not contacted Giphy with the allegations first
L1ght said that Giphy is already aware of its content moderation problems.
Gibson said Giphy moderation system &leverages a combination of
imaging technologies and human validation,& which involves users having to &apply for verification in order for their content to appear in
our searchable index.& Content is &then reviewed by a crowdsourced group of human moderators,& he said
&If a consensus for rating among moderators is not met, or if there is low confidence in the moderator decision, the content is escalated to
Giphy internal trust and safety team for additional review,& he said.
&Giphy also conducts proactive keyword searches, within and outside of
our search index, in order to find and remove content that is against our policies,& said Gibson.
L1ght researchers used their proprietary
artificial intelligence engine to uncover illegal and other offensive content
Using that platform, the researchers can find other related content, allowing them to find vast caches of illegal or banned content that
would otherwise and for the most part go unseen.
This sort of toxic content plagues online platforms, but algorithms only play a part
More tech companies are finding human moderation is critical to keeping their sites clean
But much of the focus to date has been on the larger players in the space, like Facebook, Instagram, YouTube and Twitter.
Facebook, for
example, has been routinely criticized for outsourcing moderation to teams of lowly paid contractors who often struggle to cope with the
sorts of things they have to watch, even experiencing post-traumatic stress-like symptoms as a result of their work
Meanwhile, Google YouTube this year was found to have become a haven for online sex abuse rings, where criminals had used the comments
section to guide one another to other videos to watch while making predatory remarks.
Giphy and other smaller platforms have largely stayed
out of the limelight, during the past several years
But L1ght new findings indicate that no platform is immune to these sorts of problems.
L1ght says the Giphy users sharing this sort of
content would make their accounts private so they wouldn&t be easily searchable by outsiders or the company itself
But even in the case of private accounts, the abusive content was being indexed by some search engines, like Google, Bing and Yandex, which
The firm also discovered that pedophiles were using Giphy as the means of spreading their materials online, including communicating with
each other and exchanging materials
And they weren&t just using Giphy tagging system to communicate — they were also using more advanced techniques like tags placed on images
through text overlays.
This same process was utilized in other communities, including those associated with white supremacy, bullying, child
abuse and more.
This isn&t the first time Giphy has faced criticism for content on its site
Last year a report by The Verge described the company struggles to fend off illegal and banned content
Last year the company was booted from Instagram for letting through racist content.
Giphy is far from alone, but it is the latest example of
companies not getting it right
Earlier this year and following a tip, TechCrunch commissioned then-AntiToxin to investigate the child sex abuse imagery problem on
Microsoft search engine Bing
Under close supervision by the Israeli authorities, the company found dozens of illegal images in the results from searching certain
When The New York Times followed up on TechCrunch report last week, its reporters found Bing had done little in the months that had passed
to prevent child sex abuse content appearing in its search results.
It was a damning rebuke on the company efforts to combat child abuse in
its search results, despite pioneering its PhotoDNA photo detection tool, which the software giant built a decade ago to identify illegal
images based off a huge database of hashes of known child abuse content.
Giphy Gibson said the company was &recently approved& to use
Microsoft PhotoDNA but did not say if it was currently in use.
Where some of the richest, largest and most-resourced tech companies are
failing to preemptively limit their platforms& exposure to illegal content, startups are filling in the content moderation gaps.
L1ght,
which has a commercial interest in this space, was founded a year ago to help combat online predators, bullying, hate speech, scams and
more.
The company was started by former Amobee chief executive Zohar Levkovitz and cybersecurity expert Ron Porat, previously the founder of
ad-blocker Shine, after Porat own son experienced online abuse in the online game Minecraft
The company realized the problem with these platforms was something that had outgrown users& own ability to protect themselves, and that
technology needed to come to their aid.
L1ght business involves deploying its technology in similar ways as it has done here with Giphy —
in order to identify, analyze and predict online toxicity with near real-time accuracy.
Microsoft Bing not only shows child sexual abuse,
