Please ensure Javascript is enabled for purposes of website accessibility
Study Shows AI Image-Generators Being Trained on Explicit Photos of Children
By admin
Published 1 year ago on
December 21, 2023

Share

Getting your Trinity Audio player ready...

Hidden inside the foundation of popular artificial intelligence image-generators are thousands of images of child sexual abuse, according to a new report that urges companies to take action to address a harmful flaw in the technology they built.

Those same images have made it easier for AI systems to produce realistic and explicit imagery of fake children as well as transform social media photos of fully clothed real teens into nudes, much to the alarm of schools and law enforcement around the world.

Until recently, anti-abuse researchers thought the only way that some unchecked AI tools produced abusive imagery of children was by essentially combining what they’ve learned from two separate buckets of online images — adult pornography and benign photos of kids.

But the Stanford Internet Observatory found more than 3,200 images of suspected child sexual abuse in the giant AI database LAION, an index of online images and captions that’s been used to train leading AI image-makers such as Stable Diffusion. The watchdog group based at Stanford University worked with the Canadian Centre for Child Protection and other anti-abuse charities to identify the illegal material and report the original photo links to law enforcement. It said roughly 1,000 of the images it found were externally validated.

The response was immediate. On the eve of the Wednesday release of the Stanford Internet Observatory’s report, LAION told The Associated Press it was temporarily removing its datasets.

LAION, which stands for the nonprofit Large-scale Artificial Intelligence Open Network, said in a statement that it “has a zero tolerance policy for illegal content and in an abundance of caution, we have taken down the LAION datasets to ensure they are safe before republishing them.”

While the images account for just a fraction of LAION’s index of some 5.8 billion images, the Stanford group says it is likely influencing the ability of AI tools to generate harmful outputs and reinforcing the prior abuse of real victims who appear multiple times.

Not an Easy Problem to Fix

It’s not an easy problem to fix, and traces back to many generative AI projects being “effectively rushed to market” and made widely accessible because the field is so competitive, said Stanford Internet Observatory’s chief technologist David Thiel, who authored the report.

“Taking an entire internet-wide scrape and making that dataset to train models is something that should have been confined to a research operation, if anything, and is not something that should have been open-sourced without a lot more rigorous attention,” Thiel said in an interview.

A prominent LAION user that helped shape the dataset’s development is London-based startup Stability AI, maker of the Stable Diffusion text-to-image models. New versions of Stable Diffusion have made it much harder to create harmful content, but an older version introduced last year — which Stability AI says it didn’t release — is still baked into other applications and tools and remains “the most popular model for generating explicit imagery,” according to the Stanford report.

“We can’t take that back. That model is in the hands of many people on their local machines,” said Lloyd Richardson, director of information technology at the Canadian Centre for Child Protection, which runs Canada’s hotline for reporting online sexual exploitation.

Stability AI on Wednesday said it only hosts filtered versions of Stable Diffusion and that “since taking over the exclusive development of Stable Diffusion, Stability AI has taken proactive steps to mitigate the risk of misuse.”

“Those filters remove unsafe content from reaching the models,” the company said in a prepared statement. “By removing that content before it ever reaches the model, we can help to prevent the model from generating unsafe content.”

LAION was the brainchild of a German researcher and teacher, Christoph Schuhmann, who told the AP earlier this year that part of the reason to make such a huge visual database publicly accessible was to ensure that the future of AI development isn’t controlled by a handful of powerful companies.

“It will be much safer and much more fair if we can democratize it so that the whole research community and the whole general public can benefit from it,” he said.

Much of LAION’s data comes from another source, Common Crawl, a repository of data constantly trawled from the open internet, but Common Crawl’s executive director, Rich Skrenta, said it was “incumbent on” LAION to scan and filter what it took before making use of it.

LAION said this week it developed “rigorous filters” to detect and remove illegal content before releasing its datasets and is still working to improve those filters. The Stanford report acknowledged LAION’s developers made some attempts to filter out “underage” explicit content but might have done a better job had they consulted earlier with child safety experts.

Many text-to-image generators are derived in some way from the LAION database, though it’s not always clear which ones. OpenAI, maker of DALL-E and ChatGPT, said it doesn’t use LAION and has fine-tuned its models to refuse requests for sexual content involving minors.

Google built its text-to-image Imagen model based on a LAION dataset but decided against making it public in 2022 after an audit of the database “uncovered a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes.”

Trying to clean up the data retroactively is difficult, so the Stanford Internet Observatory is calling for more drastic measures. One is for anyone who’s built training sets off of LAION‐5B — named for the more than 5 billion image-text pairs it contains — to “delete them or work with intermediaries to clean the material.” Another is to effectively make an older version of Stable Diffusion disappear from all but the darkest corners of the internet.

“Legitimate platforms can stop offering versions of it for download,” particularly if they are frequently used to generate abusive images and have no safeguards to block them, Thiel said.

As an example, Thiel called out CivitAI, a platform that’s favored by people making AI-generated pornography but which he said lacks safety measures to weigh it against making images of children. The report also calls on AI company Hugging Face, which distributes the training data for models, to implement better methods to report and remove links to abusive material.

Hugging Face said it is regularly working with regulators and child safety groups to identify and remove abusive material. Meanwhile, CivitAI said it has “strict policies” on the generation of images depicting children and has rolled out updates to provide more safeguards. The company also said it is working to ensure its policies are “adapting and growing” as the technology evolves.

The Stanford report also questions whether any photos of children — even the most benign — should be fed into AI systems without their family’s consent due to protections in the federal Children’s Online Privacy Protection Act.

Rebecca Portnoff, the director of data science at the anti-child sexual abuse organization Thorn, said her organization has conducted research that shows the prevalence of AI-generated images among abusers is small, but growing consistently.

Developers can mitigate these harms by making sure the datasets they use to develop AI models are clean of abuse materials. Portnoff said there are also opportunities to mitigate harmful uses down the line after models are already in circulation.

Tech companies and child safety groups currently assign videos and images a “hash” — unique digital signatures — to track and take down child abuse materials. According to Portnoff, the same concept can be applied to AI models that are being misused.

“It’s not currently happening,” she said. “But it’s something that in my opinion can and should be done.”

RELATED TOPICS:

DON'T MISS

What Are Fresno Real Estate Experts Predicting for 2025 and Beyond?

DON'T MISS

First California EV Mandates Hit Automakers This Year. Most Are Not Even Close

DON'T MISS

Hey PG&E Customers, Get Ready for New ‘Transaction Fees’

DON'T MISS

Fresno County Ending ‘Squaw Valley’ Fight After Latest Court Ruling

DON'T MISS

Exclusive: Tesla to Delay US Launch of Affordable EV, a Lower-Cost Model Y, Sources Say

DON'T MISS

Clovis Reconsiders Recycling Vote. Will a Campaign Contribution Matter?

DON'T MISS

Gov. Newsom Offers $50K Reward in 2022 Kings County Homicide

DON'T MISS

Trump’s White House Launches COVID Website That Criticizes WHO, Fauci and Biden

DON'T MISS

Fresno ‘Powers Up’ the Nation’s Largest Combined Solar and Battery Storage Project

DON'T MISS

Trump Admin Asserts COVID-19 Originated in Chinese Lab, Targets Fauci

DON'T MISS

Vendors Back at Fresno’s Art Hop? Survey Wants to Know What You Think

DON'T MISS

Russian Missile Attack Kills One, Wounds 112 in Ukraine’s Kharkiv, Officials Say

UP NEXT

Popular AIs Head-to-Head: OpenAI Beats DeepSeek on Sentence-Level Reasoning

UP NEXT

Al Sharpton Calls Meeting With Target’s CEO Amid DEI Backlash ‘Very Constructive and Candid’

UP NEXT

Average US Rate on a 30-Year Mortgage Climbs to 6.83%, Highest Level Since Late February

UP NEXT

Google Holds Illegal Monopolies in Ad Tech, US Judge Finds, Allowing US to Seek Breakup

UP NEXT

Temu and Shein Say They’re Raising Prices Due to Tariffs

UP NEXT

AI Action Figures Flood Social Media (Accessories Included)

UP NEXT

5 Easy Steps to Create Your Very Own “Starter Pack” Figurine Meme Image

UP NEXT

US Tariffs May Cost Chip Equipment Makers More Than $1 Billion, Industry Estimates

UP NEXT

At US Antitrust Trial, Meta’s Zuckerberg Admits He Bought Instagram Because It Was ‘Better’

UP NEXT

Former US Labor Officials Urge Contractors to Stand Firm on DEI

Clovis Reconsiders Recycling Vote. Will a Campaign Contribution Matter?

13 hours ago

Gov. Newsom Offers $50K Reward in 2022 Kings County Homicide

14 hours ago

Trump’s White House Launches COVID Website That Criticizes WHO, Fauci and Biden

14 hours ago

Fresno ‘Powers Up’ the Nation’s Largest Combined Solar and Battery Storage Project

15 hours ago

Trump Admin Asserts COVID-19 Originated in Chinese Lab, Targets Fauci

15 hours ago

Vendors Back at Fresno’s Art Hop? Survey Wants to Know What You Think

16 hours ago

Russian Missile Attack Kills One, Wounds 112 in Ukraine’s Kharkiv, Officials Say

16 hours ago

Iran Says Nuclear Deal Is Possible if Washington Is Realistic

16 hours ago

49ers Look to Strengthen Depleted Defense in NFL Draft

16 hours ago

Habit Burger & Grill Quietly Drops Impossible Burger From Menu

16 hours ago

Hey PG&E Customers, Get Ready for New ‘Transaction Fees’

Pacific Gas & Electric customers are already paying some of the nation’s highest rates for electricity, and their bills could be g...

12 hours ago

12 hours ago

Hey PG&E Customers, Get Ready for New ‘Transaction Fees’

13 hours ago

Fresno County Ending ‘Squaw Valley’ Fight After Latest Court Ruling

Tesla Inc. vehicle facility is pictured in Costa Mesa, California, U.S., November 1, 2023. (REUTERS/Mike Blake/File Photo)
13 hours ago

Exclusive: Tesla to Delay US Launch of Affordable EV, a Lower-Cost Model Y, Sources Say

13 hours ago

Clovis Reconsiders Recycling Vote. Will a Campaign Contribution Matter?

California Gov. Gavin Newsom speaks during a press conference in Los Angeles, Wednesday, Sept. 25, 2024. Newsom vetoed a landmark bill aimed at establishing first-in-the-nation safety measures for large artificial intelligence models Sunday, Sept. 29. (AP Photo/Eric Thayer, File)
14 hours ago

Gov. Newsom Offers $50K Reward in 2022 Kings County Homicide

The logo of the World Health Organization is seen at the WHO headquarters in Geneva, Switzerland, January 28, 2025. (REUTERS/Denis Balibouse/File Photo)
14 hours ago

Trump’s White House Launches COVID Website That Criticizes WHO, Fauci and Biden

15 hours ago

Fresno ‘Powers Up’ the Nation’s Largest Combined Solar and Battery Storage Project

15 hours ago

Trump Admin Asserts COVID-19 Originated in Chinese Lab, Targets Fauci

Help continue the work that gets you the news that matters most.

Search

Send this to a friend