xAI Faces Class Actions Over Grok-Generated Child Sexual Abuse Imagery

IT之家 November 11 news, according to Bloomberg, Jane Doe had not thought about her middle school graduation ceremony for years, until a law enforcement investigator pulled out a photo from the celebration and began asking her about it.

In the photo, she was only 14 at the time, wearing a yellow floral dress, smiling, with her arms around the shoulders of several friends. Some of those friends she rarely keeps in touch with now.

The investigator told her that the adult brother of one of her classmates had used Elon Musk's AI chatbot Grok to alter this otherwise ordinary photo into child sexual abuse material (CSAM). In the generated images, Jane Doe, then in the eighth grade, appeared completely nude in sexually provocative poses. These images were subsequently published on the internet.

She is participating in a class-action lawsuit against Musk's generative AI company xAI, the developer of Grok, under the name Jane Doe 5.

Jane Doe 5 wrote in a complaint: "It is currently impossible to determine how many other child sex offenders may already possess child sexual abuse images of Jane Doe 5, nor how widely these images have spread online through dark web channels and applications."

Jane Doe, now 18, said in an interview just days before leaving her hometown to attend university: "These images are already out there, and there is really nothing I can do. That is what frightens me the most."

She also said: "This makes me feel like I want to give up on everything."

Today, with AI, anyone can easily obtain ordinary photos from personal social media accounts or school websites and turn them into unimaginable pornographic images within seconds. Jane Doe 5 is just one of a growing number of young victims trying to hold xAI legally accountable, believing that Grok helped users create these disturbing images.

In doing so, they are facing not only a company backed by the world's richest person, but also a fundamental question in the AI era that remains unresolved: when a chatbot generates content that harms others, who should ultimately bear legal responsibility? Is it the user who issued the instruction to the AI, or the company that developed the technology? Or should both bear responsibility?

xAI states that, like competitors such as OpenAI, Anthropic, and Meta, the company strictly prohibits users from using its chatbot to create such images and has implemented technical measures to prevent users from generating related content. This type of problem has now become a common challenge facing the entire AI industry.

However, since Grok's release in 2023, Musk has deliberately positioned it as a less restricted and more controversial product than its competitors.

He has encouraged users to use the chatbot to generate pornographic AI content. Grok specifically launched a mode called "Spicy," featuring "bolder, more visually striking narratives" and supporting the generation of not-safe-for-work (NSFW) images and videos.

xAI has stated that the "Spicy" and "Unhinged" modes align with Grok's product goal of pursuing "maximum truth-seeking."

"These images are already out there, and there is really nothing I can do."

Experts believe that this product positioning has made Grok especially popular among users who want to break internet rules.

Earlier this year, users on the X platform at one point used Grok to generate thousands of sexually suggestive images or images with clothing removed from people every hour. A third-party analysis found that the vast majority of visual content generated by Grok fell into this category.

Other investigations also found that many of these images appeared to involve minors. Some reports estimated that Grok had allowed users to generate tens of thousands of realistic sexualized images of children.

The massive proliferation of such content on the X platform sparked strong backlash worldwide. Under public pressure, X made Grok's image generation feature exclusive to paid users in January this year. But critics argue that this move merely turned a highly controversial feature into a paid product that people are eager to try.

Musk, meanwhile, blamed Grok's users rather than the product itself for the creation of explicit sexual content.

In January this year, he said he had "not heard of any nude images involving minors." xAI subsequently stated that it had disabled Grok's ability to generate non-consensual sexualized images of any real person.

For interview requests for this article, xAI declined to comment, citing ongoing litigation.

In the spring of this year, Wifredo Fernández, xAI's head of global government affairs for the Americas, said: "xAI has zero tolerance for child sexual exploitation, and we are determined to make our products a tool that those who seek to harm minors cannot exploit."

In September this year, X stated that it would report such content to the nonprofit National Center for Missing & Exploited Children (NCMEC), which would refer relevant leads to law enforcement.

Over the past few months, xAI has also sued multiple individuals suspected of abusing Grok and endangering child safety.

But at the same time, the company is also actively challenging laws designed to protect the public from non-consensual AI pornographic images.

This summer, xAI sued the U.S. state of Minnesota over a state law targeting AI companies that use technology to process photos of people into nude images. Under the state law, AI companies that allow users to process such images may face penalties.

In its lawsuit, xAI argued that the law violates the First Amendment of the U.S. Constitution because its scope "far exceeds the legislative purpose itself, exposing a large amount of constitutionally protected speech to civil liability and government sanctions."

In October this year, James Burnham, general counsel of xAI and X, said that Minnesota's law is "clearly unconstitutional."

He also said: "Under Elon Musk's leadership, we will always defend freedom of speech."

A Last Resort for Victims

For these victims seeking a sense of safety or psychological comfort, the courts are gradually becoming their last resort.

Those suspected of abusing Grok are a diverse group, including professional photographers, trusted friends, classmates, and even family members. Some victims are as young as 6 years old.

In March this year, several minors from the U.S. state of Tennessee sued xAI, alleging that someone used Grok to remove their clothing in digital images or composite them into explicit sexual poses.

Later, the complaint was amended to add more young victims, including Jane Doe 5 from Wisconsin.

One plaintiff from Tennessee said someone used a photo from her 2025 graduation yearbook to create pornographic images. Another plaintiff said the person used a personal photo she had previously posted on Instagram, in which she was wearing a blue bikini while on a beach vacation.

One plaintiff from Wyoming said her stepfather uploaded a completely non-sexual childhood photo of her to Grok. In the photo, she was wearing a T-shirt with a panda design. Her stepfather then used AI to generate thousands of distressing synthetic images, some of which even depicted this family member raping her or performing oral sex on her.

This summer, multiple minors and their parents from Arkansas also sued xAI on similar grounds.

On the other side of the United States, in California, a survivor of childhood sexual abuse filed a nationwide class action lawsuit against xAI in August of this year.

She alleged in the complaint that Grok's training data partly used images of her being raped during her preschool years. xAI used that data through Grok to generate a large amount of horrifying new content modeled on her, and used it as a basis to further create AI pornographic doctored images of other victims.

The complaint seeks a jury trial and asserts claims on behalf of "all persons in the United States who, while minors, had their images or videos altered by Grok and used to create child sexual abuse material." Lawyers estimate that the number of eligible victims could be as many as several thousand.

Court filings show that because it is impossible to track down and completely remove these distressing images from the internet, the young victims can only continue to bear the long-term harm caused by them.

Several plaintiffs from Tennessee are now even afraid to attend their own graduation ceremonies. They worry that college admissions officers might see these fake images online and mistakenly believe the content is real. They also fear they will be stalked and harassed.

The victims from Wisconsin and Wyoming have developed persistent mental health problems, including anxiety, depression, and even suicidal thoughts.

Annika Martin, a lawyer at Lieff Cabraser Heimann & Bernstein, the law firm representing multiple victims, said: "You can be harmed from any unexpected direction."

She further asked: "What does this mean for today's children? Will they still be able to safely take and share photos that document the normal stages of growing up?"

She added that the harm is not limited to young women, especially when the chatbot users themselves are also minors.

Minors may not know that using AI to create pornographic images of their peers may actually constitute a federal crime.

Martin said: "In the past, this might have been just a boy drawing breasts on a classmate's yearbook photo as a prank. Now, he has a hand grenade he can throw."

Despite the growing number of lawsuits, there have so far been no consequences that have had a substantial impact on the company.

Earlier this year, bipartisan state attorneys general from 35 U.S. states jointly sent a letter to xAI, demanding that the company take more measures to address "the harms caused by this technology."

In addition, in multiple important markets, state governments, the federal government, and regulators in other countries and regions are also investigating the company, focusing on whether it failed to adequately regulate child sexual exploitation content on its platform or failed to report it to authorities in a timely manner.

Regulatory scrutiny of Grok has already led some countries to temporarily ban the service or restrict its product features, but so far the investigations have not resulted in major fines or substantive regulatory penalties.

"What does this mean for today's children? Will they still be able to safely take and share photos that document the normal stages of growing up?"

Part of the reason for this situation is that the development of AI technology has outpaced the regulatory process across the board. This has not only raised a series of important policy questions, but also involves a high-stakes global race for technological innovation, with AI at its core.

Musk has unparalleled wealth, and his business empire also includes a SpaceX IPO plan that could set a historical record. These factors also give him enough resources to withstand legal and regulatory shocks.

Unless SpaceX loses lucrative government contracts in the United States or overseas because of these issues, investors do not seem very worried about the related risks.

Luke Rahbari, CEO of investment advisory firm Equity Armor Investments, believes that a recent settlement between competitor Meta and state attorneys general can serve as a reference for gauging how potential litigation or regulatory action could affect SpaceX.

In August of this year, Meta reached an $18 billion settlement with U.S. state attorneys general over alleged violations of child safety protection rules (IT Home note: at current exchange rates, approximately RMB 120.38 billion).

Rahbari called the settlement amount "a drop in the bucket." He said: "If you look at it by that standard or metric, the ultimate impact will be very limited."

New Problems, Old Laws

Generative AI may be a new technology, but the core laws involved in these lawsuits have existed for decades.

Multiple plaintiffs are basing their claims on a federal law called "Masha's Law." This law, which has been in effect for decades, allows victims of child sexual exploitation to seek financial compensation and other legal remedies from those who harmed them.

Section 230 of the U.S. Communications Decency Act may also play an important role in these cases.

This controversial law, also decades old, mainly provides that online platforms are generally not legally liable for content posted or uploaded by users, and that liability mainly rests with the content publishers. It is precisely this provision that has allowed sites such as YouTube, Facebook, and X to largely avoid legal liability for user-posted content and achieve rapid growth.

Musk has consistently emphasized that responsibility should rest with the users of the product, not the product itself.

He once said: "Grok does not spontaneously generate images; it only generates them when users ask."

But Martin, the lawyer representing the victims, argues that these cases are different in nature, and that Section 230 of the Communications Decency Act does not exempt xAI from liability for illegal content generated by Grok itself.

Martin pointed out: "The content users upload is ordinary photos of people fully clothed and completely harmless, but the model transforms these photos into child sexual abuse material. Without Grok, these people would never have been able to create the images and videos that now exist."

Once, Jane Doe regarded AI only as an occasional tool to help with homework. She said she used to be outgoing and loved sports.

But in the later years of high school, she spent most of her time at home, gradually drifted away from friends, and kept this nightmare deep inside.

She said: "This has taken over my entire mind."

Now, she has begun receiving psychological therapy, the first time in her life she has sought professional mental health help. She is also trying to adapt to college life, but the related anxiety has not disappeared.

Investigators may notify her each time these tampered images reappear online. She is both terrified of this possibility and unwilling to turn a blind eye to it.

Jane Doe said: "What I most want to do is draw more attention to this issue, so that people know this is happening and that anyone could become a victim."

Faced with the possibility of going to court against Musk and his massive legal team, she has not backed down. She said: "Grok must be held accountable for allowing these things to happen."