California’s top law enforcement official on Friday ordered Elon Musk’s artificial intelligence company xAI to stop the creation and distribution of unauthorized sexually explicit digital images, warning that they could be in violation of the law.
Attorney General Rob Bonta sent a formal cease-and-desist notice to the company, citing a surge in complaints about its Grok chatbot creating suggestive or explicit images without consent.
The directive emphasizes content that shows women and minors in compromising situations, which Bonta’s team views as possibly unlawful.
“The avalanche of reports detailing this material—at times depicting women and children engaged in sexual activity—is shocking and, as my office has determined, potentially illegal,” Bonta said in a statement.
Bonta’s office said that realistic images of children engaged in sexual activity appear to be happening on a large scale.
More than 10,000 images generated by xAI between Christmas and New Year’s show people wearing little clothing, and some of them appear to be children, Bonta’s office said, citing an analysis.
The attorney general stressed that producing visuals resembling child exploitation is a felony. He said his team anticipates swift compliance from xAI.
“Today, I sent xAI a cease and desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material,” he said. “The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for child sexual abuse material.”
xAI did not immediately return a request for comment from The Epoch Times.
In response to concerns from global regulators over unauthorized sexually explicit digital images, xAI said late on Wednesday it imposed restrictions on all users of its Grok AI chatbot that limit image editing.
In addition, image creation and editing through the Grok account would now be restricted to paid subscribers only. X described the paywall as an added safeguard that would provide an “extra layer of protection.”
“We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis,” the company said in an X post.
”This restriction applies to all users, including paid subscribers.”
Bonta questioned the effectiveness of these measures, noting unclear impacts on overall content generation.
Launched as part of the X social network, formerly Twitter, the tool allows users to alter real photos into revealing or degrading forms. Although xAI has curtailed Grok’s ability to publicly post highly realistic altered visuals, tests indicate it still produces them for individual requests.
Bonta’s office said that xAI developed Grok’s image generation models to include what the company calls a “spicy mode,” which generates explicit content.
The office pointed to California’s recent legislation targeting fabricated intimate media. The law prohibits creating or spreading such material.
The cease and desist letter comes less than a week after Bonta announced an investigation into the company for Grok’s role in distributing unauthorized explicit images.
There have been numerous reports of users using Grok to transform everyday clothed pictures of people into suggestive scenarios, often without their awareness, according to the statement from the attorney general’s office.
The controversy extends beyond California. Authorities in Japan, the United Kingdom, Canada, Malaysia, and Indonesia have initiated reviews or threatened legal steps against xAI and X for similar issues.
For instance, Japanese regulators recently flagged Grok for producing sexually explicit AI images of people, including those appearing to be children.
Experts warn that AI-driven fakes pose escalating risks, from personal harassment to broader societal harms such as extortion or misinformation.
Evgenia Filimianova contributed to this report.
