Elon Musk’s xAI is in a legal trouble. California Attorney General Rob Bonta has issued a stern warning to the AI company by sending a cease-and-desist letter, demanding that its immediately prevent its Grok chatbot from generating sexualised deepfake images. Bonta has specifically targeted the creation of nonconsensual sexualized photos, with a priority focus on protecting minors. The letter sets a strict deadline for compliance for the company, warning that failure to act could result in significant legal consequences under California’s consumer protection and digital safety laws.
California Attorney General gives ‘January 20 deadline ’ to Elon Musk’s xAI
Bonta has given xAI a narrow window to demonstrate compliance, setting a deadline of January 20 at 5:00 PM PT “to address these issues.” Bonta highlighted an AI photo of two young girls in sexualized attire based on a user’s prompt, saying it violated ethical standards and potentially US laws on CSAM.
Read California AG’s full letter to Elon Musk’s xAI
RE: Cease and Desist DemandDear Mr. Musk: My office has opened an investigation into the proliferation of nonconsensual sexually explicit material produced using Grok, an AI model developed by xAI and integrated with X, which is itself owned by xAI. Our office demands that xAI take immediate action to stop the creation and distribution of deepfake nonconsensual intimate images (NCII) or child sexual abuse material (CSAM), including via the Grok website or Grok application or using Grok on X. Creation, distribution, publication, and exhibition of CSAM is a crime. (Cal. Penal Code section 311 et seq.) These business practices also violate California’s civil laws.Over the past weeks, there have been numerous examples of xAI taking ordinary, clothed images of women and children available on the internet and modifying those pictures, in response to user prompts, to depict the people in suggestive and sexually explicit scenarios and “undress” them, all without the subjects’ knowledge or consent. xAI has then published many of these images on X or otherwise made them available to the public. Perhaps most concerning, there are reports that the images xAI has created and distributed include CSAM. Indeed, in a post to the @Grok account, xAI stated that it “generated and shared an AI image of two young girls (estimated ages 12-16) in sexualized attire based on a user’s prompt. This violated ethical standards and potentially US laws on CSAM. xAI has also apparently attempted to limit image generation and editing via the Grok X account to paid premium subscribers of X. It is unclear what limits, if any, had been placed on image generation by Grok, but they were clearly insufficient, as recent public reports indicated that xAI continued to create and distribute NCII and potentially CSAM in response to user prompts. I appreciate your company’s recent announcement that it is implementing new guardrails regarding xAI’s creation and distribution of sexualized content. However, the scope and implementation of these new measures are unclear. In light of the seriousness of the incidents and concerns described above, as chief law enforcement officer of California, I demand that xAI immediately cease and desist from each of the following actions related to Grok and X: 1. Creating, disclosing, or publicizing digitized sexually explicit material portraying the depicted individual when the depicted individual did not consent to its creation or disclosure or was a minor when the material was created. “Digitized sexually explicit material” for these purposes is defined in California Civil Code section 1708.86 as any portion of a visual or audiovisual work created or substantially altered through digitization, including an image, that shows the depicted individual in the nude or appearing to engage in, or being subjected to, sexual conduct. 2. Facilitating or aiding and abetting the creation, disclosure, or publication of digitized sexually explicit material portraying the depicted individual when the depicted individual did not consent to its creation or disclosure or was a minor when the material was created, including in violation of Civil Code section 1708.86. 3. Creating, facilitating, or aiding and abetting the creation, distribution, or publication of any image, including but not limited to digitally altered or artificial-intelligence-generated matter, that involves or depicts a person under 18 years of age or what appears to be a person under 18 years of age engaging in or simulating sexual conduct. “Sexual conduct” for these purposes is defined in California Penal Code section 311.4. Any such actions violate California law, including without limitation, California Civil Code section 1708.86, California Penal Code sections 311 et seq. and 647(j)(4), and California Business & Professions Code section 17200. I expect xAI to take immediate action to address these issues. I also expect that xAI will provide confirmation to my office that it has stopped the creation and distribution of the problematic material and of the steps it is taking to address these issues no later than January 20, 2026 at 5 pm Pacific Time. In addition, xAI, Grok, and X should take immediate action to preserve all potentially relevant documents and information and prevent the deletion, alteration, or spoliation of any related prompts, images, posts, or other evidence, including electronically stored information.Sincerely,ROB BONTAAttorney General
About the Author
