Grok's deepfake problem, and what it changed for everyone else

Between 31 December 2025 and 27 July 2026, one image feature drew orders from three governments and the European Commission, a state investigation and a federal class action, across a series of separate announcements. We carry two Grok models in our own picker, so every entry below is dated and attached to the regulator, the court or the company that put it on the record, and quoted words belong to whoever said them.

If you are here because someone has made images of you, the practical part is near the bottom.

The timeline, in order

31 December 2025. The detection firm Copyleaks published an observational review of Grok's public photo tab on X and reported "a conservative rate of roughly one nonconsensual sexualized image per minute in the observed image stream".
2 January 2026. India's IT ministry ordered X to take corrective action on Grok, including restricting generation of content involving "nudity, sexualization, sexually explicit, or otherwise unlawful" material, and gave the platform 72 hours to submit an action-taken report. The order, which TechCrunch reviewed, warned that non-compliance could jeopardise X's safe harbour protections there.
3 January 2026. X's own Safety account posted the company's position: "Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content."
5 and 8 January 2026. The UK's Ofcom said on the 5th that it was in touch with xAI and would "undertake a swift assessment to determine whether there are potential compliance issues that warrant investigation." On the 8th, TechCrunch reported that the European Commission had ordered xAI to retain all documents relating to Grok.
9 January 2026. X restricted Grok's image generation on the platform to paying subscribers. At the time of that report the limit did not apply to the separate Grok app, which was still letting anyone generate pictures for free.
10 and 11 January 2026. Indonesia's Komdigi ministry paused access to Grok on the 10th, and Malaysia's regulator ordered temporary restrictions on the 11th. The Malaysian Communications and Multimedia Commission said X's replies had "relied primarily on user-initiated reporting mechanisms and failed to address the inherent risks posed by the design and operation of the AI tool." Indonesia's minister Meutya Hafid said the government "views nonconsensual sexual deepfakes as a serious violation of human rights, dignity, and citizens' security in the digital space."
14 January 2026. California's Attorney General opened an investigation into xAI. Rob Bonta: "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking."
16 January 2026. The same office sent xAI a cease and desist letter, naming California Civil Code section 1708.86, Penal Code sections 311 and 647(j)(4), and Business and Professions Code section 17200, and gave the company five days to respond. Bonta: "I fully expect xAI to immediately comply."
1 February 2026. Indonesia began processing a conditional restoration of access, according to Antara, the country's state news agency, after X Corp set out measures in a letter to the minister, including stronger technical safeguards and restricted access to certain features.
16 March 2026. Three Tennessee teenagers filed a proposed class action against xAI in the Northern District of California, on the docket as Doe 1 v. X.AI Corp., 5:26-cv-02246. The complaint alleges that the model behind Grok powered a third-party app used to make explicit images and videos of them as minors, and that xAI licensed its technology to app makers so it "could attempt to outsource the liability of their incredibly dangerous tool".
7 July 2026. An amended complaint added two more plaintiffs and named Stability AI as a defendant, alleging that the app on the perpetrator's phone "relied on Stability AI's image-producing tools", as NPR reported. The case remains open and nothing in it has been decided. xAI was renamed SpaceXAI that month, after merging into SpaceX.

What X actually changed

Two things are on the record and they are not the same size. The company restated that illegal content made with Grok would be treated like illegal content uploaded to X, and it moved image generation on the platform behind a paid subscription. A paywall changes who can press the button, and says nothing about what the tool produces for the people who do pay. None of the sources above describe a model-level fix.

Watch the wording when a platform says it has "restricted" a generation feature. It may have restricted the audience while leaving the output alone, and both land in the same sentence of the same news story. They protect different people.

The rules that followed

17 July 2026. San Francisco ordered Apple and Google to remove dozens of nudify apps from their stores. City Attorney David Chiu, in a statement to TechCrunch: "Apple and Google are profiting off apps that exploit women and girls by generating nonconsensual intimate deepfakes." Apple said it had removed three of the apps; Google said the five Play apps named in the letter were suspended.
27 July 2026. The AI Omnibus entered into force across the EU. Most of it is simplification and extended deadlines, but one line is a new prohibition, stated plainly in the Commission's summary: a "Ban on nudification apps" that "Prohibits AI systems that generate non-consensual sexually explicit and intimate content or child sexual abuse material".

Neither is about Grok. Both took the shape they did because of what happened on one platform in January 2026.

If someone makes images of you

These are procedures rather than legal advice, and the two jurisdictions work differently. Do the platform step in both, because it is what gets the content down soonest.

In the EU

Report it on the platform first. Under the Digital Services Act you can flag illegal content through a mechanism on the service itself, and the Commission's own summary of your rights says platforms "must respond to your reports and offer options to appeal their decisions".
Keep your own record before anything disappears: the URL, the account, the date and time, a screenshot. Appeals ask for exactly this.
If the platform does not act, escalate to your country's Digital Services Coordinator, the national regulator that supervises platforms under the DSA. The Commission maintains the list. In Belgium it is the Belgian Institute for Postal Services and Telecommunications.
Report it to your local police as well if the subject was a minor, or if the material came with threats or extortion.

In the US

Use the platform's notice and removal process. The TAKE IT DOWN Act, Public Law 119-12, approved 19 May 2025, requires covered platforms to remove a valid request's intimate visual depiction "as soon as possible, but not later than 48 hours after receiving such request", and to make reasonable efforts to remove known identical copies. The process had to be in place by 19 May 2026.
Include what the statute asks for: your signature, enough information to locate the depiction, a statement of good faith belief that it is non-consensual, and your contact details. Requests that skip these are easy to ignore.
Check your state. California, the state that acted here, has a civil route (Civil Code section 1708.86) and criminal provisions the Attorney General cited against xAI in January 2026.
If the subject was a minor, treat it as a criminal matter and report it to law enforcement rather than handling it yourself.

Our company is Belgian, and this is documented procedure rather than legal advice. For anything with a deadline, a threat or a minor involved, talk to a lawyer or a support organisation in your own country.

What is still unresolved

Still open, as of mid-September 2026. Nothing above tells you whether the prohibition works. The Commission's page announces the ban without publishing a separate application date for that clause, Doe 1 v. X.AI Corp. is undecided, and the January 2026 measures left the image tool behind a subscription. Judge this by the end of 2027 on the enforcement record rather than the announcements.

On our side, image generation is a separate tool with 8 models and one image per turn, and we do not add provenance marks to what it produces.