Dutch court bans Grok AI from making non-consensual nudes

Elon Musk's AI chatbot Grok is now banned from generating deepfake nudes in the Netherlands, as the court of Amsterdam sides with victim support groups in a case that reflects mounting legal pressure on AI platforms across Europe and beyond.

Grok AI is a generative chatbot developed by xAI, the artificial intelligence company founded by Elon Musk, and launched in November 2023. Grok is accessible to users via the X app, Grok.com, and standalone mobile applications for iOS and Android. 

The court imposed a hefty fine of €100,000 per day for any violation of the order, highlighting the growing legal pressure on technology platforms over the proliferation of such harmful content.

Legal action against Grok

Victim support groups Fonds Slachtofferhulp and Offlimits had initiated legal proceedings against Grok and X due to the AI tool's capacity to generate illegal content, specifically deepfake nudes, including those of minors.

In their view, the generated images are realistic and can depict minors, potentially leading to child sexual abuse material, which is illegal under Dutch law. Offlimits, an expertise center on online sexual abuse, demonstrated in court how Grok could generate topless images of women from clothed photos and create nude images of children, violating consent and Dutch law. 

While the creation and distribution of deepfake nudes are already illegal in the Netherlands, the lawsuit seeks to enforce existing laws. The organizations asked the court to ban these features and impose a penalty of €100,000 per day until the nude image generation tool remains available.

xAI's defense

xAI's lawyers argued that a '100% guarantee' against the creation of such images is impossible, as malicious users continually find ways to circumvent security measures. They contended that imposing a fine would unfairly penalize xAI for the actions of third parties.

Grok under the magnifying glass

Grok is facing global scrutiny due to its nudity function, with legal cases pending in multiple countries. In January, the European Commission launched a formal investigation into Grok to determine whether it disseminates illegal content, including manipulated sexualized images of women and children, within the EU. The EU's investigation is assessing whether X has adequately evaluated and mitigated the risks associated with Grok's functionalities. 

Large technology companies operating in the EU are subject to the Digital Services Act (DSA), which sets strict standards for privacy, transparency, and social safety within the bloc. Companies that breach the DSA risk fines of up to 6% of their global annual turnover. 

To comply with the DSA, X must conduct and transmit an ad hoc risk assessment report on Grok's functionalities and properly assess and mitigate the risks associated with its recommender systems. 

In December 2025, the EU fined X €120 million for deceptive design, insufficient advertising transparency, and insufficient data access for researchers. In response to concerns, xAI restricted image editing for Grok AI users and blocked users in jurisdictions where it is illegal from generating images of people in revealing clothing. 

Ofcom, the UK's media regulator, has also launched an investigation into whether X has complied with its duties under the UK's Online Safety Act. 

EU's crackdown on online platforms

Today, the European Commission announced it has also preliminarily found Pornhub, Stripchat, XNXX, and XVideos in breach of the DSA. According to the Commission, they failed to prevent minors from being exposed to pornographic content on their platforms. 

Concurrently, the Commission also started an investigation into Snapchat to assess its compliance with the DSA regarding children's online safety, privacy, and security.