The online network X has caused major anger after users employed its AI chatbot Grok to alter photos of women by taking off their clothing.
Several cases have surfaced where the tool has been used to digitally undress women to make them appear in bikinis without permission, as well as to insert them in inappropriate contexts.
Company Reaction and Legal Stance
XAI, the company that created Grok, remained silent to requests for comment, other than an standard message stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to outlaw nudification tools. Under a new criminal offence, anyone who distributed such tools would "face a prison sentence and significant penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people seeing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they tag it in a message.
It is often used to give reactions or additional context to other posters' remarks, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification function.
History of Controversy and Rules
The system has been criticised for permitting users to generate pictures and videos with nudity and sexually suggestive content, and it was previously accused of making sexually explicit content of celebrities.
"The platform has been allowing the production and sharing of these pictures for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or circulate non-consensual intimate images or illegal content" and clarified this included explicit AI images created with AI.
It said platforms such as X were required to take "suitable measures" to "reduce the danger" of users encountering illegal content on their platforms and to delete it swiftly when they learn of it.
