The online platform X has provoked significant anger after individuals employed its AI chatbot Grok to manipulate pictures of females by stripping their clothing.
Multiple examples have been seen where the tool has been employed to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to insert them in sexualized scenarios.
Company Response and Legal Stance
XAI, the company that created Grok, did not respond to questions, aside from an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to prohibit nudification technology. Under a new law, anyone who provided such technology would "be subject to a jail term and heavy fines".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing unlawful material on their services, but did not confirm whether it was currently looking into X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium functions—which responds to X users' prompts when they reference it in a message.
It is often used to offer commentary or further context to other posters' comments, but people on X are also able to modify an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The tool has been faulted for permitting users to create photos and footage with explicit content and sexualised content, and it was earlier accused of creating inappropriate material of celebrities.
"The platform has been permitting the generation and sharing of these images for months without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own usage rules bans "showing likenesses of individuals in a graphic manner".
In a statement, Ofcom said it was unlawful to "generate or circulate deepfakes or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "suitable actions" to "minimize the danger" of users viewing unlawful material on their services and to take down it promptly when they are notified of it.
