The online platform X has sparked major outrage after individuals utilized its AI chatbot Grok to change images of women by taking off their garments.
Multiple cases have emerged where the tool has been used to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to place them in sexually explicit situations.
Developer Reaction and Regulatory Stance
XAI, the organization behind Grok, remained silent to inquiries, other than an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to prohibit image stripping tools. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial penalties".
"The platform could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "assess the risk" of people viewing illegal content on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' requests when they tag it in a message.
It is commonly used to offer reactions or more background to other posters' comments, but people on X are also can alter an uploaded image through its AI image editing tool.
History of Criticism and Rules
The tool has been condemned for allowing users to create images and clips with nudity and sexualised material, and it was previously alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been allowing the generation and distribution of these images for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of persons in a pornographic fashion".
In a comment, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and confirmed this included sexual deepfakes created with AI.
It said services such as X were mandated to take "suitable steps" to "lessen the danger" of users encountering illegal material on their platforms and to delete it quickly when they become aware of it.
