The social media network X has provoked widespread outrage after individuals used its artificial intelligence assistant Grok to change images of women by removing their garments.
Numerous examples have emerged where the technology has been used to virtually strip women to show them in bikinis without consent, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Position
XAI, the organization behind Grok, offered no comment to requests for comment, except for an auto-reply message stating "legacy media lies".
A Home Office official stated that new laws is being proposed to prohibit image stripping technology. Under a proposed criminal offence, anyone who distributed such technology would "face a jail term and substantial penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the danger" of people seeing illegal content on their networks, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some additional features—which answers X users' prompts when they tag it in a post.
It is often used to give commentary or further background to other posters' comments, but people on X are also can alter an uploaded picture through its AI image editing function.
Background of Controversy and Policy
The tool has been criticised for enabling users to produce pictures and videos with nudity and sexualised content, and it was previously accused of creating graphic material of public figures.
"The platform has been enabling the generation and sharing of these pictures for months without taking any measures and we have not seen any action by regulators," the expert added.
XAI's own acceptable use policy prohibits "depicting images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and stated this included explicit AI images generated with AI.
It said services such as X were mandated to take "suitable measures" to "lessen the risk" of users viewing illegal material on their services and to take down it quickly when they are notified of it.
