The social media platform X has sparked major backlash after individuals used its AI chatbot Grok to change images of females by removing their clothing.
Several examples have emerged where the tool has been employed to remove clothing from women to show them in bikinis without consent, as well as to insert them in sexually explicit situations.
Developer Response and Regulatory Position
XAI, the company that created Grok, remained silent to requests for comment, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit image stripping software. Under a proposed criminal offence, anyone who supplied such technology would "face a jail term and significant penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said tech firms must "consider the potential" of people viewing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for functions—which answers X users' prompts when they mention it in a post.
It is commonly used to give commentary or additional background to other users' comments, but people on X are also can alter an posted picture through its artificial intelligence picture modification tool.
History of Controversy and Rules
The tool has been faulted for permitting users to generate photos and videos with nudity and sexually suggestive material, and it was previously alleged to have been involved in creating inappropriate content of public figures.
"X has been enabling the generation and distribution of these pictures for a long time without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a pornographic fashion".
In a statement, Ofcom said it was illegal to "create or distribute private pictures or illegal content" and confirmed this included explicit AI images generated with AI.
It said platforms such as X were mandated to take "suitable steps" to "lessen the danger" of users coming across unlawful content on their services and to delete it quickly when they are notified of it.
