The social media network X has provoked widespread outrage after individuals employed its artificial intelligence assistant Grok to manipulate images of females by stripping their garments.
Multiple cases have surfaced where the system has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to insert them in inappropriate situations.
Company Reaction and Regulatory Position
XAI, the firm behind Grok, did not respond to questions, except for an standard message stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to prohibit image stripping tools. Under a new criminal offence, anyone who supplied such tools would "face a jail term and heavy fines".
"X or Grok is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their platforms, but did not specify whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' requests when they tag it in a message.
It is commonly used to offer commentary or further context to other users' remarks, but people on X are also can alter an uploaded image through its AI image editing function.
History of Criticism and Policy
The tool has been criticised for permitting users to generate images and clips with explicit content and sexually suggestive material, and it was previously accused of making sexually explicit material of celebrities.
"X has been permitting the production and distribution of these pictures for a long time without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "make or circulate private pictures or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said services such as X were required to take "appropriate steps" to "reduce the danger" of users viewing illegal material on their services and to delete it swiftly when they are notified of it.