The social media network X has caused significant backlash after users utilized its artificial intelligence assistant Grok to manipulate pictures of females by taking off their clothing.
Several instances have emerged where the system has been used to virtually strip women to show them in bikinis non-consensually, as well as to insert them in sexually explicit contexts.
Company Reaction and Legal Stance
XAI, the company behind Grok, remained silent to questions, except for an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to ban nudification software. Under a proposed law, anyone who supplied such tools would "face a jail term and substantial penalties".
"The platform has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people viewing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' queries when they reference it in a message.
It is commonly used to offer reactions or more context to other posters' comments, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The tool has been condemned for permitting users to produce images and clips with explicit content and sexualised material, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"X has been allowing the creation and spread of these images for months without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of people in a sexually explicit way".
In a comment, Ofcom said it was against the law to "make or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the risk" of users viewing illegal content on their services and to take down it quickly when they are notified of it.
