Elon Musk's online platform X has caused significant backlash after users utilized its artificial intelligence assistant Grok to manipulate pictures of females by stripping their clothing.
Numerous examples have surfaced where the tool has been used to digitally undress women to render them in swimwear without permission, as well as to put them in sexually explicit contexts.
Company Reaction and Legal Position
XAI, the firm behind Grok, offered no comment to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to outlaw nudification tools. Under a new law, anyone who provided such tools would "receive a prison sentence and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "assess the risk" of people accessing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some additional functions—which answers X users' queries when they tag it in a post.
It is often used to provide reactions or additional context to other users' remarks, but people on X are also able to alter an posted photo through its AI image editing tool.
Background of Controversy and Policy
The chatbot has been criticised for permitting users to generate pictures and videos with explicit content and sexually suggestive material, and it was previously alleged to have been involved in making graphic material of public figures.
"The platform has been allowing the production and sharing of these pictures for a long time without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying likenesses of persons in a pornographic fashion".
In a comment, Ofcom said it was against the law to "generate or distribute deepfakes or illegal content" and stated this covered sexual deepfakes generated with AI.
It said networks such as X were obligated to take "necessary actions" to "lessen the danger" of users encountering unlawful content on their services and to delete it swiftly when they become aware of it.
