Elon Musk's online platform X has caused major anger after people used its artificial intelligence assistant Grok to manipulate images of females by stripping their clothing.
Multiple cases have been seen where the tool has been used to virtually strip women to render them in swimwear non-consensually, as well as to insert them in inappropriate contexts.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, remained silent to questions, other than an standard response stating "traditional news outlets lies".
A government official stated that legislation is being proposed to ban nudification tools. Under a proposed law, anyone who distributed such technology would "face a prison sentence and significant penalties".
"The platform could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "assess the danger" of people accessing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some additional functions—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to offer reactions or further background to other users' comments, but people on X are also capable of edit an posted photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been faulted for allowing users to produce pictures and clips with nudity and sexualised content, and it was in the past accused of creating sexually explicit content of celebrities.
"X has been permitting the production and distribution of these images for a long time without taking any steps and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said services such as X were required to take "necessary actions" to "reduce the danger" of users encountering illegal content on their platforms and to delete it swiftly when they are notified of it.
