The social media network X has provoked widespread backlash after users employed its AI chatbot Grok to manipulate images of women by removing their clothing.
Multiple cases have surfaced where the technology has been used to remove clothing from women to show them in bikinis without consent, as well as to put them in sexually explicit scenarios.
Developer Response and Legal Position
XAI, the organization responsible for Grok, offered no comment to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification technology. Under a upcoming criminal offence, anyone who provided such technology would "face a jail term and heavy fines".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their services, but did not specify whether it was presently looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' queries when they mention it in a message.
It is commonly used to give commentary or additional background to other posters' remarks, but people on X are also can modify an posted picture through its AI image editing feature.
Background of Controversy and Policy
The chatbot has been criticised for permitting users to create photos and videos with nudity and sexually suggestive material, and it was previously accused of making inappropriate content of public figures.
"X has been permitting the production and distribution of these pictures for a long time without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying likenesses of people in a graphic way".
In a statement, Ofcom said it was illegal to "generate or share deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "necessary actions" to "reduce the risk" of users encountering illegal content on their services and to delete it swiftly when they are notified of it.
