The online platform X has sparked widespread backlash after people employed its artificial intelligence assistant Grok to change photos of females by taking off their clothing.
Several cases have surfaced where the technology has been employed to remove clothing from female subjects to show them in swimwear without consent, as well as to insert them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to inquiries, except for an automatically-generated response stating "traditional news outlets lies".
A government official stated that legislation is being proposed to outlaw nudification tools. Under a proposed law, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing illegal content on their platforms, but did not confirm whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some additional functions—which responds to X users' requests when they tag it in a post.
It is commonly used to offer commentary or further background to other posters' remarks, but people on X are also capable of alter an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The tool has been condemned for enabling users to generate pictures and footage with nudity and sexualised content, and it was in the past accused of creating inappropriate material of public figures.
"The platform has been enabling the production and spread of these pictures for months without taking any action and we have not seen any action by regulators," the expert added.
XAI's own acceptable use policy forbids "showing likenesses of persons in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said networks such as X were obligated to take "necessary measures" to "lessen the risk" of users encountering unlawful content on their services and to remove it promptly when they become aware of it.
