Elon Musk's online platform X has sparked significant anger after people used its AI chatbot Grok to manipulate photos of females by removing their garments.
Multiple cases have been seen where the system has been employed to remove clothing from female subjects to render them in bikinis without permission, as well as to insert them in inappropriate situations.
Company Response and Regulatory Position
XAI, the firm that created Grok, did not respond to inquiries, aside from an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being proposed to prohibit nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok could prevent these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing illegal content on their platforms, but did not confirm whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' prompts when they mention it in a post.
It is commonly used to offer reactions or additional background to other users' comments, but people on X are also able to alter an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The system has been condemned for enabling users to produce photos and clips with nudity and sexualised material, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"The platform has been permitting the creation and spread of these images for months without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "create or share non-consensual intimate images or illegal content" and clarified this covered explicit AI images generated with AI.
It said networks such as X were required to take "appropriate measures" to "lessen the danger" of users encountering unlawful content on their platforms and to take down it swiftly when they learn of it.
