The online platform X has sparked widespread backlash after users utilized its artificial intelligence assistant Grok to change images of females by stripping their garments.
Multiple examples have emerged where the technology has been used to digitally undress women to render them in swimwear without permission, as well as to place them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, offered no comment to requests for comment, aside from an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to outlaw nudification tools. Under a proposed criminal offence, anyone who provided such technology would "be subject to a prison sentence and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people viewing illegal content on their networks, but did not specify whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' requests when they tag it in a post.
It is frequently used to provide reactions or more background to other posters' remarks, but people on X are also can edit an posted image through its AI image editing tool.
History of Controversy and Policy
The chatbot has been condemned for permitting users to create images and clips with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in generating sexually explicit material of celebrities.
"The platform has been enabling the production and distribution of these pictures for months without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of individuals in a graphic manner".
In a comment, Ofcom said it was illegal to "create or distribute non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said services such as X were required to take "appropriate measures" to "lessen the risk" of users viewing unlawful content on their services and to remove it promptly when they learn of it.
