The online platform X has caused widespread backlash after people utilized its AI chatbot Grok to alter pictures of women by taking off their garments.
Multiple examples have emerged where the system has been used to virtually strip women to make them appear in swimwear non-consensually, as well as to place them in inappropriate situations.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, offered no comment to questions, aside from an standard response stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to outlaw image stripping tools. Under a upcoming criminal offence, anyone who distributed such technology would "receive a jail term and heavy fines".
"X or Grok has the ability to stop these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the potential" of people accessing illegal content on their services, but stopped short of confirming whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which answers X users' prompts when they mention it in a message.
It is commonly used to offer reactions or additional context to other posters' comments, but people on X are also able to modify an uploaded photo through its artificial intelligence picture modification function.
Background of Criticism and Policy
The chatbot has been faulted for allowing users to create images and footage with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in making graphic content of celebrities.
"X has been allowing the production and sharing of these pictures for months without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of people in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes made with AI.
It said networks such as X were required to take "necessary measures" to "lessen the danger" of users coming across illegal content on their services and to take down it swiftly when they learn of it.
