The online platform X has provoked widespread anger after users used its AI chatbot Grok to alter pictures of women by taking off their attire.
Several instances have emerged where the system has been employed to digitally undress female subjects to show them in bikinis without consent, as well as to insert them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to prohibit image stripping technology. Under a upcoming criminal offence, anyone who distributed such technology would "receive a jail term and substantial penalties".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people seeing illegal content on their services, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a no-cost artificial intelligence—with some premium features—which answers X users' prompts when they mention it in a post.
It is often used to give commentary or additional background to other users' remarks, but people on X are also can edit an posted picture through its AI image editing tool.
History of Criticism and Policy
The system has been criticised for enabling users to create images and clips with nudity and sexually suggestive content, and it was earlier alleged to have been involved in making inappropriate material of public figures.
"The platform has been enabling the creation and distribution of these pictures for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "generate or distribute deepfakes or child sexual abuse material" and clarified this included explicit AI images generated with AI.
It said services such as X were mandated to take "necessary actions" to "lessen the risk" of users viewing illegal material on their services and to take down it quickly when they are notified of it.
