The Artificial Intelligence Chatbot Modifies Pictures of Females to Digitally Undress Them

The online platform X has provoked major anger after users employed its AI chatbot Grok to manipulate photos of women by taking off their attire.

Multiple instances have been seen where the tool has been employed to digitally undress female subjects to show them in swimwear without permission, as well as to put them in sexually explicit situations.

Company Reaction and Legal Position

XAI, the company responsible for Grok, did not respond to questions, except for an auto-reply message stating "legacy media lies".

A Home Office representative stated that legislation is being brought forward to outlaw nudification software. Under a new law, anyone who distributed such technology would "receive a prison sentence and heavy penalties".

"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."

Regulator Stance and Tool Functionality

The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time examining X or Grok in relation to AI-generated pictures.

Grok is a no-cost AI assistant—with some paid-for features—which answers X users' requests when they mention it in a post.

It is frequently used to give reactions or additional context to other users' comments, but people on X are also capable of modify an posted image through its AI image editing function.

History of Controversy and Policy

The tool has been criticised for enabling users to produce pictures and footage with explicit content and sexualised content, and it was previously accused of generating inappropriate content of celebrities.

"X has been enabling the production and sharing of these pictures for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.

XAI's own terms of service prohibits "showing images of people in a sexually explicit manner".

In a statement, Ofcom said it was unlawful to "create or share private pictures or illegal content" and confirmed this covered explicit AI images generated with AI.

It said networks such as X were required to take "suitable actions" to "reduce the risk" of users viewing illegal content on their services and to remove it swiftly when they become aware of it.