Elon Musk's online platform X has sparked widespread backlash after individuals utilized its AI chatbot Grok to manipulate pictures of women by taking off their attire.
Several instances have been seen where the tool has been employed to remove clothing from female subjects to show them in swimwear without consent, as well as to place them in sexually explicit contexts.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to inquiries, except for an standard message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being proposed to ban nudification tools. Under a new law, anyone who distributed such technology would "receive a jail term and significant penalties".
"X or Grok could prevent these types of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "consider the risk" of people viewing unlawful material on their platforms, but did not confirm whether it was presently looking into X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which answers X users' requests when they reference it in a tweet.
It is frequently used to give commentary or additional background to other posters' comments, but people on X are also can modify an posted image through its artificial intelligence picture modification feature.
History of Controversy and Rules
The system has been condemned for permitting users to produce pictures and footage with explicit content and sexualised content, and it was in the past accused of making graphic content of public figures.
"X has been permitting the generation and distribution of these images for a long time without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "depicting likenesses of persons in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and confirmed this covered sexual deepfakes made with AI.
It said networks such as X were mandated to take "appropriate steps" to "lessen the danger" of users encountering illegal content on their platforms and to delete it promptly when they are notified of it.
