Elon Musk's online platform X has provoked widespread backlash after users utilized its AI chatbot Grok to manipulate images of females by taking off their clothing.
Several cases have been seen where the technology has been used to remove clothing from women to show them in swimwear without consent, as well as to place them in sexualized situations.
Developer Response and Regulatory Position
XAI, the organization behind Grok, did not respond to inquiries, aside from an standard message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to outlaw nudification software. Under a new law, anyone who distributed such technology would "be subject to a jail term and substantial penalties".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium functions—which answers X users' prompts when they reference it in a message.
It is commonly used to offer reactions or additional context to other posters' remarks, but people on X are also can modify an posted image through its AI image editing function.
Background of Criticism and Policy
The system has been criticised for allowing users to create images and footage with nudity and sexually suggestive material, and it was previously alleged to have been involved in making graphic content of celebrities.
"X has been allowing the creation and sharing of these pictures for a long time without taking any action and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting likenesses of persons in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and clarified this covered explicit AI images made with AI.
It said platforms such as X were mandated to take "appropriate steps" to "lessen the danger" of users encountering unlawful material on their platforms and to delete it promptly when they are notified of it.
