Getty Images. Newsweek/Illustration
By Marni Rose McFall
Reporter
0
Share
Newsweek is a Trust Project member
See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.
Sexualized images of women have flooded X in recent weeks in what is being dubbed the ‘Grok bikini trend.’ The images, created using the AI chatbot 'Grok,' have prompted an outcry and widespread concerns.
Newsweek has seen multiple examples of these kinds of images and the prompts used to create them.
X has responded to the fallout and said they are taking action against it. In response to a request for comment from Newsweek on this story, an X spokesperson shared a statement, originally posted on the platform on January 4. It reads: "We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary. Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content."
Read More on News
Newsweek also contacted XAI, the company behind Grok, for comment and received the following response: "Legacy Media Lies."
X has now restricted Grok's image edit function to be only for paid users, but some have criticized this move as not going far enough. X is currently under investigation by Ofcom, a regulator in the U.K, and an Ofcom spokesperson said in a statement on Friday about the investigation, “We urgently made contact on Monday and set a firm deadline of today to explain themselves, to which we have received a response. We’re now undertaking an expedited assessment as a matter of urgency and will provide further updates shortly.”
In response to threats of a possible ban of the X platform, Musk accused the UK government of wanting to "supress free speech."
Amid the global conversation that has been prompted by the imagery, many are wondering what the legal implications of this content could be. Newsweek spoke to legal experts to find out more.
Getty Images. Newsweek/Illustration
The Context
The alarm had already been sounded over Grok, and its ability to create images of a sexual nature. But in recent weeks, the issue has become more widespread. As the likes of Princess Kate and conservative influencer Ashley St. Clair were targeted, the creation of these images and their implications turned into a major conversation.
The issue, though, is far from isolated. There have been concerns about AI-generated deepfakes and the impact they are having on women for years now. A March 2025 study from SWGfL, a charity that is dedicated to creating safe and secure use of technology, estimated that more than 40 million women globally are victims of Non-Consensual Intimate Image (NCII) abuse.
Meanwhile, a 2025 survey from the National Organization for Women (NOW) and Incogni found that some 25 percent of women have experienced harassment enabled by technology, including AI-generated deepfake pornography.
There are laws in place in the United States that are poised to deal with the sharing of non-consensual intimate imagery. The TAKE IT DOWN Act was passed by the Trump administration in 2025. That act criminalizes the publication of nonconsensual intimate imagery, with up to two years imprisonment. There are harsher penalties in place for content involving children.
What Guardrails Are in Place to Prevent the Creation of Non-Consensual Images?
There are guardrails and protections in place to try to prevent the creation of non-consensual images. For example, Grok will reject requests to create totally nude images. But social media users have found ways around this, including asking the bot to create images of a user in a bikini, with some social media users going so far as requesting said bikini to be see-through, or cling film.
This has led people and experts alike to question whether the guardrails in place are enough.
Annie Seifullah, founding partner of Incendii Law PLLC, a women-owned anti-violence law firm and co-chair of the New York Cyber Abuse Task force, told Newsweek in an email **"**There are very few meaningful guardrails preventing the creation of non‑consensual sexualized images with AI.
"Some mainstream platforms have safety systems that block users from generating harmful content. But for years, many apps and online tools have openly offered abusive image‑manipulation tools targeting real people."
Newsweek also spoke with Carrie Goldberg, a victim’s rights attorney and founder of C.A. Goldberg PLLC, a victim’s rights law firm. "Right now, most AI ‘guardrails’ are voluntary, inconsistent, and easy to evade, which means they function more like talking points than real safety measures," she said.
"This comes at the expense of users, particularly women and children. Unfortunately, victims often discover the limits of these protections only after harm has already occurred. The existence of guardrails does not mean the technology is safe. It means companies have acknowledged the risk, not that they have adequately addressed it," Goldberg added.
Can AI-generated Images Have Legal Consequences?
As to whether AI-generated imagery can have legal consequences, the answer is clear: yes. However, where things get complicated is in the implementation of this, and there are multiple variables at play.
Newsweek spoke to Michael P. Goodyear, an associate professor at the New York Law School. In an email to Newsweek, he highlighted the TAKE IT DOWN Act, which he said, "creates legal consequences for creators and platforms."
While this act is one of the most significant legal protections, Goodyear highlighted some key issues. For example, "The government is the enforcer of the TAKE IT DOWN Act. While victims are empowered to submit takedown notices, they cannot use the law to sue the creator or platform to force compliance."
"Removing the sexualized images from social media and websites has been the focus of law because the most significant harm occurs at the dissemination stage," Goodyear added*.* "The law has not focused on regulating AI tools to prevent the creation of sexualized deepfakes in the first place. While AI companies should continue to develop appropriate guardrails, the nondeterministic nature of AI models means that we can never predict each output with certainty," Goodyear added.
Goldberg, meanwhile, unpacked how legal consequences can work at an individual level. "Individuals who create or distribute non-consensual sexual images, including those generated by AI, can face serious civil and criminal consequences under laws addressing sexual exploitation, harassment, privacy violations, and abuse," she said.
And platforms can face liability too, when "Their tools materially contribute to the creation of the harm through product design choices that make misuse foreseeable," Goldberg said.
Seifullah though highlighted that geography can play a role in consequences for individuals. "The legal consequences depend on factors including where the images were created and distributed," she said. "In New York, for example, victims can sue someone who creates or shares non‑consensual, sexually explicit deepfake imagery. Perpetrators often hide behind anonymous accounts, making it difficult to tie the evidence to the abuser," she said.
Looking Ahead
AI is a rapidly emerging technology that is changing the shape of the world at lightning speed. This in turn means that lawmakers, legislators, and creators are like need to continue to adapt to the technology, and the potential harms it poses.
The conversation sparked by the creation of these images on X has seen widespread calls for change. Lindsey Song, the director of Technology-Facilitated Abuse Initiative Sanctuary for Families and Co-Chair of the New York Cyber Abuse Task Force, shared these sentiments with Newsweek.
"Without a significant change in guardrails and legislative protections, I believe we are looking at a devastating wave of abuse that will be unleashed upon domestic violence survivors," she said.
"With every advent in technology, the ease of creating social media accounts, messaging, Bluetooth trackers, connected cars, and more, we have immediately seen these technologies weaponized against survivors almost immediately.
"Powerful AI technology that creates realistic sexual images of clients is an extraordinarily dangerous tool that must be regulated."
View 0 Comments
Submit Correction
View Editorial & AI Guidelines