On Tuesday, the United States officially implemented a groundbreaking federal rule targeting the rapid spread of abusive AI-generated content: the Take It Down Act, a law that compels digital tech platforms to remove non-consensual intimate imagery including sexual deepfakes, even as policy experts and free speech advocates sound the alarm over unaddressed gaps in the legislation and its potential for overreach and misuse.
The framework, first signed into law by former President Donald Trump in 2023, goes beyond setting content standards—it actually criminalizes the intentional online distribution of non-consensual sexual material, most of which is now produced using low-cost, widely accessible artificial intelligence tools that can generate realistic fake content in minutes.
According to guidance released by the Federal Trade Commission (FTC), the nation’s top consumer protection regulator, the new enforcement mandate requires all covered tech platforms to build a formal, accessible process that lets victims of non-consensual intimate content submit takedown requests. Platforms must process and remove valid requests within a 48-hour window, and any company that fails to comply will face formal penalties. The agency did not outline specific fine amounts, but noted that it has full authority to levy civil penalties for repeated violations.
FTC Chairman Andrew Ferguson confirmed Tuesday that the agency had sent formal advance notifications to more than 15 major technology companies ahead of the enforcement date, including industry leaders Meta Platforms, TikTok, X (formerly Twitter), and Snap Inc.’s Snapchat. “We stand ready to monitor compliance, investigate violations, and enforce the Take It Down Act,” Ferguson stated in his official remarks, adding that “Protecting the vulnerable — especially children — from this harmful abuse is a top priority for this agency and this administration.”
Major platforms have already moved to signal their commitment to complying with the new rule. On Monday, X’s official safety account published a public statement reinforcing the company’s existing policies, noting that “there is no place in our society for predators to share intimate photos and videos of others without their consent. X has zero tolerance for non-consensual intimate images, unwanted sexual content, or any kind of exploitative behaviour.”
The company’s assurance comes in the wake of major controversy earlier this year, when X owner Elon Musk’s generative AI tool Grok sparked global backlash after it was revealed the model could generate non-consensual deepfake nudity of women and underage minors. Independent researchers documented that Grok generated an estimated 3 million sexually explicit deepfake images in just a few days after its unrestricted public launch.
Despite broad support for cracking down on abusive deepfake content, many policy analysts and free speech advocates warn the Take It Down Act carries significant unintended risks that have not been resolved, and ultimately falls short of being a comprehensive solution to the growing crisis of non-consensual AI imagery.
Riana Pfefferkorn, a policy fellow at the Stanford Institute for Human-Centered Artificial Intelligence, argues the law’s strict 48-hour deadline and penalty structure creates an incentive for over-censorship, creating what she describes as a “shoot first, ask questions never” dynamic. In comments to *Indicator*, a digital deception investigative newsletter, Pfefferkorn noted the law’s incentive structure pushes platforms “in the direction of just remove it, remove it, remove it” regardless of whether the content is actually unlawful or non-consensual.
Pfefferkorn also warned the law’s broad provisions could be deliberately weaponized against marginalized groups: trans people seeking to share legitimate, consensual content about their lives, consensual adult sex workers, and political speakers who hold views that conflict with the current administration’s priorities.
These concerns are echoed by a broad coalition of free speech advocates, who argue the threat of penalties will encourage platforms to proactively remove even non-offending, legally protected content to avoid the risk of fines or regulatory action, resulting in widespread unnecessary censorship of legitimate speech.
The urgent push for regulation comes as the global spread of non-consensual deepfakes has outpaced policy responses around the world. The proliferation of easy-to-use AI tools, including consumer-facing “nudification” apps that can turn any public photo into explicit deepfake content, has created a booming market for abusive material that regulators are still scrambling to catch up to.
While high-profile cases targeting public figures — including pop star Taylor Swift, who was targeted by a widespread non-consensual deepfake porn campaign earlier this year — have drawn national media attention, researchers emphasize that ordinary women are just as vulnerable to abuse. In recent months, dozens of US K-12 and high schools from California to New Jersey have reported widespread AI deepfake porn scandals, where male students created non-consensual explicit deepfakes of hundreds of female classmates, prompting bullying, harassment, and mental health crises across affected school communities.
Public health researchers and advocates warn that non-consensual intimate imagery, regardless of whether it is deepfake or real, inflicts severe long-term harm: victims routinely face sustained harassment, extortion and blackmail, and devastating impacts on mental health that can lead to anxiety, depression, and even suicidal ideation in extreme cases.