by Robin Roenker
In October 2023, male students at Westfield High School in New Jersey were caught distributing nude images of several female classmates in group text chats.
The images in question were not real, but rather so-called “deepfakes”— images or videos that have been generated by artificial intelligence (AI) with such accuracy that they appear to be real. Students behind the cyberbullying attack in Westfield used deepfake technology to blend the girls’ faces, sourced from real social media photos, onto realistic-looking, AI-produced nude bodies.
Westfield student Francesca Mani, one of the victims, appeared on The New York Times’ podcast, The Daily, and told technology reporter Natasha Singer how she felt when she realized that she was one of the victims
“In the beginning, I felt shocked because I didn’t think it could happen to me. And that’s the funny part because it could happen to anyone,” Mani said. “I’m kind of sad, too, because it’s surprising, because I didn’t think my classmates could do this to me.”
Mani isn’t wrong; the deepfake threat is real and wide-reaching. According to Sensity AI, an image-detection firm, more than 90% of deepfakes are pornographic. AI advances have made it possible for attackers, with no specialized knowledge or training, to create sexually explicit images and videos of their intended victims in a matter of minutes.
Ben Colman, chief executive of Reality Defender, which works with companies and the government to detect AI-generated fake images, told The Wall Street Journal, “You would have needed an entire cluster of computers to generate images a few years ago. Now you just need an iPhone.”
In addition to Westfield, cases of classmate-created nude deepfakes targeting middle or high school students have been reported in dozens of cities across the U.S., including Newtown Township, Pennsylvania; Laguna Beach, California; Aledo, Texas; and Miami, Florida.
According to a survey conducted by the Center for Democracy and Technology, a nonprofit organization that advocates for digital rights and freedom of expression, 40% of students and 29% of teachers were aware of deepfakes depicting individuals in their schools during the 2023-2024 school year, though not all of those cases involved pornography.
Spreading nude or explicit images without the depicted person’s permission—often called non-consensual intimate image (NCII) abuse—can leave victims feeling deep trauma. These attacks represent not only an inherent privacy violation but also a serious blow to victims’ self-esteem and reputation within their peer groups.
Emotional toll
Beyond student-driven deepfake incidents, there has also been an alarming rise in so-called “sextortion” cases. In these NCII scenarios, which often target pre-teen and teen boys, an internet bad actor threatens to post real or digitally created intimate images of victims online without their consent, unless they pay a ransom.
The shame and embarrassment that stem from an NCII attack—no matter the type—can lead victims to experience profound depression, anxiety or even thoughts of self-harm. Earlier this year in Kentucky, 16-year-old student Elijah Heacock died by suicide after someone created fake, AI-generated nude images of him and threatened to post them online unless he paid them $3,000.
The National Center for Missing and Exploited Children told CBS News that it received more than 500,000 reports of sextortion scams targeting minors in 2024. The Federal Bureau of Investigation estimates that since 2021 at least 20 teenagers have committed suicide due to sextortion scams.
Enacting protections
Since the Westfield incident, Mani and her parents have pushed for stronger protection for victims of real or AI-generated NCII attacks, both in New Jersey and across the nation.
In April 2025, Governor Phil Murphy signed a bipartisan bill dealing with deepfakes into law. The new law establishes civil and criminal penalties “for production or dissemination of deceptive audio or visual media, commonly known as “deepfakes.”
Violating the new law is a third-degree crime subject to imprisonment and a fine up to $30,000. In addition, the law has a provision that enables victims to sue for damages.
“I am proud to sign today’s legislation and take a stand against deceptive and dangerous deepfakes,” Governor Murphy said in a press release. “While artificial intelligence has proven to be a powerful tool, it must be used responsibly. My administration is laser-focused on combatting misinformation and ensuring media integrity. We stand with the victims of deepfake imagery and will continue to prioritize the safety and well-being of all New Jerseyans.”
Now 16, Mani was present for the signing of the New Jersey law. She also appeared at a Washington, D.C. press conference with First Lady Melania Trump in March 2025, where they called for passage of then pending federal legislation to address the growing deepfake threat. The federal legislation was dubbed the “TAKE IT DOWN Act.”
“It’s heartbreaking to witness young teens, especially girls, grappling with the overwhelming challenges posed by malicious online content like deepfakes,” First Lady Melania Trump said, while advocating for the new privacy protections. “This toxic environment can be severely damaging.”
In February 2025, the U.S. Senate passed the TAKE IT DOWN Act unanimously. In April 2025 the House of Representatives approved the legislation by a vote of 409-2. The Act represents the first federal law to take action against the growing problem of NCII.
In May 2025, Mani and her parents were again invited to the U.S. capital to witness President Donald Trump’s White House signing of the “TAKE IT DOWN Act” into law. The act prohibits non-consensual distribution of nude images or videos, whether real or deepfakes. The act also requires digital platforms to remove such content within 48 hours, once its presence is reported.
The new law, whose full title is the “Tools to Address Known Exploitation by Immobilizing Technological Deepfakes On Websites and Networks Act,” makes it a federal crime to post or to threaten to post sexually explicit images without the permission of the person being shown.
Online platforms, including popular social media outlets like Instagram, Facebook and Twitter (now X), will be required to create a streamlined process that enables users to report non-consensual images and to request their removal. Websites and social media platforms must enact such a reporting framework by May 2026.
Platforms must also fully document their “good faith” efforts to comply with the new law, including steps to remove NCII promptly, once alerted to its presence on their site. The Act empowers the Federal Trade Commission (FTC) to bring legal charges against any platform that does not comply, citing existing consumer protection laws.
TAKE IT DOWN also asserts that individuals behind NCII attacks can now face fines and up to two years imprisonment for posting or threatening to post nonconsensual images of adults. Individuals who post or threaten to post nonconsensual images of minors (people under age 18) can face fines and up to three years of jail time.
While the distribution of child pornography is already criminalized under U.S. federal law, proponents of TAKE IT DOWN argue that the law fills an important legal gap by criminalizing the non-consensual distribution of nude or exploitative images of children—including deepfakes—with an intent to harass the victim. In some scenarios, such actions had been difficult to prosecute under traditional child pornography restrictions.
Possible challenges
In addition to vast bipartisan support on Capitol Hill, the legislation has also received widespread approval from leading technology companies, including Google, TikTok, Microsoft and Meta (parent company of Facebook, Instagram and WhatsApp).
In a statement, victims’ rights advocates, including RAINN (Rape, Abuse & Incest National Network), a leading anti-sexual violence organization, praised passage of the new law as “historic legislation…represent[ing] a monumental shift in protecting victims—especially children and survivors of tech-enabled sexual abuse—from exploitation in the digital age.”
The new law is not without its detractors. Some civil liberties advocates have pushed back against TAKE IT DOWN, citing concerns that it may trigger removal of creative content that should, lawfully, be shielded by existing First Amendment free speech protections. One such group, the Electronic Frontier Foundation, a non-profit digital rights organization, says the law will lead to overreach and censorship of lawful speech.
“There’s a tension between First Amendment values and the need to protect vulnerable people, including younger people, from content like deepfakes or non-consensual sexual content,” says David Opderbeck, a professor at Seton Hall Law School and co-director of Seton Hall’s Institute for Privacy Protection and its Gibbons Institute for Law Science and Technology.
Professor Opderbeck says the TAKE IT DOWN law “deals with really tricky First Amendment issues on freedom of speech.” As a result, he expects constitutional challenges to the law and says that the U.S. Supreme Court may eventually have to weigh in.
In the meantime, students should strive to keep an open line of communication with a parent, teacher or other trusted adult if they become targets of a deepfake or other NCII attack. With adult support, students can take steps to alert appropriate school and local authorities to the situation—and leverage the TAKE IT DOWN ACT’s new guidelines to ensure exploitative images are erased from social media sites.
Discussion Questions
- In terms of creating deepfakes, do you think technology is moving faster than the public can handle? Why or why not?
- Supporters of the TAKE IT DOWN Act say it provides needed protection for victims of a deepfake attack, while detractors contend that it conflicts with First Amendment rights. Make your best argument for either side.
- What do you think is the best way to protect yourself from having a deepfake made of you?
Glossary Words
bipartisan—supported by two political parties.
minor—a person under 18 years of age.
non-consensual— without the agreement or permission of all parties involved.
third-degree crime—a crime that carries a maximum sentence of five years in prison.
This article originally appeared in the Fall 2025 issue of The Legal Eagle