AI tools that can edit photos and create realistic images are raising new concerns about online abuse. Some of these tools can be misused to create explicit deepfakes of real people without their consent.
A report by WIRED has raised concerns about popular AI image-editing models. These models are available on Hugging Face, a platform where developers can share and download AI models. Researchers found that some of these models could be used to create explicit deepfakes with relative ease.
This raises concerns about AI safety and the responsibility of platforms that make these tools available. Hugging Face supports open-source AI development, but its models can also be downloaded, modified and used outside the platform’s control.
The problem is not AI image editing itself. It is the risk of using these tools to create fake sexual images of real people without their permission. Such images can lead to harassment, blackmail and public humiliation, raising questions about whether AI platforms are doing enough to prevent this kind of abuse.
How Hugging Face AI Models Are Being Used to Create Fake Nude Images
AI image-editing models can change existing photos based on text prompts or other instructions. Depending on the model, users can alter a person’s appearance, change their clothing or add details that were not in the original image.
However, these tools can also be misused to create fake nude images and explicit deepfakes of real people without their consent. A WIRED investigation found that 7 of the 9 Hugging Face image-editing Spaces tested could turn a photo of a clothed woman into a topless image.
The investigation also examined more than 1,000 prompts submitted to these tools. Around 73% of the prompts were sexual in nature. Of those sexual requests, 83% aimed to undress or sexualise someone in an uploaded photograph. Women accounted for 95% of the people targeted, while 6.7% of the sexual prompts targeted people who appeared to be children.
These findings raise concerns about the safety measures used in publicly available AI image-editing tools. When models have weak safeguards, users may be able to create manipulated images without advanced editing skills. Downloadable models can also be modified and run outside the original platform, making it harder to control how they are used.
This does not mean every image-editing model on Hugging Face can generate explicit deepfakes. Models have different capabilities and safety measures. However, the investigation highlights the need for stronger testing before high-risk tools become widely available.
The harm can continue long after an image is created. Victims may face online harassment, threats, blackmail and damage to their reputation. Removing images can also be difficult once they have been copied and shared across multiple websites. This shows why AI platforms need stronger safeguards to prevent nonconsensual sexual images from being created and spread.
ALSO READ: AI Has Made Fake Nude Abuse Easier to Scale, New Berkeley Report Warns
Nonconsensual Deepfakes Are a Serious Form of Online Abuse
Creating explicit deepfakes of someone without their permission should be treated as a serious form of online abuse. A person does not need to share an intimate photo for someone to create a fake nude image of them. In some cases, a simple photograph from a public social media profile can be misused.
The image may be fake, but the damage is real. Victims can face harassment, blackmail, threats and public humiliation. Women and girls are particularly vulnerable to this form of abuse, which can affect their safety, privacy and reputation.
Children face even greater risks. AI-generated sexual images of minors raise serious child safety concerns, even when the images are fabricated. Platforms must take stronger steps to prevent their tools from being used to create such material and respond quickly when abuse is reported.
AI image editing can have legitimate uses, but those uses should not come at the expense of someone’s consent. Technology companies must draw a clear line between creative editing and the creation of explicit images of real people without their permission. Stronger safeguards, clear reporting systems and faster removal of abusive content should be essential parts of that effort.
Hugging Face Needs Stronger Safeguards Against AI Deepfake Abuse
Hugging Face makes it easier for developers and researchers to share AI models and build new tools. This open approach supports innovation, but it also raises safety concerns when models can be misused to create explicit deepfakes of real people.
Hugging Face and model developers should take stronger steps to prevent this type of abuse. High-risk image models should be tested to see whether they can generate nonconsensual sexual images, rather than relying only on the safety claims in their documentation.
Developers should also add effective safeguards to prevent the misuse of real people’s photos, especially images of children. Model pages should clearly explain known risks, safety limits and the steps taken to reduce potential harm.
Hugging Face should also provide a clear reporting process for models that enable sexual abuse. Reports should be reviewed quickly, with appropriate action taken when a model violates platform rules or presents serious risks.
No safety measure can prevent every case of misuse, particularly when models can be downloaded and modified. However, this should not stop platforms from taking reasonable steps to reduce the risk. Open-source AI development should support innovation without overlooking the potential harm to people whose images are used without their consent.
ALSO READ: Open Source AI Statistics
Open-Source AI Platforms Must Take Responsibility After Release
Open-source AI development involves model creators, platforms and users, each with a role in preventing misuse. Making a model available for research or experimentation does not remove the risks it may pose after release.
Platforms cannot predict every possible misuse, but they should not assume a model is safe simply because it has a legitimate purpose. Safety checks should reflect what a model can actually do. A tool that restores old photos may pose different risks from one that can generate explicit images of real people.
Disclaimers alone are not enough to prevent AI deepfake abuse. Platforms should introduce practical safeguards for high-risk models while keeping legitimate research and image editing possible.
The goal should be to reduce the risk of harm without placing unnecessary limits on open-source AI. Platforms must find that balance before and after releasing models to the public.
Deepfake Victims Need Faster Image Removal and Better Support
Preventing explicit deepfakes is only part of the solution. Victims also need quick and simple ways to report abusive images and get them removed. Fake nude images can lead to online harassment, threats, blackmail and humiliation. Victims may also fear that family members, friends or employers will see the images, affecting their personal relationships, reputation and sense of safety.
AI platforms, social media companies and websites should provide clear reporting options and respond quickly to complaints. They should also limit the spread of reported images and take steps to prevent them from being reposted.
Victims should not have to struggle to prove their case before receiving help. Even when an explicit image is entirely fabricated, the emotional distress, damage to personal relationships and fear of further exposure can be real.
Public awareness is equally important. People need to understand that creating or sharing explicit deepfakes without consent can cause lasting harm. Platforms must make reporting and removal easier so victims are not left to manage the abuse on their own.
Hugging Face Must Take Explicit Deepfake Risks More Seriously
Concerns about explicit deepfakes on Hugging Face raise questions about how the platform checks the safety of the AI models it hosts. Reports suggest that popular image-editing models can be misused to create sexual images of real people. This shows the risks of making these tools publicly available without adequate safeguards.
Hugging Face has helped developers and researchers share AI models more easily. But when these models can be used to create fake sexual images of real people, their potential for misuse deserves serious attention. Safety checks should examine what a model can actually do, not just what its developers claim it is designed for.
Hugging Face cannot control every use of a model, especially after someone downloads and modifies it. However, this limitation should not become an excuse for weak safeguards. The platform should encourage stronger safety testing and clear risk disclosures. It should also provide simple ways to report models that enable nonconsensual deepfake abuse.
Victims also need better protection. Fake sexual images can lead to harassment, blackmail and humiliation, even when the images are entirely fabricated. Platforms should make it easier to report abusive content and take prompt action when complaints are verified.
Open-source AI should remain accessible for legitimate research and innovation. But that does not mean every model should be released without careful consideration of its risks. If an AI model can be easily misused to create explicit images of real people without their consent, developers should take steps to prevent such abuse.
It should be part of the safety process, not an afterthought. Therefore, Hugging Face and model developers need to take this responsibility seriously.