Inside the marketplace powering bespoke AI deepfakes of real women
The Dark Side of AI: Inside the Marketplace Powering Bespoke Deepfakes of Real Women
A recent study by researchers at Stanford and Indiana University has shed light on the growing concern of AI-generated deepfakes, specifically targeting real women. The study, which analyzed the online marketplace Civitai, found that a significant portion of requests for celebrity deepfakes were for LoRAs (Local Random Access) files, which can coach mainstream AI models like Stable Diffusion into generating explicit content.
Civitai: A Platform for AI Model Sharing
Civitai is an online marketplace that allows users to buy and sell AI-generated content, including images, videos, and models. The platform has gained popularity among AI enthusiasts and developers, who use it to share and collaborate on AI projects. However, the study's findings suggest that Civitai has become a hub for creating and disseminating explicit deepfakes.
The Problem with LoRAs
LoRAs are instruction files that can be used to coach AI models into generating specific types of content. In the case of Civitai, many users have requested LoRAs that can generate explicit deepfakes of real women. These files can be used to create highly realistic and personalized deepfakes, which can be used for a variety of purposes, including harassment and exploitation.
Civitai's Response
Civitai has taken steps to address the issue of explicit deepfakes on its platform. In May 2025, the company announced that it would ban all deepfake content, including LoRAs. However, the study's findings suggest that many of the winning submissions fulfilling these requests remain available for purchase on the platform.
The Implications of AI-Generated Deepfakes
The study's findings have significant implications for the use of AI-generated deepfakes. While AI models can be used to create highly realistic and personalized content, they can also be used to create explicit and exploitative material. The ease with which LoRAs can be used to generate explicit deepfakes raises concerns about the potential for widespread harassment and exploitation.
The Role of Venture Capital Firms
Civitai has received significant investment from venture capital firms, including Andreessen Horowitz. The study's findings suggest that these firms may not be doing enough to address the issue of explicit deepfakes on their platforms. The lack of attention to this issue raises concerns about the potential for widespread exploitation and harassment.
The Need for Regulation
The study's findings suggest that there is a need for regulation around the use of AI-generated deepfakes. While AI models can be used to create highly realistic and personalized content, they can also be used to create explicit and exploitative material. The ease with which LoRAs can be used to generate explicit deepfakes raises concerns about the potential for widespread harassment and exploitation.
Conclusion
The study's findings suggest that Civitai has become a hub for creating and disseminating explicit deepfakes. The ease with which LoRAs can be used to generate explicit deepfakes raises concerns about the potential for widespread harassment and exploitation. The lack of attention to this issue by venture capital firms raises concerns about the potential for widespread exploitation and harassment. The need for regulation around the use of AI-generated deepfakes is clear.
Forward-Looking Thoughts
The study's findings suggest that the use of AI-generated deepfakes is a growing concern. As AI models become more advanced and widely available, the potential for widespread harassment and exploitation increases. The need for regulation and oversight is clear. The development of new technologies and tools to detect and prevent the creation and dissemination of explicit deepfakes is also essential.
Final Thoughts
The study's findings suggest that the use of AI-generated deepfakes is a complex issue that requires a multifaceted approach. The need for regulation, oversight, and education is clear. The development of new technologies and tools to detect and prevent the creation and dissemination of explicit deepfakes is also essential. The future of AI-generated deepfakes is uncertain, but one thing is clear: the need for responsible innovation and regulation is paramount.
