Generative artificial intelligence (GenAI) tools like OpenAI’s DALL-E, Midjourney, and Stability AI’s Stable Diffusion have considerably reduced the cost and increased the accessibility of media production. However, the proliferation of GenAI models, especially those traded in the illicit market and dark forums, has also leveraged large-scale, cross-jurisdictional criminal activities including scams, sexual abuse, and political disinformation campaigns, where deepfake technology plays a critical role.
Deepfakes, as a form of synthetic media, are digitally manipulated audio or video clips generated through artificial intelligence (AI) that mimic an individual’s voice and facial features, therefore fabricating deceptive representations of events that never transpired.
The term “deepfake”, a blend of “deep learning” and “fake”, was first coined in 2017 by a Reddit user who created and maintained the subreddit “r/deepfakes” to share face-swapped pornographic content. In 2018, Reddit suspended the subreddit and banned sexual or suggestive content involving minors and involuntary pornography.
Throughout the years, the use of deepfakes has had adversarial impacts on public figures, such as politicians and celebrities, as well as ordinary individuals. The technologies have been used with the intention of misleading others for a range of exploitative purposes that go beyond sexual abuse, although non-consensual pornography constitutes 96% of all deepfakes found online with 99.9% depicting women.
Controversies surrounding deepfakes involve issues of dehumanisation, defamation, copyright infringement, financial security, and the threats to national interests and political democracy. These concerns span moral, ethical, and legal domains.
The creation of deepfakes typically relies on abundant images from various angles, which is why public figures are often targeted due to their extensive online exposure. However, publicly available visual posts from social media and personal data leaks, including facial, physiological, and biometric information, can render everyday individuals vulnerable to deepfakes. These breaches occur through tech firms, law enforcement bodies, and government authorities.
In Australia, on 5 June 2024, Attorney General Mark Dreyfus introduced the Criminal Code Amendment (Deepfake Sexual Material) Bill 2024 (the Bill) to “ban the sharing of non-consensual deepfake sexually explicit material” by imposing “serious criminal penalties” of “up to 6 years imprisonment.” The Bill targets deepfake technologies while maintaining a broad scope covering both existing and emerging technologies especially with their potential implications for gender-based violence. Nevertheless, the Bill lacks clarity regarding the creation of sexualised deepfakes. The regulation focused on “distribution” may prove ineffective for such content circulated through encrypted messaging services like WhatsApp and Telegram. The legal judgement for “non-consensual” could impose a substantial burden on victims in substantiating their claims.
Australian laws do not currently prohibit political deepfake videos or phone calls, despite real AI-generated threats observed in elections in the US, Indonesia, and South Korea. The Combating Misinformation and Disinformation Bill primarily targets online platforms but does not specifically cover political campaigns from political parties, candidates, and supporters. Regulating disinformation propagated by domestic actors further raises concerns about freedom of speech and expression.
The technology neutrality approach prevails in this context. Legislation refrains from prescribing specific implementations for particular technologies, opting instead for flexibility and relevance to ensure existing laws remain adaptable and effective amidst technological change. To some extent, this approach allows room for malicious actors exploiting sophisticated technologies for adversarial purposes.
For cross-jurisdictional international actors, countries including Australia rely on platforms’ self-regulation. Major tech companies such as Meta, Google, Reddit, Twitter (now X), and ByteDance ban deepfakes through their content moderation mechanisms. However, these restrictions are often influenced by their business imperatives and advertising revenue rather than a genuine commitment to public good. Often, platforms maintain a strategic balance in their regulations, ensuring they are both flexible and restrictive to retain users and satisfy advertisers.
Regulating deepfakes poses a significant challenge due to their supply chain, from its production, distribution, and downstream deployment and use, spanning multiple jurisdictions and markets, both public and illicit. Effective regulation requires international collaboration, yet such efforts may face obstacles due to varying national interests and stakes concerning deepfake technologies.
Detection software developed by research institute and tech entities can exhibit inaccuracies in identifying both authenticity and AI-generated inauthenticity. Currently, methods of combating deepfakes are reactive rather than proactive, relying on voluntary efforts from citizens and civil organisations to enhance public AI literacy and establish mechanisms for screening and filtering deepfakes.
It is imperative for regulators to comprehend the broader operation of AI and the long-standing risks that are exacerbated by more sophisticated technologies. The emphasis on the economic efficiency and productivity generated by AI should not overshadow the profound concerns of fairness, safety, privacy and human rights, which these technologies can compromise.
