AI Fraud and Cybercrime: How Criminals Are Using Deepfakes, Voice Cloning and Generative AI
Quick Answer: Artificial intelligence can be used to make phishing, impersonation, financial fraud, identity theft and other cybercrimes more convincing and scalable. The use of AI does not necessarily create a completely new criminal offence. In many cases, existing laws concerning fraud, identity theft, unauthorised access, extortion, financial crime and computer misuse can apply to AI-assisted conduct.
A company's chief executive appears to send a video instructing an employee to transfer money.
The employee recognises the face.
The voice sounds authentic.
The instruction appears urgent.
But the executive never sent the message.
The video was generated using artificial intelligence.
In another case, a criminal clones a person's voice and calls a family member asking for an emergency payment.
In another, an AI system generates thousands of convincing phishing emails in multiple languages.
In yet another, criminals use synthetic identities to open financial accounts or evade identity-verification systems.
These are no longer purely theoretical risks.
Generative AI can reduce the cost of producing convincing fraudulent communications while increasing the scale at which criminals can operate.
The legal question is therefore becoming increasingly important:
When AI becomes the tool used to commit fraud or cybercrime, which laws apply and who can be held responsible?
The answer usually begins with existing criminal law.
Fraud remains fraud even when AI is used to generate the deception.
Identity theft remains identity theft even when the impersonation is created using a voice clone.
Unauthorised access to a computer system does not become lawful merely because AI helped the attacker.
At the same time, AI introduces new questions concerning attribution, evidence, intent and platform responsibility.
Legal disclaimer: This article provides general educational information and is not legal advice. Cybercrime laws vary significantly between jurisdictions.
Key Takeaways
- AI can significantly increase the scale and sophistication of cybercrime.
- Deepfakes can be used for impersonation, fraud and extortion.
- Voice cloning can facilitate payment fraud and social engineering.
- Generative AI can assist phishing and fraudulent communications.
- Synthetic identities can create risks for financial institutions and digital services.
- Existing fraud and computer-crime laws can apply to AI-assisted offences.
- AI-generated content can make attribution more difficult.
- Businesses should not rely solely on facial or voice recognition for high-risk transactions.
- Multi-factor authentication and independent verification remain important controls.
- Financial instructions involving large transfers should use verification procedures that do not depend on a single communication channel.
- Preserving digital evidence is critical when investigating AI-enabled fraud.
- AI providers and platforms can face separate regulatory and compliance questions from the criminals using their systems.
What Is AI-Powered Cybercrime?
Quick Answer: AI-powered cybercrime refers broadly to criminal conduct in which artificial intelligence is used to facilitate, automate, scale or enhance an unlawful cyber activity.
Examples include:
- Phishing.
- Social engineering.
- Voice cloning.
- Deepfake impersonation.
- Identity theft.
- Financial fraud.
- Malware development.
- Credential theft.
- Extortion.
- Online scams.
The phrase does not necessarily mean that AI itself is committing a crime.
The criminal actor remains the person or group using the technology.
Why Is AI Changing Cybercrime?
Quick Answer: AI can lower the cost of producing convincing fraudulent material, automate repetitive tasks and help criminals personalise attacks at scale.
Traditional phishing campaigns may contain obvious spelling errors.
AI can generate polished messages.
Traditional scams may be limited by language barriers.
Generative AI can produce content in multiple languages.
Traditional impersonation may rely on text.
AI can add realistic voice and video.
The result is a potentially more persuasive attack.
What Is Deepfake Fraud?
Quick Answer: Deepfake fraud occurs when AI-generated or manipulated images, video or audio are used to deceive someone for financial or other unlawful gain.
A criminal may impersonate:
- A company executive.
- A family member.
- A bank employee.
- A government official.
- A business partner.
- A celebrity.
The objective is usually not the deepfake itself.
The deepfake is the mechanism of deception.
What Is Voice-Cloning Fraud?
Quick Answer: Voice-cloning fraud uses AI to reproduce characteristics of a person's voice and then uses the synthetic voice in a deceptive communication.
A typical attack can look like this:
- The criminal obtains publicly available voice recordings.
- An AI system is used to create a synthetic voice.
- The criminal contacts the victim.
- The victim recognises the voice.
- The criminal creates an urgent situation.
- The victim transfers money or reveals information.
The technology exploits a basic human assumption:
“I recognise the voice, therefore I know who is speaking.”
That assumption is becoming unreliable.
Can Voice Cloning Be Used for Financial Fraud?
Quick Answer: Yes. Voice cloning can be used in social-engineering attacks designed to induce victims to make payments, disclose credentials or perform other actions.
A common corporate scenario involves an apparent executive instructing an employee to make an urgent payment.
Businesses should therefore avoid treating voice recognition as sufficient authentication for high-value transactions.
What Is Business Email Compromise?
Quick Answer: Business email compromise is a form of fraud in which criminals compromise or impersonate business communications to induce victims to transfer money or disclose information.
AI can potentially strengthen these attacks by helping criminals:
- Generate convincing messages.
- Imitate communication styles.
- Translate communications.
- Personalise messages.
- Respond automatically to victims.
AI therefore does not necessarily replace traditional business-email-compromise techniques.
It can make them more effective.
What Is AI-Generated Phishing?
Quick Answer: AI-generated phishing involves using artificial intelligence to create or customise deceptive communications designed to trick recipients into revealing information, transferring money or clicking malicious links.
Potential targets include:
- Employees.
- Customers.
- Executives.
- Government officials.
- Financial institutions.
AI can also make phishing messages appear more grammatically polished and contextually relevant.
Can AI Make Phishing More Dangerous?
Quick Answer: Yes. AI can help attackers automate content generation and personalise communications, potentially increasing the effectiveness and scale of phishing campaigns.
However, organisations should not assume that AI makes every attack sophisticated.
Traditional security controls remain effective against many forms of phishing.
What Is Synthetic Identity Fraud?
Quick Answer: Synthetic identity fraud involves creating or combining identity attributes to construct an identity that does not correspond to a genuine person in the way it appears.
AI can potentially assist criminals by generating:
- Fake photographs.
- Fake documents.
- Fake voices.
- Fake biographies.
- Fake social-media profiles.
This creates significant risks for organisations that rely on remote identity verification.
AI and Identity Theft
Quick Answer: AI can facilitate identity theft by helping criminals imitate a victim's appearance, voice or personal characteristics.
Potential consequences include:
- Unauthorised account access.
- Financial fraud.
- Fraudulent account creation.
- Social-engineering attacks.
- Reputational harm.
Identity protection therefore increasingly requires more than passwords.
Can AI Be Used to Bypass Biometric Verification?
Quick Answer: AI-generated images, video and other synthetic media can create risks for remote biometric verification systems, although modern identity-verification systems increasingly use liveness detection, presentation-attack detection and other controls.
NIST's current digital identity guidance recognises the threat posed by forged media and requires controls addressing presentation attacks and forged media in applicable remote identity-proofing contexts. ([pages.nist.gov](https://pages.nist.gov/800-63-4/sp800-63a/ial-general/?utm_source=chatgpt.com))
Businesses should therefore treat biometric verification as one layer of authentication rather than an absolute guarantee of identity.
AI and Malware
Quick Answer: AI can potentially assist malicious actors with aspects of malware development, code generation, vulnerability research and attack automation, although safeguards and technical barriers can limit these capabilities.
AI-related cyber risk is therefore broader than deepfakes.
It can include the use of AI throughout different stages of an attack.
Can AI Create Malware?
Quick Answer: AI systems can generate or modify code, which creates potential cybersecurity risks when malicious actors attempt to use those capabilities for malware development or exploitation.
However, the legal issue remains focused on the underlying conduct.
If a criminal uses AI to create malicious software and deploys it against a victim, existing computer-crime laws may apply to the resulting conduct.
AI and Ransomware
Quick Answer: AI can potentially assist ransomware operations by helping criminals automate reconnaissance, communication, social engineering or other attack activities.
Ransomware remains a conventional form of cybercrime in many respects.
The use of AI may simply change how efficiently an attacker performs particular steps.
What Criminal Laws Apply to AI Fraud?
Quick Answer: Depending on the conduct and jurisdiction, AI-assisted fraud can implicate laws concerning fraud, wire fraud, identity theft, computer misuse, unauthorised access, extortion, money laundering and other offences.
In the United States, potential federal offences can include:
- Wire fraud.
- Computer fraud.
- Identity theft.
- Unauthorised computer access.
- Money laundering.
- Extortion.
The exact offence depends on the conduct and statutory elements.
What Is Wire Fraud?
Quick Answer: U.S. federal wire-fraud law generally addresses schemes to defraud involving interstate or foreign wire communications.
AI-generated emails, voice communications or video messages could potentially become instruments of such a scheme.
The important point is that the AI technology does not necessarily create the underlying offence.
It may be the mechanism used to execute the fraudulent scheme.
Can Deepfake Fraud Lead to Criminal Liability?
Quick Answer: Yes. If a deepfake is used as part of a criminal scheme, the person creating or deploying it may face criminal liability under applicable laws.
Potential conduct can include:
- Fraud.
- Identity theft.
- Extortion.
- Harassment.
- Financial crimes.
- Computer offences.
What About AI-Generated Extortion?
Quick Answer: AI can be used to create threatening or fabricated material for extortion or blackmail, including synthetic intimate imagery or fabricated evidence.
The legal analysis depends on the nature of the threat and the applicable criminal law.
The fact that the threatened material is synthetic does not necessarily prevent criminal liability.
AI Fraud Against Financial Institutions
Quick Answer: Financial institutions face particular risks because AI can be used to impersonate customers, generate fraudulent documents or manipulate remote identity-verification processes.
Potential attack vectors include:
- Account opening.
- Account takeover.
- Payment authorisation.
- Loan applications.
- Customer-service impersonation.
- Fraudulent identity documents.
AI Fraud Against Businesses
Quick Answer: Businesses are particularly vulnerable to attacks targeting employees who have authority to approve payments, release information or access systems.
High-risk scenarios include:
- Fake CEO instructions.
- Fake supplier communications.
- Fake bank communications.
- Fake lawyers.
- Fake government officials.
The solution is procedural as much as technological.
Why Human Verification Matters
Quick Answer: Independent verification can prevent a deepfake or voice clone from becoming sufficient to authorise a high-risk transaction.
For example, an employee receiving a voice call requesting an urgent payment can independently call the executive using a verified number.
The employee should not simply call the number provided by the suspected attacker.
What Is a Callback Verification Procedure?
Quick Answer: Callback verification requires the recipient of a high-risk instruction to independently contact the alleged sender through a trusted communication channel.
For example:
- Employee receives payment instruction.
- Instruction appears to come from CFO.
- Employee does not rely on the email or voice message.
- Employee contacts CFO through the company's verified directory.
- CFO confirms or rejects the instruction.
This simple control can defeat many impersonation attacks.
AI Cybercrime and Evidence
Quick Answer: AI-enabled cybercrime creates significant evidence challenges because investigators may need to establish both the fraudulent communication and the technical process used to create or distribute it.
Relevant evidence can include:
- Emails.
- IP addresses.
- Server logs.
- Device records.
- Payment records.
- Chat logs.
- AI prompts.
- Generated files.
- Metadata.
- Blockchain transactions.
This connects directly with the digital-evidence issues discussed in Article #61.
Can AI Fraud Be Detected?
Quick Answer: AI fraud can sometimes be detected through a combination of technical systems, behavioural analysis and human verification, but no single detection method is guaranteed to identify every AI-enabled attack.
Organisations should therefore use layered controls.
AI Fraud Prevention Framework
| Threat | Recommended Control |
|---|---|
| Voice cloning | Independent callback |
| Deepfake video | Multi-factor verification |
| AI phishing | Email security and employee training |
| Account takeover | Strong authentication |
| Synthetic identity | Layered identity verification |
| Payment fraud | Dual authorisation |
| Fake supplier | Independent payment verification |
| AI-generated documents | Document authenticity checks |
Should Businesses Ban Generative AI?
Quick Answer: A complete ban is not necessarily the most effective response. Businesses should instead identify high-risk uses and establish proportionate governance and security controls.
Generative AI can provide legitimate benefits.
The objective should be controlled use rather than assuming that all AI is inherently dangerous.
AI Cybersecurity Policy for Employees
Quick Answer: Organisations should establish clear employee policies concerning AI-generated communications and high-risk transactions.
Employees should be instructed:
- Never to approve a major payment solely on the basis of a voice or video.
- To verify unusual instructions independently.
- To report suspected impersonation.
- Not to share confidential information with unapproved AI systems.
- To treat unexpected urgency as a potential fraud indicator.
AI Fraud Incident Response
Quick Answer: Organisations should preserve evidence immediately after discovering suspected AI-enabled fraud.
An incident-response process should include:
- Stop further transactions.
- Secure affected accounts.
- Preserve emails and messages.
- Preserve call recordings where lawful.
- Document the fraudulent communication.
- Identify affected systems.
- Notify relevant internal teams.
- Assess legal reporting obligations.
- Contact financial institutions where appropriate.
- Preserve evidence for law enforcement.
Can AI Providers Be Responsible for AI Crime?
Quick Answer: Criminal responsibility generally focuses on the conduct and intent of the person committing the offence, while providers may face separate questions concerning product design, safety, regulatory compliance or other legal duties.
The fact that criminals use a legitimate AI service does not automatically make the provider criminally responsible for their conduct.
At the same time, providers may have legal responsibilities depending on the jurisdiction and the nature of their services.
AI Crime and Platform Responsibility
Quick Answer: Platforms can face separate legal and regulatory obligations concerning fraud, harmful content, cybersecurity and user safety, depending on the applicable jurisdiction.
Businesses should distinguish between:
- The criminal actor.
- The AI provider.
- The platform.
- The victim.
Each may occupy a different legal position.
AI Fraud and International Cybercrime
Quick Answer: AI-enabled cybercrime can cross borders easily because attackers, infrastructure, victims and financial accounts may be located in different countries.
This creates challenges involving:
- Jurisdiction.
- Extradition.
- Evidence preservation.
- Cross-border investigations.
- International cooperation.
- Asset recovery.
A victim in one country may be attacked from another country using infrastructure located in a third.
Why Attribution Is Difficult
Quick Answer: Attribution can be difficult because attackers may use compromised accounts, anonymisation technologies, cryptocurrency, synthetic identities and infrastructure located across multiple jurisdictions.
AI adds another layer.
Investigators may need to determine:
- Who operated the account?
- Who generated the content?
- Who controlled the infrastructure?
- Who received the money?
- Who benefited from the fraud?
AI Cybercrime and Criminal Intent
Quick Answer: Criminal intent remains important because many offences require proof of a particular mental state.
The investigator therefore needs to establish more than:
“AI was used.”
The investigation may need to demonstrate:
“The accused intentionally used AI as part of the unlawful scheme.”
Evidence may include:
- Prompts.
- Messages.
- Search history.
- Financial records.
- Device data.
- Communications with accomplices.
AI Fraud Compliance Checklist
| Control | Purpose |
|---|---|
| Multi-factor authentication | Reduce account compromise |
| Independent payment verification | Reduce impersonation fraud |
| Callback procedures | Verify urgent instructions |
| Employee training | Improve detection |
| Identity verification | Reduce synthetic identity risk |
| Email security | Reduce phishing |
| Transaction monitoring | Identify unusual payments |
| Incident response | Limit damage |
| Evidence preservation | Support investigation |
| AI governance | Control internal AI use |
Frequently Asked Questions
What is AI fraud?
AI fraud refers broadly to fraudulent conduct in which artificial intelligence is used to facilitate, automate or enhance deception.
What is AI cybercrime?
AI cybercrime refers broadly to cybercriminal activity in which artificial intelligence is used as a tool to facilitate or scale unlawful conduct.
Can criminals use AI to commit fraud?
Yes. AI can assist with impersonation, phishing, voice cloning, deepfake creation, synthetic identities and other fraudulent activities.
What is a voice-cloning scam?
A voice-cloning scam uses AI-generated speech that imitates a person's voice to deceive a victim.
Can a deepfake be used to steal money?
Yes. Deepfake video and audio can be used to impersonate executives, family members or other trusted individuals in financial scams.
Can AI be used for phishing?
Yes. Generative AI can assist criminals in producing personalised or multilingual phishing communications.
Can AI be used for identity theft?
Yes. AI-generated images, voices, documents and synthetic identities can facilitate identity-related fraud.
Can AI bypass biometric verification?
AI-generated media can create risks for remote biometric verification systems, although modern systems use liveness detection and other anti-spoofing measures.
Is AI fraud a new criminal offence?
Not necessarily. Many AI-assisted crimes can fall within existing offences such as fraud, identity theft, computer crime and extortion.
Can businesses prevent AI fraud?
Businesses can significantly reduce risk through layered controls such as multi-factor authentication, independent payment verification, employee training and transaction monitoring.
Should employees trust a CEO's voice?
No high-risk transaction should rely solely on voice recognition. Employees should independently verify unusual or urgent financial instructions.
What should a company do after a deepfake fraud attack?
The company should stop further transactions, preserve digital evidence, secure affected accounts, assess legal obligations and investigate how the fraudulent communication was created and distributed.
Can AI-generated fraud be prosecuted?
Potentially. If the underlying conduct satisfies the elements of a criminal offence, the use of AI does not necessarily prevent prosecution.
Can AI companies be criminally liable for crimes committed by users?
The answer depends on the provider's conduct, knowledge, intent and applicable law. Merely providing a legitimate AI tool does not automatically make the provider criminally responsible for every unlawful use by a third party.
Can AI help criminals create malware?
AI can potentially assist malicious actors with coding and other technical tasks, although security controls and model restrictions may limit those capabilities.
How can banks protect against deepfake fraud?
Banks can use layered identity verification, transaction monitoring, behavioural analytics, liveness detection and independent confirmation procedures for high-risk activity.
What is the biggest AI fraud risk for businesses?
There is no single universal risk. High-value impersonation, payment fraud, account takeover, phishing and synthetic identity attacks are among the significant risks organisations should assess.
Conclusion
Artificial intelligence has not invented fraud.
It has changed the economics of fraud.
A criminal no longer necessarily needs professional video-production skills to create a convincing impersonation.
A criminal may not need to speak the victim's language.
A criminal may not need to write a convincing corporate email manually.
AI can assist with all of these activities.
That creates a significant cybersecurity and legal challenge.
But businesses should avoid responding to the problem with panic.
The most effective controls are often straightforward.
- Verify high-risk instructions independently.
- Use multi-factor authentication.
- Require dual approval for major transactions.
- Train employees to recognise synthetic impersonation.
- Monitor unusual transactions.
- Preserve evidence quickly.
- Maintain an AI-use policy.
The legal framework is also not starting from zero.
Fraud laws already prohibit deceptive schemes.
Computer-crime laws already address unauthorised access and certain forms of computer misuse.
Identity-theft laws address misuse of another person's identity.
Financial-crime laws address fraudulent transactions.
AI becomes the tool through which these offences may be committed.
Attribution, however, is becoming harder.
Investigators must increasingly determine not only what happened but also whether digital communications, voices, images and documents were generated or manipulated by artificial intelligence.
This makes the evidentiary issues discussed in Article #61 particularly important.
Deepfake evidence may need to be authenticated.
AI-generated communications may need to be traced.
Prompts and model outputs may become relevant evidence.
Metadata and device records may help establish provenance.
The result is a new convergence between cybersecurity, criminal law and digital forensics.
The central legal lesson is simple: AI does not create a new moral category of crime. It creates a more powerful technological instrument for committing existing crimes—and forces the legal system to become better at attribution, evidence and prevention.
Legal Disclaimer
This article is provided for general educational and informational purposes only. It is not criminal, cybersecurity, regulatory or legal advice and does not create an attorney-client relationship. Cybercrime laws differ between jurisdictions. Businesses and individuals facing an actual cyber incident should obtain appropriate legal and cybersecurity assistance.