AI Voice and Video Impersonation: When You Can No Longer Trust What You See or Hear
Updated: Oct 5
For years, organizations have trained employees to be cautious about suspicious emails, unexpected links, and fraudulent text messages. Artificial intelligence is making that problem more complicated.
AI-generated audio can imitate familiar voices, while synthetic and manipulated video can make fraudulent communications appear to come from executives, coworkers, vendors, and other trusted contacts. The technology does not replace the familiar mechanics of social engineering. It changes some of the evidence people have traditionally relied on to decide whether a communication is legitimate.
In early 2024, one incident demonstrated how consequential that shift could become.
A finance employee in the Hong Kong office of global engineering firm Arup received a phishing message that appeared to come from the company's chief financial officer regarding confidential transactions. The employee was reportedly suspicious of the message. Then came what appeared to be stronger evidence.
The employee joined a group video conference in which the CFO and several apparent colleagues appeared on screen. Hong Kong authorities later said the conference had been fabricated using deepfake technology and publicly available video and audio material. The employee ultimately authorized transfers totaling approximately HK$200 million, roughly US$25.6 million.
The incident raises a difficult question:
What happens when seeing and hearing someone is no longer enough to establish that the person is actually there?
The Arup Fraud Changed the Verification Problem
The Arup case is particularly useful because the employee reportedly questioned the initial phishing message. The employee did not simply receive an email and immediately act on it. Suspicion existed. What appeared to resolve that suspicion was additional confirmation: a video conference involving people the employee believed they recognized.
That is what makes synthetic media important from a security perspective.
For years, an employee who doubted an email might reasonably have responded by calling the person or getting them on video to confirm the request. Those are still valuable steps when the employee independently initiates contact through a trusted channel. Simply moving from email to voice or video, however, no longer automatically establishes identity.
A familiar voice is evidence. A familiar face is evidence.
Neither should be treated as proof on its own.
The Arup fraud illustrates how an attacker can manipulate several trust signals at once. The initial message created the scenario. The apparent presence of familiar people provided additional social proof. Authority, familiarity, confidentiality, and urgency combined with synthetic media to make the request more convincing.
The technology may be new, but the psychology is not.
Deepfakes Are More Than a Detection Problem
It is tempting to treat AI impersonation as another detection challenge. Train employees to look for strange facial movements, listen for unusual pauses in speech, watch for visual inconsistencies, or learn the warning signs of synthetic audio.
Some awareness of deepfake techniques can be useful. But organizations should be careful about making employees responsible for reliably deciding whether sophisticated media is genuine.
Synthetic media will continue to improve, and an attacker does not need to create a flawless Hollywood-quality forgery. The impersonation only needs to be convincing enough in the situation where the employee is making a decision.
That changes the security question.
The second question leads to stronger controls.
If a convincing voice call alone can authorize a major payment, the organization is relying heavily on an employee's ability to determine whether that voice is authentic. If a video appearance by an executive can override an established procedure, the problem extends beyond synthetic media.
It becomes a process-design problem.
Familiarity Is Not Verification
Social engineering has always relied on trust. Attackers impersonate executives, vendors, customers, coworkers, IT personnel, and other people employees are accustomed to dealing with.
AI makes those impersonations more convincing because attackers can reproduce signals that once seemed difficult to fake: voice, appearance, speaking style, familiar terminology, and knowledge of organizational relationships.
An employee may recognize the voice, hear internal details, see the person on camera, and still be communicating with an attacker.
Those observations may provide context, but sensitive actions should not depend solely on them.
The objective of verification is not to determine whether a communication feels authentic. It is to establish that the requested action is genuinely authorized.
That distinction already matters in Business Email Compromise. A perfectly written email can still be fraudulent. A message sent from a compromised legitimate account can still request an unauthorized action. Synthetic media extends the same problem from text into audio and video.
HumanSecIQ explores the email side of this problem separately in AI-Enabled Business Email Compromise: Why the Old Warning Signs Aren't Enough.
The Human Decision Still Happens Inside a System
It would be easy to describe the Arup case as an employee being fooled by a deepfake. That explanation is incomplete.
The employee made a decision, but that decision happened inside a larger system. The attacker created a convincing scenario. The apparent authority of senior leadership mattered. The visual presence of familiar colleagues mattered. The financial process surrounding the transaction mattered. The organization's verification procedures and available escalation paths mattered too.
So did the employee's understanding of what was expected in an unusual situation.
Human behavior is part of the attack path, but it does not occur in isolation.
That is where human risk management becomes more useful than blame.
Organizations should assume that, eventually, someone will encounter a fraudulent communication convincing enough to pass their personal suspicion threshold. Security processes should be designed with that reality in mind.
Verification Should Be an Organizational Control
Independent verification is one of the strongest responses to impersonation risk because it does not require an employee to determine whether synthetic media is authentic.
For consequential actions, verification should occur through a channel or procedure that does not depend on the communication being verified.
If an unexpected call instructs an employee to transfer funds and provides a phone number for confirmation, calling that number does not create independent verification. The employee may simply be returning to the attacker.
A stronger process uses contact information or workflows the organization already trusts. That might mean independently contacting the requester through a previously verified number, using an established approval system, requiring another authorized person to approve a high-risk transaction, or validating unusual changes through a separate business process.
The appropriate controls will depend on the organization and the action involved, but the principle is straightforward:
Verification should be independent of the communication being verified.
The goal is not to eliminate trust from business. It is to prevent trust from becoming the only control protecting a consequential decision.
Security Culture Determines Whether Verification Works
A verification procedure on paper is not enough. Employees also need to feel authorized to use it.
Imagine an employee receiving what appears to be an urgent request from the CEO. The procedure says the employee should independently confirm the request before acting. But if the organization's culture teaches employees that senior leaders should not be questioned, the procedure becomes harder to use.
The same problem appears when slowing down a transaction is treated as poor performance or when employees fear embarrassment for raising a concern that turns out to be legitimate. In those conditions, the control may exist without being fully usable.
Security culture influences whether employees feel comfortable challenging unusual requests, delaying high-risk actions, escalating uncertainty, and following procedures even when the requester appears powerful or impatient.
Leaders can reinforce that culture by making verification normal. An employee should know that saying, "I need to verify this before I proceed," is not disrespectful.
It is the process working as designed.
That becomes even more important as attackers gain better tools for imitating authority.
Executive Impersonation Raises the Stakes
Senior executives are particularly valuable impersonation targets because authority can influence how employees interpret an unusual request.
A message appearing to come from a CEO or CFO might ask someone to process a payment, release confidential information, change banking details, grant access, or bypass a normal procedure. The request may also contain plausible explanations for why ordinary verification is inconvenient.
The executive is traveling.
The matter is confidential.
The transaction is urgent.
The employee should not involve anyone else yet.
None of those scenarios is new. What AI can change is the quality of the evidence supporting them.
A written message may be followed by a cloned voice. A call may be accompanied by synthetic video. A familiar face may appear to confirm exactly what the attacker wants the employee to believe.
The attempted impersonation of WPP CEO Mark Read in 2024 illustrates the broader pattern. Attackers reportedly combined a fake WhatsApp profile, publicly available imagery, cloned audio, and a Microsoft Teams interaction in an unsuccessful attempt to deceive a senior employee.
The attack did not succeed, but the technique matters. Attackers can combine multiple communication channels to reinforce the same false identity.
Moving from one channel to another therefore does not automatically create independent confirmation.
The Goal Is Not Perfect Deepfake Detection
Employees should understand that AI-generated audio and video can be used maliciously. That awareness has value. But the long-term defense cannot depend on every employee becoming skilled at forensic media analysis.
The technology will change too quickly.
A better objective is to design important business processes so that a convincing impersonation still cannot produce an unauthorized action on its own.
That means examining the decisions that matter most. Could one apparent executive authorize a major financial transfer? Can vendor payment information be changed based on a call or email alone? Could a help desk reset a powerful account after a convincing voice interaction? Could sensitive data be released solely because the requester appears familiar?
And if the answer to any of those questions is yes, what additional validation protects the action?
These questions move the organization away from trying to identify every fake and toward designing processes that remain reliable even when deception succeeds.
What Leaders Should Be Asking
AI-enabled impersonation creates a useful test of an organization's assumptions about trust.
Could employees independently verify a request that appears to come from an executive? Would they feel comfortable delaying that request long enough to verify it? Are high-risk financial actions protected by appropriate approval processes? Are changes to sensitive account or payment information independently confirmed?
Employees should also know where to escalate unusual communications, and social-engineering exercises should evolve beyond traditional phishing emails to reflect the ways attackers now combine text, audio, video, and multiple communication channels.
The most important question may be the simplest:
Could a convincing voice or video bypass a business control that should exist regardless of who appears to be making the request?
The answer reveals more than whether employees know what a deepfake is. It shows whether the organization is prepared to operate in an environment where familiar signals of identity are becoming less reliable.
Trust can begin the conversation. Verification should authorize the decision.
Comments