OPINION
Your phone lights up with your bank’s name.
The caller says there has been suspicious activity on your account and you must act immediately. You challenge them and say you will call the bank yourself.
Instead of becoming defensive, the caller softens.
“That is exactly what you should do,” they say. “I just want to keep your money safe.”
A second “fraud specialist” joins the call. They know your name and use the same reassuring language as your bank.
The scam has changed because you resisted.
Australia’s scam education still focuses on what a scam looks like: poor grammar, strange links, generic greetings and suspicious websites.
But AI-enabled scams may look polished, sound professional and adjust their tactics while the conversation is still happening.
We now need to teach people not only to spot suspicious messages, but to recognise what a conversation is trying to make them feel and do.
From social engineering to emotional engineering
Traditional scams often follow a fixed script. An AI-assisted scam can be more flexible.
It may generate different replies, remember details disclosed earlier and shift from urgency to reassurance, from authority to affection, or from fear to guilt.
The old scam repeated the script. The new scam can rewrite it.
Scamwatch already warns that scammers use impersonation, emotion and urgency. What is changing is the speed and scale at which these tactics can be personalised.
A useful, although imperfect, metaphor is the ‘dark empath’, someone who understands another person’s emotions but uses that understanding to manipulate rather than help.
AI is not empathetic. It does not care whether someone is frightened, lonely or confused.
But it can identify patterns in language, tone and behaviour and produce a response designed to influence what happens next.
The system does not need to understand you. It only needs to predict what you might do.
How emotional adaptation works
Think of sentiment analysis as a mood detector. It may interpret hesitation, word choice or changes in tone as signs of fear, suspicion or excitement.
Conversational memory acts like a notebook that remembers what the target has revealed. Generative AI becomes the scriptwriter, producing the next line in real time.
If fear stops working, the scam may switch to reassurance. When a victim says, “This sounds suspicious,” the caller might reply, “You are right to be careful. I’ll transfer you to my senior fraud supervisor.”
The fake supervisor adds authority and makes the story appear independently verified. They may repeat personal details and insist that immediate action is required.
Scammers may also create guilt by saying, “I am trying to protect your account, but I cannot help unless you follow these steps,” or, in a family impersonation scam, “Please do not tell anyone. I need you to help me.”
The aim is to keep the victim engaged by combining reassurance, authority, secrecy and emotional pressure.
Europol and Australian cyber authorities have warned that generative AI, deepfakes and synthetic voices can increase the credibility and personalisation of impersonation attempts.
Research has also demonstrated that AI agents can conduct multi-turn scam conversations and react to new information.
That does not mean fully autonomous emotional scam engines are already everywhere.
Human scammers remain central to many operations.
But the direction is clear: scams are becoming less like static messages and more like adaptive conversations.
Why traditional scam awareness is no longer enough
For years, people were told to look for spelling mistakes.
Generative AI has quietly retired that warning as a reliable defence.
Bad grammar and clumsy websites may still expose some scams, but polished language is now cheap.
A message can imitate a bank, manager or family member in seconds.
The more important clues may be behavioural:
· Why am I being rushed?
· Why am I being told to keep this secret?
· Why does this person object to independent verification?
Scammers can create cognitive overload through rapid instructions, threats and multiple apparent authorities.
The aim is to keep the person emotionally occupied long enough to prevent calm verification.
This is why victim-blaming is so damaging. Scam victims are not necessarily careless or technically inexperienced.
These techniques exploit normal human responses to fear, authority, trust, compassion and uncertainty.
Teaching people to interrupt the emotional process
Australia should redesign scam education around interruption, not recognition alone.
A simple model is: Stop. Separate. Verify.
Stop when a conversation creates urgency, fear, secrecy or emotional pressure.
Separate from the channel. End the call, close the message or step away.
Verify through a trusted source, such as the number printed on a bank card, an official application, a known colleague or a family member contacted independently.
Organisations also need to make safe verification easier.
Banks, government agencies and technology companies should design processes that tolerate delay and encourage second-person checks before money or credentials are transferred.
Workplace training should include adaptive scenarios rather than predictable phishing quizzes.
Incident reports should capture not only the malicious link or number, but also the emotional tactics used: urgency, reassurance, authority, isolation or guilt.
Public messaging must also avoid implying that victims failed an intelligence test.
A scam is a deliberately engineered environment designed to narrow attention and accelerate action.
The next generation of scams may not look suspicious.
It may sound patient, informed and unusually understanding.
That is why the most important security control may be a human pause: the moment we step outside the conversation, interrupt its emotional momentum and verify the request independently.
