A new artificial intelligence system is being built to understand emotions and help emergency phone line operators monitor a caller’s distress in real time.

Dr Rajib Rana, a computer scientist with the University of Southern Queensland, is getting $300,000 to develop the AI system that builds upon years of research into how machine learning can be used to sense a person’s emotions through their speech.

Far from taking jobs away from mental health and emergency service workers, Dr Rana wants to make their jobs easier and more effective.

“My vision is for this to become a complementary tool for clinicians – kind of like how a thermometer provides information for doctors and nurses,” he told Information Age.

“When someone is distressed they experience physiological changes that actually alters their speech production; that is the scientific foundation of this research.

“So we are developing techniques to measure those changes in real-time and with a very high accuracy.”

The system will be trained using a “guided unsupervised representation learning” method leveraging massive publicly available datasets which comprise human speech samples.

These audio samples have been labelled for their emotional features and will teach the machine learning algorithm how to recognise the digital patterns of certain emotions in speech.

Dr Rana’s preliminary results have been promising and he expects it to have a strong impact on clinical settings – especially for weeding out hoax calls in emergency services hotlines.

“Time is precious when responding to calls that are life-and-death situations, but unfortunately about one-third of calls made to emergency helplines are hoax calls, which waste valuable resources and place lives at risk,” Dr Rana said.

“Health services experience a large number of mental health service requests, but this distress interference system will enable call operators to effectively manage high volumes of calls, provide earlier intervention and assist in faster ambulance deployment times to potentially save lives.”

Though still in the research and development stage, Dr Rana has ideas about how the tool could be used to augment the important work of emergency and mental health phone line operators.

“It could be that the operator has a window open on their screen during a call that gives them a live measurement of the caller’s distress level,” Dr Rana told Information Age.

“They could use that information to make further decisions about referring the caller to other services or even if there is a need to dispatch emergency services.

“The tool could even be used as a first-order triage which will be helpful for mental health lines that have only a limited number of trained personnel but a high volume of callers.”

COVID-19 has put added pressure on mental health services with Lifeline noting a 25 per cent increase in its number of calls in March.

Funding for the AI project came through an Advance Queensland COVID-19 Industry Research Fellowship.

While the system won’t be in use to help clinicians during the current pandemic, Dr Rana is confident it will improve services in the future.

“The advantage of this project is not only can it prepare for future health crises, but it is also well-positioned to help prepare for other future crises known to cause distress in people, such as bushfires, droughts and floods,” he said.

“We’ve already received interest from the Queensland Police Service, Lifeline and Cancer Council Queensland on trialling the system when it is developed.”