Building artificial intelligence (AI) capabilities from the ground up can be daunting, but the analytics expertise of a University of Technology Sydney (UTS) team helped Australian HR software developer Elmo Software climb the steep learning curve to build AI that predicts when employees are likely to leave the company.

Given that the company has a massive historical data set stretching back to 2002, CEO Danny Lessem told Information Age, it was clear early on that the company already had the data it would need to effectively train an AI for predicting employee attrition.

But building that capability was another thing altogether – which is why the company decided in 2019 to engage the university and its UTS Data Arena visualisation system.

As one of the strongest data sciences faculties in the southern hemisphere, Lessem explained, UTS has “fantastic equipment for data visualisations; in working closely with them, we were looking for patterns in incredibly large data sets.”

Those data sets included payroll and other HR details related to thousands of companies and millions employees across Australia and New Zealand, searching for cause-and-effect patterns related to factors like people’s duration in the organisation, or the consequences in terms of their performance.

“With that data visualisation, we started to ascertain these patterns and we could create hypotheses,” Lessem said.

Working closely with some of the company’s clients, the Elmo and UTS teams began testing the hypotheses “to see if there was credibility in the real world,” he continued, “and then fine-tuning them and going back and forth to get those hypotheses accurate.”

Even as the data engineers built and refined the hypothetical models for accuracy, a simultaneous stream of work was focused on building the algorithms that would ultimately be integrated into the company’s software-as-a-service (SaaS) suite as the newly launched Predictive People Analytics (PPA) module.

Through extensive iterative testing and refinement, the collaborative teams were able to test previous outcomes against preceding data across nearly two decades – comparing the overall data set with client-specific data sets to develop a cause-and-effect map that ultimately provided the confidence to launch the PPA module into production.

Potential correlations could come from anywhere in the data set maintained by the HR system – including someone who suddenly takes a lot of owing holiday leave, changes their work hours, increases their sick leave, and so on.

Testing the associations between these and other factors, and employee outcomes such as attrition, were fundamental to the iterative process of “ensuring that the hypothesis is accurate enough that it works on several small data sets with the same accuracy as the large data set,” Lessem explained.

“Once we got it accurate on the smaller data sets, we had the confidence to take it out, and we knew that we had a credible predictive analytics coming up for future data sets with other organisations outside the test bed.”

“But it takes a lot of work to get to that confidence.”

Think global, AI local

The importance of clean, consistent training data for AI models is well recognised, but in practice it can be difficult for companies to source and vet their data to the required degree – creating a trail littered with the corpses of poorly-defined AI failures.

This has made machine-learning model development exceedingly complex for many applications, keeping it out of the reach of many smaller software developers even as large businesses build their own AI teams and industry leaders call for greater investment in AI.

By partnering with UTS, Lessem said, Elmo Software’s developers were able to build their own ML skills while working towards a concrete deliverable – benefiting both Elmo’s code base and the real-world expertise of UTS doctoral candidates.

“Working closely with them has been fantastic for both us, and for the university and their doctoral program,” he explained. “Without access to their equipment, it would have been a lot more difficult to come up with those accurate hypotheses within such a short period of time.”

PPA’s go-live is just the beginning for Elmo and UTS, he said, flagging the significant opportunities to unleash further value from a data set that “is unique in being this first really credible predictive analytics tool in the region, based on big data from the region.”

“It’s no good if you’ve got a predictive analytics tool based on big data from, say, your US Fortune 500 companies,” he explained, “because it’s not going to be accurate in terms of your outputs and predictive analytics for organisations in this region.”

In guiding a business function as critical as employee retention, the difference can be significant – particularly because the focus of PPA is to help companies figure out which of their own critical talent might be contemplating an exit.

By flagging such changes early on, managers can focus attention on finding out what would keep the employee from leaving – or start training a replacement to ensure that a future defection doesn’t create a skills deficiency.

“Early on, organisations will be able to chart a really positive pathway for its top talent,” Lessem said, “so those employees – the ones that have had a direction towards high achievement – will find that pathway opens up quicker for them in terms of professional development, remuneration package, and promotion.”

“It’s a deliverable that assists all parties in that employment relationship. And the deeper you get into it, the more possibilities there are.”