Australian businesses may have embraced big-data strategies but more than half are struggling to keep up with the volume, variety, and velocity of data, according to new surveys highlighting just how much poor data management is costing.

Fully 15 per cent of 2100 surveyed IT decision makers said they aren’t coping well with the explosion of data and a further 45 per cent said they were coping but were having “some issues”, Cloudera’s new Enterprise Data Maturity Research Report found.

Similar proportions reported issues around the rest of the ‘5 Vs’ of data – variety, veracity, velocity, and value of data – with 90 per cent of respondents admitting that their organisation would be generating more revenues if it could figure out how to manage data more effectively.

Just half of respondents said their organisation has had an enterprise data strategy in place for more than a year, with a third saying they had implemented such a strategy in the past year and 14 per cent still working on a strategy.

Although sectors such as financial services, retail, and telecommunications had become “extremely data savvy”, Cloudera vice president and managing director for Australia & New Zealand Robert Yue said, Australian organisations were behind global benchmarks when it came to turning data strategies into measurable business outcomes.

“The challenge now,” Yue said, “is how to do more with data as we focus our sights on the anticipated economic upswing of 2022 despite the continued challenges of COVID-19.”

“We’re seeing a significant behavioural shift related to the broader industry trend that ‘every company is a software company’,” he continued, noting moves such as Commonwealth Bank of Australia’s investments in AI firm H2O.ai and property-tech startup Own Home.

“Many of our customers are finding innovative ways to overcome pain points like speed, to anticipate and stay ahead of their own customer needs.”

Fully 79 per cent of respondents admitted that they have not yet centralised and integrated their big-data infrastructure across the organisation – compared with 65 per cent globally – and 71 per cent said they have not yet “democratised” their systems by providing access to analytics tools to all employees.

Better data means better business

Despite pandemic-era pressures to increase digital transformation and improve the use of data, many companies still aren’t good enough at collecting and analysing their data.

Fully 87 per cent of respondents to Experian’s latest Global Data Management Research Report said they had become more reliant on quality data and insights, with 83 per cent flagging the importance of AI-driven ‘DataOps’ strategies to integrate data into everyday decision-making.

Although widespread access to good-quality data is closely linked to better decision-making, 72 per cent of respondents said they have so much data that it’s hard to see where better data management can add the most value.

“The last year has tested every industry, with a new requirement for business models to be agile and change in line with their customers’ rapidly shifting demands,” said Andrew Abraham, Experian’s global managing director for data quality.

“Businesses who have improved their data quality were not just better equipped for this but exceeded their performance expectations.”

That said, many companies are struggling to get the skills they need to actually improve data quality – with 84 per cent admitting that skills deficiencies are hampering organisational agility and flexibility.

“Business experiences with data accuracy, and issues around how data is managed, remain and are unlikely to improve,” Abraham said, “unless businesses upskill current employees and continue to work with wider industry and government on addressing the data skills gap.”

Democratisation of data was a critical way of closing this gap by leaning on cleaner, more-accessible data to help every employee get the insights they need.

Making this happen is still proving challenging, however: more than a third of respondents to a recent InterSystems study said they weren’t satisfied with their current data management technology stack, and 55 per cent said they would prioritise technologies for building data-driven organisations this year.

Companies like Allianz Australia, Cloudera’s Yue said, are addressing this deficiency by focusing on building internal culture and “empowering their people to become data champions.”

“This, in turn, supports their customers through access to new levels of actionable data and information about their portfolio. They are a great example of an organisation getting it right.”