Companies are racing to embed AI into everything from customer service to pricing and hiring.
But if they're not telling people exactly how those systems make important decisions, experts warn they could soon fall foul of new Australian privacy laws.
From 10 December, businesses using automated decision making (ADM) systems that significantly affect people's rights or interests will have to be far more transparent about how those systems work.
The changes, introduced through amendments to the Privacy Act 1988, are designed to stop AI becoming a 'black box' that makes life-changing decisions without people understanding how or why.
Under the new rules, organisations will need to publicly explain: what types of personal information their AI uses; what decisions are made solely by computer programs; and which decisions rely heavily on automated systems.
The Office of the Australian Information Commissioner (OAIC), which consulted on the reforms earlier this year, said the new transparency obligations would help consumers understand when automated decisions are being made and make it easier to challenge or review those decisions under other laws, including anti-discrimination legislation.
Although the final rules have not yet taken effect, Veronica Scott, partner at international law firm Pinsent Masons, said businesses should already be preparing.
"The new obligations are an important step in Australia's evolving privacy and AI regulatory landscape," she said.
Scott warned many everyday digital practices could fall within the rules, including targeted advertising, personalised pricing and algorithmic content curation that could influence access to employment opportunities.
She also noted that a decision can still have a "significant effect" even if it benefits someone, such as giving them faster access to a service.
The new plain-language requirements, she said, signal the OAIC expects organisations to move beyond vague privacy policies and provide meaningful explanations of how automated decisions are made.
Are we giving AI too much control?
The changes reflect growing concern over AI systems making important decisions with little transparency.
Early AI tools exposed problems ranging from gender bias in recruitment to threats to doctor-patient confidentiality and concerns about automated policing.
Australia has already seen the consequences of opaque automated decision making through the Robodebt scandal, while an OAIC review earlier this year found government agencies still needed to improve transparency around their automated systems.
Parliament has also warned that poorly designed workplace AI could lead to excessive collection of personal information, increased employee surveillance and automated systems overriding human judgement.
More recently, a government committee recommended delaying AI-driven eligibility assessments for the NDIS, while automated aged-care assessment tools have also attracted criticism.
Those concerns have only intensified as generative AI becomes mainstream.
Global AI spending is expected to jump 63.4 per cent this year to US$64 billion, according to Gartner, as businesses rapidly expand their use of AI platforms and generative AI models.
Australian organisations are embracing the technology just as quickly.
Around 85 per cent have provided staff with enterprise AI tools, yet 56 per cent of employees remain resistant to using AI and roughly half say they have received no guidance on its use.
Aligning with global standards
The new rules also bring Australia closer to international AI governance standards.
Initiatives such as the Hiroshima AI Process and the OECD AI Principles both encourage organisations to explain, in plain language, what data AI systems use and the reasoning behind their decisions.
Until now, those principles have largely been voluntary.
From December, however, transparency around automated decision making will become a legal obligation rather than best practice.
That deadline is closer than many businesses realise.
"Most business owners I speak with think AI regulation in Australia is still years away," finance industry AI and automation expert Amjid Ali wrote.
"But on the transparency of automated decisions, it is not.
"There is a hard date, it applies to ordinary businesses, and the clock is already running."
