Australians should have a legal right to refuse the use of AI scribes during medical appointments without being denied care, as a digital rights watchdog warns the technology is being rolled out across the healthcare system faster than regulators can keep up.

A new discussion paper from Digital Rights Watch argues that AI scribes – which use generative AI to record, transcribe and summarise consultations – should be regulated as medical devices and independently tested for accuracy and bias before they become entrenched in healthcare.

The warning comes as adoption of the technology accelerates.

Research cited in the report shows the proportion of doctors using AI scribes more than doubled from 22 per cent in August 2024 to 40 per cent by late last year.

This week, digital mental health platform Innowell announced it had integrated AI healthcare scribe Heidi, saying the technology could give mental health workers up to five hours back each week by reducing administrative work.

But Digital Rights Watch said the efficiency gains are being prioritised over patient rights, privacy and safety.

"AI scribes are rapidly becoming embedded in the Australian healthcare system, by claiming to ease administrative burdens and the workloads of GPs and medical practitioners across the country," the report said.

"While these tools may offer benefits in terms of efficiency, the risks in respect of privacy, the potential for bias, and the impact on patient-doctor relationships must be appropriately considered and mitigated."

Patients turned away

A central recommendation of the paper is that patients should have a statutory right to refuse the use of AI scribes while still receiving treatment.

The recommendation follows several high-profile incidents in Australia where patients who declined the technology were effectively denied care.

Medical AI scribe Heidi has been integrated into digital mental health platform Innowell. Photo: Shutterstock

In August last year, an Australian AI expert refused consent for AI transcription software to be used during her child's medical appointment and was told she was "welcome to seek an alternative provider".

Earlier this year, it also emerged that a Melbourne psychiatrist refused to see a patient who would not consent to an AI scribe and instead advised them to seek a referral elsewhere.

Digital Rights Watch said those cases demonstrate why consent alone is insufficient if patients risk losing access to healthcare.

Regulation failing to keep pace

The advocacy group argues the rapid uptake of AI scribes has outpaced Australia's regulatory framework, leaving significant gaps around privacy, safety and accountability.

"The current regulatory environment in Australia is inadequate relative to the serious nature of the use case, and the system risks of the technology have neither been considered nor addressed," the report said.

It is calling for AI scribes to be regulated as clinical devices, with mandatory testing for bias and accuracy.

Currently, AI scribes are generally only regulated as medical devices if they provide diagnoses or treatment recommendations.

In England, however, AI scribes that perform summarisation functions are already classified as medical devices.

The report also found widespread failures in transparency.

After reviewing the privacy policies of 60 Australian healthcare providers known to use AI scribes, Digital Rights Watch found only four clearly disclosed their use of the technology.

Among its recommendations, the organisation called for the Office of the Australian Information Commissioner to issue specific guidance on AI scribes, for regulators to receive additional funding, and for the federal government's AI Safety Institute to independently test AI scribe models for accuracy and assess vendors' productivity claims.

The Therapeutic Goods Administration last year said it was "stepping up its efforts" to regulate AI scribes and is currently reviewing their use in Australia's healthcare system.