Existing tools for detecting text generated by ChatGPT are “neither accurate nor reliable”, a team of researchers has found, further emphasising the need for educators to rethink how they assess students’ work.

The study from a group of European academics, which has not yet been peer reviewed, sought to test the efficacy of 14 AI detection tools including one from popular academic plagiarism software Turnitin.

They found only five tools had an overall accuracy above 70 per cent.

All the tools occasionally misdiagnosed human-written work as being AI-generated and vice versa.

Obfuscation techniques also appear to be extremely effective at tricking detection tools with around half the AI-generated texts that were subsequently edited by humans getting past the AI detectors.

“The different types of failures may have serious implications,” the researchers warn. “False positives could lead to wrong accusations of students, the false negatives allow students to evade detection of unauthorised content generation gaining unfair advantages and promoting impunity.”

Because the tools lack much in the way of evidence for their claims, the researchers warn that this software is nearly useless at proving academic misconduct and at best can only offer "a hint that some sort of misconduct may have happened”

Overall, the researchers conclude that AI detection tools “should not be used in academic settings” and instead educators need to adopt a “prevention-focused approach”.

“The focus should instead be on the preventive strategies on how to ethically use generative AI tools, including a discussion about the benefits and limitations of such tools.”

Students in Australian public schools are currently banned from using ChatGPT, although some private schools have already started teaching with AI.

Last week, Australia’s education ministers met to discuss national issues in the country’s schools like vaping, the use of mobile phones, and, of course, artificial intelligence.

According to a communique from that meeting, a Draft AI Framework for Schools is currently in development.

Education Minister Jason Clare said AI-generation tools like ChatGPT were akin to “a calculator or the internet”.

“We need to learn how to grapple with this new technology, but there are challenges with it,” he told journalists on Friday, citing issues around fairness and privacy.

“How do you make sure that we use it for good, to help children to learn, but we don’t have children getting the marks that they don’t deserve by using generative AI?

“How do we make sure that, if our children are using generative AI technology, like ChatGPT, that what they’re typing into the system doesn’t get spat back at them as an ad on TikTok?”

This will ultimately be a matter for the states and territories to work out on their own, Clare said, but he is hopeful a national framework will provide some consistency.