Consulting giant Deloitte will partially refund the federal government after acknowledging that generative AI was used in a report containing several fabricated references and quotes.
The initial report, submitted to the Department of Employment and Workplace Relations (DEWR) in June, investigated the department’s Targeted Compliance Framework and the IT system used to automate welfare penalties.
It included references to non-existent academic reports, a fake reference to a court decision, and an incorrect reference to a quote from a Justice that also misspelled their name, raising concerns over AI “hallucinations” in official work.
At the time, it was speculated that the use of generative AI tools was to blame for the made-up references and quotes.
Deloitte has now resubmitted a corrected version of the report to the federal government with the bogus references and quotes removed and admitted that AI was used to assist with some parts of the investigation.
The company has agreed to repay the final instalment of its contract, though the amount has not been disclosed.
The contract, which ran from late last year to July, was worth just under $440,000.
The report was published by the Department on Friday last week, in the lead-up to a public holiday across much of the country.
Made up reports and quotes
The new report includes a concession at the start that the previous version contained a number of errors.
“The report has been updated to correct those citations and reference list entries which contained errors in the previously issued version, to amend the summary of the Amato proceeding which contained errors, and to make revisions to improve clarity and readability,” the report said.
“The updates made in no way impact or affect the substantive content, findings and recommendations in the report.”
The report also confirmed that Deloitte workers used a generative AI large language model to assess whether “system code state can be mapped to business requirements”.
The original report included 12 references to a made-up report supposedly by a law professor at the University of Sydney, and two references to a non-existent report by a Swedish professor.
These references have been removed in the new version of the report.
The first iteration of the report also included an incorrect reference to a court decision in a major Robodebt case, and the misspelling of a judge’s name that also included a made-up quote attributed to them.
These were also removed from the revised report.
A spokesperson for DEWR told the Australian Financial Review that the "substance of the independent review is retained, and there are no changes to the recommendations”.
In its earnings report last week, Deloitte said that it works with clients to “imagine how generative AI and other technological advances can help reshape business models and execute their vision now and in the future”.
The company has allocated more than $4.5 billion ($US3 billion) to generative AI to the 2030 financial year.
AI hallucinations now common in the workplace
The growing use of generative AI tools in the workplace has led to a number of embarrassing and serious incidents of AI-generated “hallucinations”.
This has been particularly rife in the legal sector, where lawyers have submitted false or inaccurate references and cases to courts around the world.
Last month an Australian lawyer was punished for submitting fake cases to a court for the first time.
