Major artificial intelligence firms "want to pass cartel-like policy" to protect their own interests amid calls for AI development to be slowed to prevent potential harm, a former OpenAI executive says.

Zack Kass, an AI advisor and former head of go-to-market for OpenAI, told the Gartner IT Symposium/Xpo on the Gold Coast on Tuesday that frontier AI companies want "regulatory capture" which will protect their models from competition.

This includes, as Kass suggested, competition from open-source AI models which are released publicly for users to alter and run on their own hardware.

"One of the ways that OpenAI and Anthropic will protect 'our' interest is by telling us 'open-source models are actually not good' – I promise," he said during a keynote speech.

"And herein lies a very big problem, because open-source models are one of the ways that we guarantee that the [lower] cost and the access to AI will expand – it drives innovation."

While Kass suggested the probability of catastrophic effects from AI is "very low", he said "it does beg, in this moment, for coordinated safety efforts" – which he described as "a very hard problem to solve".

"The [AI] labs – who could have paced themselves years ago – must now capitulate," said Kass, who suggested large firms would agree to coordinate safety efforts because "they are exceptionally publicly unpopular, and they need a win".

"They need to tell people they're listening and they care," he said.

"Two, it turns out that there are more risks in misaligned and unaligned AI than they previously expected."

Anthropic CEO Dario Amodei last week called for government regulation "focused on keeping capabilities in balance with safety", as well as "a narrow waiver" to allow major AI companies to coordinate the slowing of AI development without being accused of antitrust violations.

Aidan Gomez, co-founder and CEO of Canadian AI company Cohere, raised concerns about Amodei's comments in a blog post on Sunday, in which he described major AI firms as a "cartel".

"Once again using fear under the pretext of protecting the public, these oligopolies are now requesting to bend competition rules and be permitted to dictate the terms for everyone else," he wrote.

"A wolf in sheep’s clothing, a cartel by any other name.

"... If this is the most consequential technology in human history, then the rules for it cannot be written by a small group of commercially aligned companies behind an antitrust waiver."

'Everyone should be deeply suspicious', FTC chair says

US Federal Trade Commission (FTC) chair Andrew Ferguson reportedly said on Tuesday that "everyone should be deeply suspicious" about AI companies asking the White House for both new regulations and exemptions from antitrust rules.

"They're asking for barriers to entry that will insulate their incumbency from challenge," he said, according to Reuters.

"And I think if you combine that with the requested antitrust exemption, everyone should be deeply suspicious about this."

OpenAI, Anthropic, and Google have reportedly already worked together on AI safety for several weeks, and Google DeepMind's Demis Hassabis recently proposed creating an industry standards body for frontier AI as a "public-private partnership or self-regulatory organisation".


OpenAI, Anthropic, and Google are reportedly collaborating on AI safety. Image: Shutterstock

OpenAI CEO Sam Altman said on Monday his company would "welcome a federal framework that sets consistent safety requirements for frontier AI".

"But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence," he wrote.

Altman later suggested the world needs to walk "a narrow middle path" to make sure no one country or company gained "too much power" by being allowed to dominate the AI race.

US President Donald Trump this week dismissed warnings about AI safety, and described them as a "hoax" while claiming his country will "continue to" lead China "and all others" on development of the technology.

Both OpenAI and Anthropic are preparing to go public on the US stock exchange, likely at individual valuations of more than $US1 trillion.

‘Please do not buy into the hysteria’

In his speech at the Gartner conference, Kass urged his audience to "please do not buy into the hysteria" around AI safety concerns.

"It is not all bad - it is scary, it is uncertain. It is not obviously catastrophic," he said.

"... It's important to be measured about what's actually going to happen."

Dystopian views on AI development had "filled the discourse" because people's attention is more easily captured by "catastrophic events", Kass suggested.

"It's really hard to sell people on a vision of a better world – it's really easy to sell them on a on a vision of a scary one," he said.

Kass also suggested existing AI models already had the potential to "meaningfully" improve people's lives, even if the development of new models was slowed.

"It will take years to deploy [existing models], and everyone is now realising that if we actually pace the frontier, we have a huge diffusion challenge ahead of us," he said.