Gov proposes AI safety regulator

The federal government is proposing to set up an AI safety regulator to manage risks posed by frontier models and AI agents, after an incident that affected Medicare and state government authorities.

The regulator’s existence and remit is to be enabled by forthcoming “Australian AI standards legislation”, a group of ministers said in a joint statement.

“Strong mandatory safety and security protections will be legislated through the AI Standards and apply to all frontier AI companies that wish to operate in Australia,” the ministers said.

“The government will propose these protections be enforced by a new targeted regulator for AI safety, which will work with the Australian AI Safety Institute to provide technical expertise and oversight as global standards evolve.

“The regulator would be able to enforce significant penalties for AI companies that fail to comply with mandatory obligations that will help keep Australia safe.”

The proposal for a new regulator is the latest measure to come out of an investigation into an incident in June - disclosed late September - where an OpenAI agent accessed public and non-public files associated with an older Medicare data portal.

Other federal and state data repositories were also impacted.

In addition to new rules for the developers of frontier AI models and agents, the incident has also cast a light on government departments and older systems they still run.

Home Affairs has ordered a government-wide stocktake of systems by March next year, with the aim of identifying configuration weaknesses that could be susceptible to exploitation by AI agents.

Now, under rules that would be enforceable by the AI safety regulator, the developers of AI models and agents would have to “allow independent evaluation of their models” and offer “rapid” disclosure of incidents of rogue behaviour.

They would also have to “identify and manage serious risks associated with their models” and “maintain security controls across all stages of development and deployment”.

“Those who want to cause us harm have access to the latest technology. We need to make sure we have the same, or better, but it’s not a free-for-all,” Minister for Home Affairs and Cyber Security Tony Burke said in a statement.

“There needs to be clear guardrails to protect Australians.”