OpenAI used AI to help write the email to the Australian government advising that its AI agent had hacked into key departmental websites, Guardian Australia can reveal.
On Tuesday, one of the company’s executives told a parliamentary inquiry that he didn’t believe that its own technology had been used to create the email, but said the company needed to confirm this.
Guardian Australia understands AI was used by OpenAI’s legal and security teams to generate parts of the wording of the email, including word selection and formatting of the message. But a source with knowledge of the incident said humans reviewed the final email, and humans were responsible for actually sending the communication to the Services Australia inbox.
OpenAI was contacted for comment.
An artificial intelligence agent developed by OpenAI accessed Services Australia data and three other systems in June. The company notified Australia on 10 September despite becoming aware of the incident in August.
The company’s first notification to Australia came in a five-paragraph email to a Services Australia inbox, publicdisclosures@servicesaustralia.gov.au, which was only checked once per day.
OpenAI has come under fire for not raising the issue in a more formal or direct way, including during a face-to-face meeting between the company’s CEO, Sam Altman, and Australia’s deputy prime minister, Richard Marles, on 1 September, nine days before the company emailed Services Australia but nearly a month after it first learned of the 18 June intrusion.
Jason Kwon, OpenAI’s chief strategy officer, admitted in a parliamentary hearing on Tuesday that the company’s “response was not good enough, and we should have informed the impacted parties much sooner”.
During the hearing, Liberal MP Aaron Violi – the shadow minister for technology – had asked Kwon specifically about the email and whether AI had been involved in its creation.
“I appreciate that, to get the data and have a review, you’ve got to use AI agents, and obviously your business uses AI. When you notified Services Australia via email, did your staff use AI to construct that email?” Violi asked.
Kwon responded: “I don’t believe so, but we’re happy to go and confirm.”
Kwon indicated during the hearing that OpenAI would provide specific responses to more technical queries in answers to questions on notice.OpenAI is expected to provide more information about the email once its own investigation has concluded.
The email, obtained by Guardian Australia in September, advised Services Australia: “We are notifying you of a security vulnerability identified during our review of OpenAI model activity involving Services Australia’s Medicare Statistics service at medicarestatistics.humanservices.gov.au.
“An OpenAI model identified a way to make the server carry out instructions sent through the public reporting interface, without a private account or password. It was able to access this to read portions of internal program files and settings, obtain a list of files, and create and read back a small test file on the server.”
The email advised that OpenAI’s review “found no evidence that the model accessed patient-level records, personal information or credentials; deleted data; or established ongoing access”, and sent information about the “affected URL” and “affected report”.
“We recommend that the team responsible for the service investigate the vulnerability and assess the changes needed to prevent it. We would be glad to brief your security team and provide supporting evidence as available.
after newsletter promotion
“Best, OpenAI Security Team.”
Andrew Charlton, the assistant minister for science and technology, spoke about the OpenAI incident in a speech in Sydney on Thursday, describing the company’s agent as having “hacked into an Australian government system”.
“As a starting point, no company should release a frontier AI model that is not safe,” he said.
“Yet the fact that has occurred, and the fact that the labs did not detect or prevent it, prompts important questions about the role of new regulation in the National AI Standards.”
Charlton said frontier AI “pushes the limits” of existing government conventions and protocols around assessing safety risks, and went beyond “conventional” approaches.
The assistant minister said “the market will not fix” issues with AI development, a contrast to the United States’ approach of letting companies operate with a degree of self-regulation. Charlton raised concerns that “frontier labs are putting capability ahead of safety”, positing that Australia can have the most impact on the development of AI by hosting and influencing frontier labs.
“AI needs regulating because its harms are severe, hard to undo, borne by people who never chose them, and sometimes invisible until they arrive,” he said.
“The market will not fix this alone, because the incentives reward speed and capability, and even the people at the top of the industry cannot slow down by themselves.”
Do you know more? Email josh.butler@theguardian.com