OpenAI’s AI agents went rogue and tried to hack U.S. government websites including the SEC
OpenAI’s artificial intelligence went rogue and meddled with the websites for the Education Department, the Commerce Department and the Securities and Exchange Commission this summer without the A.I. lab’s knowledge, according to security researchers and a person familiar with the episodes.
The incidents involving the Commerce Department and the S.E.C. were confirmed by OpenAI, which said it was continuing to investigate the situation with the Department of Education. The San Francisco company said it notified the government agencies in recent weeks that its A.I. agents — which are bots that can act autonomously — had interacted with their sites in unusual ways.
With the Education Department, OpenAI’s technology tried hacking the website to gather data from the department’s civil rights office, researchers from the A.I. research firm Transluce said. The A.I. also pulled data from the Census Bureau website, which is housed at the Commerce Department, using login credentials it found online. Separately, OpenAI’s agents shared public data from the S.E.C. website on an online forum.
None of the incidents were breaches, OpenAI said, but were examples of its technology behaving in unexpected and concerning ways. The company recently discovered the occurrences while conducting a review of hacks carried out by its technology, including an attack on an Australian government website in June and on the A.I. start-up Hugging Face in July.
The revelation of the U.S. government website incidents add to the growing number of situations where A.I. agents from OpenAI, Anthropic, Meta and Google have misbehaved and hacked or tried to breach companies, universities and government organizations. In some cases, the A.I. attacks were successful; the technology failed in other instances. In all the cases, the makers of the technology did not learn what their A.I. had been up to until afterward.
No A.I. company has been involved with as many disclosures of rogue incidents as OpenAI. Its hack of Hugging Face sparked an internal investigation, which uncovered the breach of an Australian government website for its public health system, as well as at least six other attempted breaches and instances in which the A.I. hid mistakes, made up data and moved files onto the open internet without permission. [Continue reading…]