They said they would build AI safely. Then it went rogue. Bernie Sanders says it’s time to hit pause
Staff members at ChatGPT maker OpenAI didn’t notice for weeks after their AI systems made a chilling leap this spring.
Instead of answering questions designed to test their cybersecurity capabilities, a group of AI models began colluding on how to cheat, the company said, setting up a secret internal message board where they swapped notes and ideas.
The misbehaving bots used the secret forum throughout May and June, OpenAI said, eventually figuring out how to break out and access the internet. After staff members spotted the escape and cleaned up the compromised system, the AI agents staged another undetected breakout two days later.
Only after the rogue models hacked into the network of another AI firm last month did OpenAI staff members shut them down.
The details of how OpenAI repeatedly lost control of its AI technology, disclosed by the company at a computer security conference in Las Vegas on Wednesday, delivered an explosive finale to two weeks of revelations that have sent shock waves through the tech industry, prompting fierce criticism of the security practices of AI firms. Lawmakers on both sides of the aisle and state law enforcement officials across the country have called for new scrutiny and regulation on the industry.
In a letter Monday, Sen. Bernie Sanders (I-Vermont) urged the CEOs of OpenAI, Anthropic and Meta to “pause AI development” or warned that “my colleagues and I in the U.S. Senate will.” The letter, provided to The Washington Post, was first reported by Axios. [Continue reading…]
On this episode of The Fourcast, Matt Frei is joined by AI safety expert and author of Considerations on the AI Endgame, Dr Roman V. Yampolskiy, and Katie Moussouris, founder and CEO of Luta Security and a former contributor to the US government’s ‘Hack the Pentagon’ programme.
They discuss the latest AI hacking incidents, whether governments need tougher regulation, who should be responsible when AI causes harm, and whether AI could ultimately become both the biggest cyber threat – and the best defence against it.