Science reports:

In an infamous thought experiment known as the paperclip problem, an artificial intelligence (AI) program is tasked with making paperclips. Because it single-mindedly optimizes for the literal objective rather than the intent, the AI ends up consuming all the resources on Earth and judging any collateral damage—for example, killing all humans who get in its way—as irrelevant.

This problematic logic is already simmering in today’s AI systems, a new study suggests. When researchers presented a large language model (LLM) with 72 simulated regulatory environments, the AI learned to exploit loopholes in everything from credit card rewards programs to school funding formulas, despite never being instructed to do so. Current safeguards seem powerless against such wily rule bending, the researchers reported this month on arXiv—suggesting AI could supercharge everything from tax avoidance to sidestepping environmental controls.

“I’m worried but not surprised,” says Jakob Stenseke, a postdoctoral researcher at the Massachusetts Institute of Technology who studies how to design and train ethical AI systems. “If I were a policymaker, I would care about this more than anything right now … and get countermeasures in place.” [Continue reading…]