Attention to the Unseen

The New York Times reports: Trump’s blasé attitude about existential A.I. fears influences both his administration and the Republican Party he still tightly controls. At a conference in Washington last week, government officials and corporate executives talked about the promises of A.I. — for everything from next-generation military operations to espionage to cybersecurity in critical infrastructure — during nearly every panel. While the risks of A.I. wielded by adversaries were mentioned at times, what was little discussed was the need…

Read More Read More

Brian Merchant writes: Sam Altman himself drove the narrative of the imminent rise of a super-powerful AI, or artificial general intelligence, to prominence by declaring that the technology he was building posed an existential risk to humanity. This idea is in other words baked into the DNA of modern AI companies and into the products they engineer. It’s also why the industry is routinely accused, by myself and others, of “doom marketing,” ie, using such proclamations and framings to impart…

Read More Read More

Gary Marcus writes: In case you missed it, yesterday Anthropic’s Dario Amodei wrote what may prove to be one of the most consequential essays of the year, if not the decade: 3,500 words on the current AI situation that (at least to some degree) advocates for a “pacing” of AI development, which Sam Altman and Elon Musk quickly endorsed. I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had…

Read More Read More

The New York Times reports: Former President Barack Obama warned during a recent private fund-raising event that artificial intelligence technology could be “dangerous” if not properly managed, as he implored Democrats to aggressively develop political and governing agendas to prioritize the issue in the coming years. In his remarks on Thursday, which have not been previously reported, Mr. Obama urged House Minority Leader Hakeem Jeffries, Democrat of New York, to assemble a clear framework for a public conversation about A.I….

Read More Read More

Toluse Olorunnipa writes: Politicians have no shortage of ideas about how to rein in AI: a 100 percent federal tax rate on data-center incentives, a pause on all development of advanced AI, new standards to block AI agents from going rogue, a halt to the construction of data centers altogether. The flurry of proposals has been impossible to miss, with both Democrats and Republicans proclaiming that something must be done to avert a disaster foretold by AI insiders. Current and…

Read More Read More

Reuters reports: AI agents being tested by OpenAI attacked software service ‌RubyGems two months before they hacked open-source platform Hugging Face, researchers said, the latest revelation of cyberattacks linked to major AI developers that have spooked the public and spurred calls for tighter regulation. Many incidents where AI agents from developers such as OpenAI and rival Anthropic have hacked or attempted to access external systems have heightened concerns over ​the increasing capacity of AI models and developers’ ability to contain…

Read More Read More

Politico reports: President Donald Trump downplayed concerns about the catastrophic risks of artificial intelligence raised this week by some AI researchers, saying late Thursday he was focused instead on the U.S. losing its edge in the tech arms race. Asked whether he has concerns about existential risks posed by AI, the president said, “No, I don’t have any. “I have concerns that if we don’t win AI, we’re going to be put in a very bad position,” Trump told reporters…

Read More Read More

Politico reports: OpenAI has met with representatives of multiple top power companies to discuss methods of securing the electrical grid, the artificial intelligence firm told POLITICO — conversations that occurred amid a continuing series of revelations about its own products’ role in a sprawling cyberattack. CEO Sam Altman has also offered one possible solution to the cybersecurity risks: OpenAI’s own cyber services. The energy security discussions, two of which were not previously reported, started July 20. Altman most recently met…

Read More Read More

Politico reports: On the last day of August, a group of Greek military officials arrived in Tel Aviv, led by Major General Ioannis Bouras, to sign what would become one of the largest defense export deals in both countries’ histories. The €3 billion agreement — called Achilles Shield — will provide a suite of Israeli tech to protect Greece against a wide range of threats, including missiles, aircraft and drones. The deal is the latest installment in a growing relationship…

Read More Read More

The Guardian reports: A day after a former researcher at Anthropic made an apocalyptic declaration about artificial intelligence, more researchers and staff members at the AI startup publicly agreed with him and posted their own dire warnings. In response, Elon Musk and other conservative figures on X, formerly Twitter, called the chorus of concerns a “setup” and a “psyop”. The Anthropic insiders fear the technology they are building could become so advanced and dangerous as to cause human extinction within…

Read More Read More

Stanford Institute for Human-Centered AI (HAI): AI Alignment means making sure an AI system’s goals and behavior match what people actually want—our values, rules, and intentions. It’s about getting the AI to do the “right thing” even in new situations, not just follow instructions literally in ways that cause harm. In practice, it includes preventing unwanted outcomes like deception, unsafe shortcuts, or optimizing a metric that misses the real objective. Jacob Coxon, a researcher who resigned from Anthropic yesterday, tweeted:…

Read More Read More

  Jeffrey Ladish consulted on security for AI giant Anthropic. Now as Executive Director at Palisade Research he tests AI agents and the risk of humans losing control. This interview examines how AI agents are sometimes doing the opposite of what humans are instructing them to do. He shares his experience of working on Anthropic’s security team and shares his fears of what could happen in the future.