Vox reports:

Advocates say it is a modest law setting “clear, predictable, common-sense safety standards” for artificial intelligence. Opponents say it is a dangerous and arrogant step that will “stifle innovation.”

In any event, SB 1047 — California state Sen. Scott Wiener’s proposal to regulate advanced AI models offered by companies doing business in the state — has now passed the California State Assembly by a margin of 48 to 16. Back in May, it passed the Senate by 32 to 1. Once the Senate agrees to the assembly’s changes to the bill, which it is expected to do shortly, the measure goes to Gov. Gavin Newsom’s desk.

The bill, which would hold AI companies liable for catastrophic harms their “frontier” models may cause, is backed by a wide array of AI safety groups, as well as luminaries in the field like Geoffrey Hinton, Yoshua Bengio, and Stuart Russell, who have warned of the technology’s potential to pose massive, even existential dangers to humankind. It got a surprise last-minute endorsement from Elon Musk, who among his other ventures runs the AI firm xAI.

Lined up against SB 1047 is nearly all of the tech industry, including OpenAI, Facebook, the powerful investors Y Combinator and Andreessen Horowitz, and some academic researchers who fear it threatens open source AI models. Anthropic, another AI heavyweight, lobbied to water down the bill. After many of its proposed amendments were adopted in August, the company said the bill’s “benefits likely outweigh its costs.”

Despite the industry backlash, the bill seems to be popular with Californians, though all surveys on it have been funded by interested parties. A recent poll by the pro-bill AI Policy Institute found 70 percent of residents in favor, with even higher approval ratings among Californians working in tech. The California Chamber of Commerce commissioned a bill finding a plurality of Californians opposed, but the poll’s wording was slanted, to say the least, describing the bill as requiring developers to “pay tens of millions of dollars in fines if they don’t implement orders from state bureaucrats.” The AI Policy Institute’s poll presented pro and con arguments, but the California Chamber of Commerce only bothered with a “con” argument.

The wide, bipartisan margins by which the bill passed the Assembly and Senate, and the public’s general support (when not asked in a biased way), might suggest that Gov. Newsom is likely to sign. But it’s not so simple. Andreessen Horowitz, the $43 billion venture capital giant, has hired Newsom’s close friend and Democratic operative Jason Kinney to lobby against the bill, and a number of powerful Democrats, including eight members of the US House from California and former Speaker Nancy Pelosi, have urged a veto, echoing talking points from the tech industry.

So there’s a strong chance that Newsom will veto the bill, keeping California — the center of the AI industry — from becoming the first state with robust AI liability rules. At stake is not just AI safety in California, but also in the US and potentially the world. [Continue reading…]