California AI bill SB 1047 aims to prevent AI disasters, but Silicon Valley warns it will cause one | TechCrunch

SB 1047 aims to prevent large AI models from being used to cause “critical harms” against humanity.

The bill would only apply to the world’s largest AI models: ones that cost at least $100 million and use 10 ^ 26 FLOPS during training.

A new agency, the Frontier Model Division ( FMD ), would oversee the rules.

A growing chorus of Silicon Valley players oppose SB 1047 .

Two AI researchers who are sometimes called the “godfathers of AI ,” Geoffrey Hinton and Yoshua Bengio , have thrown their support behind this bill.

A16z, the venture firm founded by Marc Andreessen and Ben Horowitz , has strongly opposed the bill.

As AI technology advances, it will get more expensive, meaning more startups will cross that $100 million threshold.

The Chamber of Progress , representing Google , Apple , Amazon and other Big Tech giants, opposes SB 1047 .

The bill will be sent to the California Senate’s Assembly floor with whatever amendments get approved.

Anthropic is the first developer of a state-of-the-art AI model to publicly signal it's willing to work with Wiener on the bill.