California Gov. Gavin Newsom's executive order is an opportunity for a needed, thoughtful conversation about artificial intelligence and its potential harms. Everyday Californians are feeling real anxiety about the risks of artificial intelligence, and as an organization that works to ensure technology empowers people, EFF welcomes this order as a way for the state of California to lead a much-needed dialogue that addresses these concerns. 

DONATE TO EFF

ensure ai serves people, not power

Nonetheless, the most immediate and current concerns with this technology are not about sci-fi scenarios concerning rogue super-intelligence. They are happening right now through biased algorithmic decision-making for employment or government benefits, AI-powered surveillance systems such as Flock cameras, and artificially inflated personalized pricing. People want state and federal leaders to act, and we urge Gov. Newsom to develop thoughtful policies to address those concerns. Today’s EO is a good start. 

To that end, EFF supports the focus on expanding the reporting requirements under SB 53 (2025) for loss-of-control incidents, alongside third-party investigations. We urge the administration to consider how to make these third-party investigations available for smaller developers. As the Government Operations Agency prepares its recommendations for the governor, we urge leaders to also realize that the effectiveness of kill switches in advanced AI systems remains an area of active research. As such, they should ensure that - as we’ve previously mentioned - any technology regulation targeting cybersecurity practices at AI labs must be careful, precise, and practical. Moreover, we also caution that government-controlled kill switches run the risk of being used as a form of retaliation against protected speech, as demonstrated by the Trump Administration’s retaliatory actions against Anthropic earlier this year.

Ultimately, true safety requires California to focus on concrete, immediate, and urgent harms of AI technologies by ensuring that algorithmic decision-making in both the government and private sectors respects people’s rights and well-being. We urge Gov. Newsom and the state of California to develop thoughtful policy in collaboration with those most at risk of harm to address these and other concerns.

With doomsday AI scenarios dominating the news, lawmakers are rightly concerned about reports concerning security breaches at major US AI labs, such as the OpenAI–Hugging Face incident and the many others reported in its aftermath. As they consider potentially regulating frontier AI, they should focus any new legislation on the immediate, demonstrated risks from those incidents. 

Post-incident reports show that the Hugging Face incident could have been mitigated or prevented by following longstanding cybersecurity best practices, like stronger sandboxing and monitoring. Any new legislation should focus on closing gaps in existing law to prevent AI companies from taking unreasonable risks with the public's security.

When an AI developer or deployer runs a test or a task that has a high likelihood of causing harm to third parties—for instance, by breaking into someone else's computers—there should be clear minimum safety requirements. Such tests should run in a properly sandboxed test environment, disconnected from other systems, and be monitored and logged. Following these fundamental best practices would have prevented or substantially mitigated all of the incidents at AI labs that we currently know about.

That said, any proposal must be flexible enough to evolve with changing technology. Minimum safety requirements specific only to current AI technologies are likely to become obsolete; legal standards tied to well-established cybersecurity best practices are far more likely to stand the test of time. Tying any new mandates to evidence-backed security protocols also protects the public without impeding future AI development.

Strong legislation should also mandate and fund independent third-party investigations into any serious security incidents that may occur during AI labs’ tests of new tools, and make reports of these investigations available to the public. This important transparency measure would go a long way toward providing public oversight of the industry.

As with any technology regulation, those targeting cybersecurity practices at AI labs must be careful, precise, and practical.