The threat of human extinction will get Congress to act on AI safety…right?

As many AI researchers have been increasingly fraught with existential terror about their own creations this summer, their alarm is spreading among policymakers and the media.

There are some policy ideas to address the risks: requiring “kill switches” for AI models, setting federal standards for safe research, or even shutting down development of cutting-edge “frontier” models altogether.

But a stable national policy would take an act of Congress. That looks unlikely this session, even as AI developers are calling for regulations to slow down their own research on the grounds that it could be racing toward widespread doom.

I asked Stephen Casper, a computer scientist who studies AI safety and governance at the Harvard Kennedy School, about some of the risks policymakers are mulling. He told me that we don’t know if leading companies even could completely shut down their frontier models in an emergency.

“I don’t think there’s any public knowledge of AI companies doing anything equivalent to a fire drill,” Casper said.