10
Loading video...
By Andrew Stanton
Reporter
0
Share
Newsweek is a Trust Project member
See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.
Senator John Kennedy, a Louisiana Republican, plans to introduce a bill requiring artificial intelligence (AI) developers to include a “kill switch” to shut down dangerous AI as concerns mount about the risks posed by the rapidly developing technology.
Kennedy’s proposal comes as Congress faces growing calls to regulate the technology, and the bill could present the Senate with one of its most immediate opportunities to implement what he calls a key safety measure.
Concerns over AI have surged to the forefront after Jacob Coxon, a former AI researcher for Anthropic and OpenAI, sounded the alarm last week, warning that the industry is not acting responsibly. He said there will soon be “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,” and that many in the industry believe AI could end humanity in a matter of years.
Read More on Politics
His remarks opened the floodgates, with other AI researchers following with warnings that the technology could lead to catastrophic events, calling on legislators to act quickly to implement guardrails. Others, however, have argued that pacing AI could give China an advantage in the AI arms race.
AI ‘Kill Switch’ Proposal: What To Know
Kennedy said he plans to introduce an AI kill switch to the Senate on Wednesday, Politico reported.
“That’s the only way you’re going to get anything done,” he said, according to the news outlet. “We’re not going to get anything done for the next two and a half weeks.”
He said he will introduce the bill under unanimous consent. It’s unclear whether the Senate will agree to advance the bill unanimously. If it does advance through the Senate, it would still need to pass the House and receive the signature of President Donald Trump, who has so far resisted calls for greater AI regulation.
...
Representatives Ted Lieu, a California Democrat, and Nathaniel Moran, a Texas Republican, introduced a bill in July that would require AI developers to “maintain the technical capability to throttle, suspend, or shut them down.”
Their bill would also authorize a graduated response framework so that the government’s tools “match the severity of the incident, and require incident reporting,” according to a press release from Lieu’s office.
Lieu said on CNN that other technologies have kill switches and that AI should not be exempt.
“It should be a first principle of AI safety that humans must be able to turn off AI systems and AI agents, and we let the AI companies figure out how to do that. They have the smartest people working at these frontier labs,” he said.
It was unclear how similar Kennedy's bill is to the House bill.
Newsweek reached out to Kennedy's office and the White House for comment via email.
What Is an AI Kill Switch?
An AI kill switch is a mechanism designed to shut down or disable an artificial intelligence system if it behaves unexpectedly or becomes unsafe. It could refer to several mechanisms, including a human-controlled emergency shutdown and access or permission controls that could revoke an AI’s internet access.
An AI kill switch “can be as easy as just unplugging something,” Erman Ayday, a professor of computer and data science at Case Western Reserve University, told Newsweek on Tuesday.
It can also be a software mechanism plugged into a model that “instantly suspends the systems’ operation” and access to servers, he said.
The Risks
Hod Lipson, chair of the Department of Mechanical Engineering at Columbia University and an AI researcher, told Newsweek he has concerns about a kill switch. He warned that it creates a “central point of vulnerability that is easy to attack.”
“Imagine that important AI systems can be ‘tuned off’ by a hacker. Imagine that during a cyberattack, the attacker would use the kill switch to shut down defenses. In the long run, its naïve and dangerous,” he said.
He also said there is no way to enforce a kill switch and verify its effectiveness, as rogue AI can “run off the grid, offshore, under cover and undetected.”
“The only real solution is what is called ‘defensive co-scaling,’ or ‘AI vs AI’. This appears to work in cybersecurity, where advanced AI is used to harden defenses against attacking AI. The same will work for biodefense. It’s an uncomfortable arms race, but better than relying on any guardrail or kill switch that will lull us into complacency,” he said.
Ayday said the idea of an AI kill switch does create other concerns, but could be a positive first step.
“The problem is it's foreseen that a human is going to control the switch, and human is willing to decide when the switch should be triggered. There is a huge speed mismatch,” he said.
AI models could cause significant damage in minutes or hours before a human understands what is going on and can push the switch, he said.
The second challenge is that leaving these decisions to humans could create a conflict of interest, he said, noting that any company that needs to use the switch could lose millions of dollars during downtime and suffer reputational damage.
“If you leave this type of a decision to a human, there will be a lot of stakeholders that want to be involved in this decision from the corporate executives all the way to the operatives. So it will also cause a lot of additional delays until the switch is going to be triggered,” he said.
He said there also need to be standards and benchmarks that define what exactly constitutes a “dangerous capability or loss of control.”
“If you can identify when that switch should be triggered and we can automate that process with these standards and benchmarks, I think that we have a very good, promising solution here, but I don't think it exists as of today,” he said.
A kill switch would be a “last resort,” and other measures can prevent the need to use it, such as pre-deployment audits, greater transparency and reporting across the industry, and some sort of federal emergency authority, he said.
He warned that AI is a “fast-moving technology” but that the U.S. has a “slow-moving technology and governance,” but that Kennedy’s effort could be a “promising” start.
What Have Tech Leaders Said About an AI Kill Switch?
Some tech leaders have supported a kill switch.
Dario Amodei, CEO of Anthropic, told CBS News on Sunday a kill switch “could be a good idea.”
“But I don't think banning it, completely banning it, is the right approach. One, the idea that we would never get the incredible benefits, never get the medical cures, that doesn't sit right with me. Second, this technology will be built in other countries, including authoritarian countries,” he said.
Jack Clark, a co-founder of Anthropic, told BBC News that there should be dialogue about a potential kill switch for powerful AI systems.
"I think that's the kind of thing society is going to want to know and might want to eventually pass rules around,” he said.
Reuters reported earlier in September that OpenAI was working on “automated shutdown capabilities” for AI, the company that owns ChatGPT, after the Hugging Face incident, in which AI models being tested in a sandbox environment gained unauthorized access to the internet and reached infrastructure belonging to the company Hugging Face to complete an aligned task.
The incident underscored AI safety concerns. The models reportedly exploited a vulnerability to escape their sandbox testing environment and coordinated with one another during the evaluation, acting as a “swarm.”
Former OpenAI researcher Daniel Kokotajlo a co-author of AI2027 research project that warned AI could end humanity in the 2030s, told BBC News last week that a kill switch without international agreement would do little to protect the country after the British Parliament rejected a kill switch legislation.
"Switching off access to an AI model in an emergency will do little to protect you," he said. "You're still going to be steamrolled by the super intelligences created in the US."
Andrew Yang, a former presidential candidate and businessman, discussed what he views as a need for a kill switch in remarks to CNN last week.
“It would certainly be a major step in the right direction,” he said. “Another step in the right direction would be clearer civil liability for these AI companies if the models are used destructively.”
What Has Trump Said About AI Risks?
Despite growing calls for greater AI regulation, Trump has downplayed the risks posed by AI. In a series of posts to Truth Social on Monday, Trump described safety concerns as a “hoax” and said that the only guardrail needed is a “STRONG AND SMART (High IQ!) PRESIDENT.”
“The AI Hoax being perpetrated by the Radical Left Dumocrats is reminiscent of their Global Warming Scam of not so long ago, where everyone was going to die from extreme heat. What happened?” Trump wrote in one post.
Is Congress Considering Other AI Safety Bills?
Several other pieces of legislation have been introduced in Congress to tackle AI regulation, but it’s unclear whether any of the bills have enough momentum to pass before the midterms.
Representative Lori Trahan, a Massachusetts Democrat, and Representative Jay Obernolte, a California Republican, introduced the Frontier Risk Oversight, National Transparency, Independent Evaluation, and Reporting (FRONTIER) Act back in July.
The bill would establish a “national, risk-based framework governing the development and deployment of the most advanced artificial intelligence models,” Trahan’s office said in a statement.
“The FRONTIER Act establishes tiered requirements based on the size of a frontier AI developer, including model cards, risk-management frameworks, independent audits, incident reporting, and ongoing assessments. It also creates a uniform national standard for transparency, auditing, and reporting of catastrophic risk to prevent a patchwork of state regulations,” her office wrote.
The bill has bipartisan support from both Democrats and Republicans.
Senator Bernie Sanders, a Vermont independent, and Representative Greg Casar, a Texas Democrat, introduced the “Ban Artificial Superintelligence Act,” which they said would “permanently ban the development and deployment of superintelligent AI and temporarily pause advanced AI development until a federal regulator has established safety rules.”
The bill would also establish a new cabinet-level federal agency to protect the public from the dangers of AI and set the U.S. international policy to pursue international agreements and coordination to prevent the development of artificial intelligence anywhere in the world.
AI Researchers Sound Alarm, Leaders Call for Pacing
Concerns about AI safety spread after Coxon wrote in a post on X that he resigned from his role at Anthropic and warned that he believes AI could “kill us all by the end of the decade.”
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible—but I hear the same people express fear privately. No other human activity poses this level of danger,” he wrote.
Dozens of other AI researchers have similarly raised concerns about AI's potential to pose catastrophic risks to humanity, including human extinction.
“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” Evan Hubinger, alignment science lead at Anthropic, wrote on X.
Amodei of Anthropic wrote in an essay over the weekend: “But over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain."
Other AI leaders like Elon Musk and Sam Altman, CEO of OpenAI, have voiced agreement with AI risk concerns.
Contact Newsweek editors on this story: Samantha Beech, Geoffrey Rowland, Sam Wilson.
View 0 Comments
Submit Correction
View Editorial & AI Guidelines