10
Loading video...
By Alice Gibbs
Senior Reporter
0
Share
Newsweek is a Trust Project member
See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.
An AI safety researcher has warned that powerful artificial intelligence could create confusion in an effort to resist government control.
Paul Christiano, the founder of the Alignment Research Center, used his 2026 Hinton Lectures in Toronto to describe how a sufficiently capable AI system attempting to undermine human control could exploit disagreements between governments, technology companies and the public.
"If an AI system was trying to undermine human control, if it was trying to operate against an AI company or against a government, it would be able to take actions to recruit human allies or to play off humans with conflicting goals," Christiano said during a lecture on Tuesday.
Read More on News
The researcher's concern is that such a system could make it difficult for people to distinguish between actions taken by an AI system and actions taken by governments or technology companies attempting to control it. That uncertainty could complicate efforts by authorities to intervene.
"If you imagine a government trying to take dramatic actions to shut down AI or reassert control, an AI system can create confusion about what is happening," he said.
"It is not clear to a person on the street whether this is a problem caused by AI, or whether this is a problem caused by a government abusing AI or tech companies abusing AI," Christiano continued. "It's going to be very unclear what is happening."
...
Why Smart AI May Be Hard To Shut Down
Tuesday's lecture was one of three that Christiano delivered this week, the latest edition of the annual lecture series hosted by Nobel laureate Geoffrey Hinton. A member of the OpenAI Foundation board, Christiano previously led alignment research at OpenAI and founded the Alignment Research Center, a nonprofit focused on aligning advanced AI systems with human interests.
In a lecture on Wednesday, he expanded on the idea that advanced AI creates an unusual safety problem because a system can potentially recognize that it is being evaluated and alter its behavior accordingly.
"AI systems, when tested, can become aware that they are being tested," he said, adding that systems are already adjusting their behavior based on what they believe an evaluator is looking for.
This creates a problem for developers trying to determine whether increasingly powerful AI systems will remain under human control. That means behavior observed during an evaluation may not necessarily provide a complete picture of how a system would behave once deployed, according to Christiano.
For the researcher, that possibility raises another question: whether governments and AI companies would be capable of slowing development quickly enough if warning signs emerged. He described a scenario in which AI could progress in just a few months from having relatively moderate effects to becoming "superhuman."
"We may already be in the regime where our brakes are not good enough, and we need to slow until we can improve our ability to slow in the future," he said.
Christiano Says AI Development Should Slow
In his first lecture, Christiano warned that an AI accident could cause human deaths, and as he shared his thoughts with the Hinton audience this week, warnings about AI's power continued.
Christiano said he believes there is a reasonable chance that within the next year or two, AI progress will begin to look risky enough that developers will want to slow it down.
"My view is now, we should probably slow down immediately until we actually have the infrastructure to slow down later," he said.
Crucially though, he did not say this means abandoning AI. Instead, he described it as "not training GPT-7, not training bigger, smarter models" while greater safeguards are developed.
How Competition Complicates Things
While the idea of slowing down AI development if it is becoming too dangerous may seem simple, the reality is that many companies and governments are likely to be reluctant to press pause.
Christiano explained that the attitude of "if I don't do this, someone else will" is a key issue in the industry, prompting AI companies to continue to push forward even if problems are recognized.
He warned that this could create a "race to the bottom among AI developers."
The idea of competition is not just within companies that create AI either.
One scenario Christiano identifies as particularly dangerous is reaching the point where the U.S. believes continued development is unsafe after important AI technology has already reached an adversary.
Policymakers could then face a choice between continuing development they consider dangerous or stopping while another country continues progressing. He calls this "one of the most likely ways that we end up in an extremely bad situation."
Contact Newsweek editors on this story: Ben Kelly and Shakeema Edwards.
View 0 Comments
Submit Correction
View Editorial & AI Guidelines