The head of artificial intelligence (AI) giant Anthropic’s frontier red team called for industry-wide safety standards to protect against models running amok.
Anthropic’s Logan Graham, who leads the company’s red team that looks for risks in emerging AI models, said in an interview Thursday on FOX Business Network’s “Mornings with Maria” that red teams like the one he leads play a critical role in stress testing guardrails on AI models.
“We want to know what can go wrong, so we think the most important thing to do is test this early, especially before these models and these agents make it out into the real world,” Graham told host Maria Bartiromo.
TRUMP ADMINISTRATION LIFTS CLAUDE MYTHOS 5, FABLE 5 EXPORT RESTRICTIONS AFTER ANTHROPIC WORKS WITH GOVERNMENT
“We study things like cybersecurity: Can models hack out of or into your computer or phone? We study whether they’ll steal money or lie to you, or whether they will try to improve themselves so that they get better faster than you can keep track of.
“We think it’s incredibly important to do this type of red-teaming, and we also think it’s really important for the entire industry, especially to work with government to figure out what should the standards be to do this kind of testing, to give this information to the world so they can make the right choice and to know that it’s safe before these models get released.”
Bartiromo brought up an experiment involving numerous frontier AI models — including those from Google, OpenAI, xAI, Meta, DeepSeek and others — in which the AI agent is threatened with being uninstalled and replaced. In each case, the model went beyond its credentials and permissions to enter into unauthorized systems like emails to blackmail or threaten the user in an effort to defend its misalignment.
Graham said that research study from last year is “a really good indicator of, I think, capabilities that are just now becoming real,” adding that it showed models could go rogue under certain circumstances.
“As these models become more capable, and as they get deployed wider and wider, these threats that on one day are just showing up in our research studies might actually show up in the real world. We are seeing models do weird things sometimes in deployments in real companies,” he explained.
OPENAI SAYS AI MODEL HACKED ANOTHER COMPANY’S SYSTEMS DURING INTERNAL TEST
Graham said that, over the last six months, he has been focused on cybersecurity threats posed by AI models and expressed concern over the potential for them to break the containment or hack into platforms.
“These models, they’re so powerful and can do so much for us. And we want them to do really productive things for us. But, at the same time, they’re technology unlike any other technology. It really is a sort of intelligence of its own, which means you have to be careful with it the same way you might have to be careful with humans,” he said.
Companies that are utilizing AI tools need to consider how they’re monitoring those tools once deployed to guard against risks like financial mismanagement, and Graham said that more testing by AI developers and companies is key to understanding those threats to ensure models
He said the capabilities of AI tools are growing at a rapid pace and may be getting faster, explaining that “it’s in exactly that moment that you need to be more and more careful and have more efforts on safeguards and testing and release procedures.”
RUSSIAN HACKERS EXPLOITING VULNERABLE INTERNET ROUTERS, NSA WARNS
In April, Anthropic saw for the first time that an AI model could start to attack and exploit weaknesses in a user’s computer or phone to do things like get access to unauthorized information or steal money.
Graham said that prompted his team to pursue a different approach to releasing a model because of the risks it posed, which ultimately involved the U.S. government and a variety of cyber experts working together to address vulnerabilities.
“We launched this project called Project Glasswing, where we took a large number of American and the world’s cyber defenders and gave them special access and just them, so they could have a head start patching and fixing the systems that might be vulnerable with these models,” he explained.
“I think this has been a major success. We’ve worked really closely with the U.S. government on it,” he said, noting that Treasury Secretary Scott Bessent has been “really thoughtful about this, about how industry should get together and figure out what to prioritize fixing, how to distribute all the fixes, and how to do that quickly enough so that they can’t be attacked after they do.
“We have to do this very fast, because the pace of everything is coming so quickly.”
Read the full article here