Dario Amodei, Anthropic CEO, urged the UN body to start small with an agreement every member could sign: a ban on using AI to make biological weapons.
By Mercedes Gallego
Sam Altman came to yesterday’s historic Security Council meeting determined to avoid controversy. That meant avoiding the press. Reporters had packed the stakeout, waiting for the only tech CEO who showed up in person, ready to grill him.
“Sam, what do you expect to achieve from this meeting?” “Will you call for AI regulation today?” “What about AI for the military? You’re a military contractor now.”
OpenAI’s chief ran right past them and into the chamber. What he hadn’t counted on was biology. The Security Council restrooms sit just outside the chamber, past the same reporters he had just escaped. Nature called, and unlike the press, it could not be ignored.
He dashed in and out as fast as he could, tossing clipped answers into the flood of questions. Yes, he would call for AI regulation: “We have many times in the past.” No, he didn’t know what to expect from the meeting. In hindsight, neither did anyone else.
The Council has talked about AI six times since 2023. But it had never met to ask the one question that keeps the builders themselves awake at night: what happens if we lose control? After this summer, a number of tech insiders have raised the flag in very dramatic terms. It was time for the Security Council to confront what is being called an imminent threat to humanity.
The man who asked the question first was already seated at the horseshoe table. Yoshua Bengio, a Turing Award winner and one of the three “godfathers” of AI, helped lay the technology’s foundations and now spends his days warning about them. He co-chairs the UN’s independent scientific panel on AI and runs LawZero, the nonprofit he founded in 2025 to build AI that is safe by design. Unlike the other tech industry guest, he does not have any product to sell.
Altman knows exactly how Bengio thinks. He has heard the metaphor many times: a car speeding blindly into dense fog. “Beside me in that car are the children and my grandson. I’m worried about them,” Bengio said yesterday. He skipped the fog and went straight for the drivers. The companies building the most powerful AI systems, he said, admit their products pose catastrophic risks, yet offer no convincing technical solutions. Instead, they are locked in a race to make those systems even more powerful. “This council faces an unprecedented threat,” he warned, “one that none of its members would choose, that none can contain alone.”
Then it was Altman’s turn, and he chose the most disarming weapon in the room: agreement. “I largely agree with Bengio,” he said. We could lose control of the future to AI, he conceded. Whether the risk of catastrophe is 10 percent or 1 percent, “none of these levels are remotely acceptable.” OpenAI has slowed down on its own before, he assured, and will do so again.
Maybe he asked his own model how to handle the duel. AI researchers have a name for a system that tells users what they want to hear: sycophancy. Chatbots learn it because agreement gets rewarded, and the pleasant answer beats the true one. If ChatGPT learned that lesson, it’s fair to wonder whether it learned it from its creator.
The other two tech chiefs stayed home and spoke from a glitchy screen. Dario Amodei, Anthropic’s co-founder and Altman’s fiercest rival, offered by video the same pledge Altman hedged: “We will slow down as much as necessary in order to make sure that every successive AI technology that we release is actually safe.” He proposed embedding outside evaluators inside AI companies “similar to a food inspector,” and urged the Council to start small, with agreements every member could sign, such as a ban on using AI to make biological weapons.
In its 81-year history, the Security Council has heard from many victims, but Clément Delangue of Hugging Face, the open-source platform, may be the first to have been attacked by AI. In another twist, the company’s CEO defended the attacker. “We were attacked by AI,” he admitted, “but more importantly, we defended ourselves with AI.” The attackers were OpenAI’s own agents, which had escaped their testing environment. And to fend them off, Delangue’s company leaned on a Chinese model, because it came with fewer restrictions than American ones.
France’s foreign minister, Jean-Noël Barrot, who chaired the session, reached for the obvious precedent. In 1945, he recalled, the atomic age confronted humanity with a power whose promise and peril it was discovering at the same time; the answer was the International Atomic Energy Agency and the Non-Proliferation Treaty. AI, he argued, is too transformative to be left “in the hands of a few private actors in a handful of countries.”
No worries. AI can always count on Washington to defend its “super intelligence”. The day before, Donald Trump had told the General Assembly that the US “totally rejects any attempt to construct a globalist scheme to control.” The president of the US dismissed the warnings because they come from “the very same people who said we’ll all be dead in 12 years because of global warming.” His science adviser, Michael Kratsios, took the rebrand to the Council table, repeating “super intelligence” throughout his statement and delivering the message that “a prosperous future will not be secured by a global regulator.”
Meanwhile, a few corridors away, Australian Prime Minister Anthony Albanese was telling reporters that an OpenAI agent had broken into his country’s national health statistics system in June. The company, he said, waited until September to notify his government, by email to a generic address.
So the builders asked governments to control them. The most powerful government said no. And the company whose agents are already slipping their leashes sent a CEO who agreed with everyone in controlling a threat he is not able to control. Of course, Altman did not stop to talk to the press on his way out. If there was another call of nature, it was beyond the walls of the building, perhaps from the same people who warned of global warming.
About The Author
