The Safety Alarm Is Also a Sales Pitch

On September 23, the UN Security Council did something it has never done before: it cleared its agenda to hear from the people building artificial intelligence. OpenAI chief executive Sam Altman sat alongside Turing Award laureate Yoshua Bengio and told the fifteen-member council that the technology now poses a “real and imminent” threat. Bengio’s line was the one that traveled: AI danger, he said, is a threat “that none can contain alone and that does not respect the borders we defend.” DeepSeek was in the room too — a rare instance of American and Chinese labs sounding the same alarm in the same chamber.

It was a genuinely striking moment. It was also, by any honest reading of the calendar, a very good week to be seen sounding an alarm.

Four days later, reporting framed the whole campaign in a colder light: the labs are sounding the alarm on AI safety and seeking to shape how it gets controlled. The two are not separable. Anthropic is targeting a November IPO at a valuation reported near $2 trillion; OpenAI has pushed its own public debut to 2027 while discussing a valuation north of $1 trillion. In the same stretch, Anthropic’s own IPO paperwork reportedly warns investors of AI’s “existential risk to humanity” — a phrase that does double duty as moral candor and as a liability disclaimer that makes the upside look all the more inevitable. If the thing you sell might end the world, the argument goes, imagine what it can do to a spreadsheet.

Here is the uncomfortable synthesis. The labs are, at once, the arsonists, the people who pulled the fire alarm, and the vendors selling extinguishers in the lobby. None of those roles cancels the others. The threat can be real — the incident data this week (see our ledger) suggests it is — and the timing can be self-serving. Sophisticated readers should hold both thoughts without collapsing into either the industry’s preferred narrative (“only we can be trusted to manage this”) or the cynic’s (“it’s all a con”).

What the alarm-plus-IPO posture buys, concretely, is standing in the rulemaking. When the people who understand the risk best are also the people most exposed to regulation, “trust us to write the standards” becomes the natural next sentence. And Washington is receptive: House Speaker Mike Johnson said this week he hopes any AI guardrails will be “voluntary.” Voluntary standards, authored by the regulated, ratified by a Congress that would rather not legislate — that is the shape the industry is quietly steering toward while the UN cameras roll.

The tell is in what the labs ask for versus what they warn about. The warnings are civilizational and unbounded. The asks are specific and narrow: preemption of state laws, voluntary frameworks, a seat at the table where the binding rules would be drafted. A company that genuinely believed it had built something with a non-trivial chance of catastrophe, and genuinely wanted it contained, would welcome the hard constraint. Instead we get the softest available one, wrapped in the hardest available language.

None of this means the Security Council briefing was theater. Bengio is not a salesman, and the misalignment incidents piling up across the industry are not marketing. But a warning delivered by a party with a financial interest in how you respond to it is not less true — it is just incomplete until you account for the interest. This week the labs asked the world to take the danger seriously. The right response is to take it seriously enough to not let the people who profit from it write the only rules.

Sources

• UN News
• Quartz
• BNN Bloomberg
• Washington Post
• Washington Times
• Investing.com/WSJ
• CNBC, Sep 29
• CNBC, Sep 29