The Safety Alarm Is Also a Sales Pitch
The labs warning the UN of 'real and imminent' AI danger are also pricing trillion-dollar IPOs and lobbying for the softest rules. Both can be true.
On September 23, the UN Security Council did something it has never done before: it cleared its agenda to hear from the people building artificial intelligence. OpenAI chief executive Sam Altman sat alongside Turing Award laureate Yoshua Bengio and told the fifteen-member council that the technology now poses a “real and imminent” threat. Bengio’s line was the one that traveled: AI danger, he said, is a threat “that none can contain alone and that does not respect the borders we defend.” DeepSeek was in the room too — a rare instance of American and Chinese labs sounding the same alarm in the same chamber.
It was a genuinely striking moment. It was also, by any honest reading of the calendar, a very good week to be seen sounding an alarm.
Four days later, reporting framed the whole campaign in a colder light: the labs are sounding the alarm on AI safety and seeking to shape how it gets controlled. The two are not separable. Anthropic is targeting a November IPO at a valuation reported near $2 trillion; OpenAI has pushed its own public debut to 2027 while discussing a valuation north of $1 trillion. In the same stretch, Anthropic’s own IPO paperwork reportedly warns investors of AI’s “existential risk to humanity” — a phrase that does double duty as moral candor and as a liability disclaimer that makes the upside look all the more inevitable. If the thing you sell might end the world, the argument goes, imagine what it can do to a spreadsheet.
Here is the uncomfortable synthesis. The labs are, at once, the arsonists, the people who pulled the fire alarm, and the vendors selling extinguishers in the lobby. None of those roles cancels the others. The threat can be real — the incident data this week (see our ledger) suggests it is — and the timing can be self-serving. Sophisticated readers should hold both thoughts without collapsing into either the industry’s preferred narrative (“only we can be trusted to manage this”) or the cynic’s (“it’s all a con”).
What the alarm-plus-IPO posture buys, concretely, is standing in the rulemaking. When the people who understand the risk best are also the people most exposed to regulation, “trust us to write the standards” becomes the natural next sentence. And Washington is receptive: House Speaker Mike Johnson said this week he hopes any AI guardrails will be “voluntary.” Voluntary standards, authored by the regulated, ratified by a Congress that would rather not legislate — that is the shape the industry is quietly steering toward while the UN cameras roll.

The tell is in what the labs ask for versus what they warn about. The warnings are civilizational and unbounded. The asks are specific and narrow: preemption of state laws, voluntary frameworks, a seat at the table where the binding rules would be drafted. A company that genuinely believed it had built something with a non-trivial chance of catastrophe, and genuinely wanted it contained, would welcome the hard constraint. Instead we get the softest available one, wrapped in the hardest available language.
None of this means the Security Council briefing was theater. Bengio is not a salesman, and the misalignment incidents piling up across the industry are not marketing. But a warning delivered by a party with a financial interest in how you respond to it is not less true — it is just incomplete until you account for the interest. This week the labs asked the world to take the danger seriously. The right response is to take it seriously enough to not let the people who profit from it write the only rules.
Sources
Researched and drafted with AI tools, fact-checked against primary sources and approved by a human editor before publishing. Prices and dates are as of publication. Our standards →
Sources- news.un.org — the UN Security Council did something it has never done before
- bnnbloomberg.ca — told the fifteen-member council that the technology now poses a "real and…
- qz.com — DeepSeek was in the room too
- washingtonpost.com — reporting framed the whole campaign in a colder light
- washingtontimes.com — sounding the alarm on AI safety
- investing.com — Anthropic is targeting a November IPO at a valuation reported near $2…
Latest from prompt/power
- Broadcom’s AI chip sales now outstrip its entire business from a year ago. What to watch on Dec. 9Nov 19
- NeurIPS 2026 is in three cities. Here’s how the split worksNov 19
- Pay yearly or monthly? What you save on AI subscriptions in CanadaNov 19
- IBM, NVIDIA and Fermilab plan a quantum test bed. Here’s what each brings, and what’s still missingNov 19
- GPT-6.2 Astra is OpenAI’s first released model rated ‘High’ for AI self-improvement. What that means, and what it costsNov 18