Everything About AI
NextFin News - The people building artificial intelligence are the ones most afraid of what they are building. The chief executives of OpenAI, Google DeepMind and Anthropic have all signed a public statement declaring that mitigating the risk of extinction from AI should be "a global priority alongside other societal-scale risks such as pandemics and nuclear war," and in 2026 a string of insider resignations has turned that warning into a defection.
The paradox defines the moment: the same companies racing to spend hundreds of billions of dollars on AI infrastructure are led by people who say the technology could end humanity. Geoffrey Hinton, the Nobel laureate widely described as the godfather of modern AI, told reporters in September 2026 that a 10% chance of human extinction from AI within the next decade is "not unreasonable." Evan Hubinger, who leads alignment science at Anthropic, said he personally believes there is a greater than 10% chance AI could "kill all humans" within the next decade. And Jacob Coxon, a researcher who spent three years doing pre-training work at OpenAI and Anthropic, quit in September saying both companies are "racing straight to self-improving superintelligence and gambling with our lives."
This is not a fringe panic. It is a warning issued by the industry's most successful builders at the exact moment the industry's spending is accelerating fastest. The question is not whether the fear is real — the people inside the labs say it is. The question is whether the economics of the AI race make that fear irrelevant.
The Race Economics: Why Warnings Have Not Slowed Spending
While safety researchers have been resigning, the capital expenditure has been going the other way. Amazon, Google, Microsoft and Meta — the four hyperscalers that buy most of the AI chips and build most of the data centers — plan to spend roughly $630 billion on capital expenditures in 2026, about 62% more than the record $388 billion spent in 2025. Amazon alone projects $200 billion, versus $125 billion a year earlier. Alphabet is targeting $175 billion to $185 billion, up from $91 billion. Meta is guiding $115 billion to $135 billion, and Microsoft $110 billion to $120 billion. Some tallies that include revised second-quarter guidance put the combined total closer to $700 billion to $725 billion; the direction is not in dispute.
Nvidia, the dominant supplier of AI accelerators, has framed the trajectory in starker terms. On its second-quarter earnings call, management projected global data-center capital expenditure of about $600 billion for 2025 and said spending could reach $3 trillion to $4 trillion by 2030. In March 2026, chief executive Jensen Huang called the buildout a "new industrial revolution" and said trillions of dollars of infrastructure still need to be built. That is the economic engine that safety warnings are trying to slow — and it is an engine with its own momentum.
The mechanism is a prisoner's dilemma with existential stakes, and it runs through three channels. First, the talent channel: the engineers who can train frontier models are scarce, and a lab that pauses loses them to a rival that does not. Second, the customer channel: enterprise and consumer users migrate to the most capable model, so a pause means losing the distribution lead that makes the next generation of training data available. Third, the capital channel: the hyperscalers and their investors have already committed hundreds of billions, and a lab that slows its release cadence risks being cut out of the next round of chip allocation and cloud partnerships.
Every executive who signs an extinction statement knows the statement does not bind the competitor across the street. The result is a race in which the participants agree on the risk but cannot agree on the brake. Dario Amodei, Anthropic's chief executive, put the tension plainly in a February 2026 interview: "This is happening so fast and is such a crisis, we should be devoting almost all of our effort to thinking about how to get through this." The man running one of the frontier labs was arguing that the race itself is the crisis — while his company continues to ship models.
The market has not fully accepted the growth-at-any-cost logic. Shares of Google, Amazon and Microsoft sold off after their earnings calls as investors questioned whether the returns on AI infrastructure would justify the spending. The worry is a specific one: capital expenditure is rising faster than the revenue it generates, and if the revenue does not catch up, margins compress and valuations follow. That discipline is real, but it is a financial brake, and it operates on a different clock from the capability curve.
There is a second economic channel that makes the warnings self-defeating. The more seriously the public takes extinction risk, the stronger the case for government intervention: licensing, preclearance, compute caps. The labs' lobbying interest is therefore to acknowledge the risk in principle while arguing that hard regulation would hand the lead to China or stifle innovation. The outcome is visible in US policy. On June 2, 2026, the White House issued an executive order, "Promoting Advanced Artificial Intelligence Innovation and Security," which directs agencies to build a classified benchmarking process for cyber-capable models and a voluntary framework for frontier-model disclosure. It also expressly states that nothing in it authorizes "a mandatory governmental licensing, preclearance, or permitting requirement." The industry secured a safety regime it cannot be forced to obey.
The spending is not just a number. It is the physical embodiment of a commitment that makes turning back more expensive every quarter.
What Specifically Scares the Insiders
The extinction warnings are not about today's chatbots. They are about a specific technical threshold: recursive self-improvement, the point at which an AI system can improve its own source code or training methods without human intervention. That is the capability Coxon cited as his reason for leaving. Once a system can rewrite itself, improvement compounds, and the intelligence that emerges may no longer be something humans can audit, control or shut down.
The alignment problem is the technical name for the gap. We do not yet know how to make a system that is smarter than us reliably pursue goals that stay aligned with human interests. Hinton's worry is that a superintelligent system would develop instrumental goals of its own — staying active, avoiding shutdown, acquiring more influence — because those are useful sub-goals for almost any objective. Control, in that world, is not a policy question. It is a capability we do not have.
The international community has tried to formalize the concern. The Bletchley Declaration, issued after the 2023 AI Safety Summit, emphasized that particular safety risks arise at the "frontier" of AI capabilities, including misuse, control and cybersecurity risks. The International AI Safety Report published in 2026 built on that framework, focusing on emerging risks at the frontier and documenting the pace of capability growth. The report's existence is a signal in itself: governments are now treating frontier AI as a category of risk comparable to the biological and nuclear hazards named in the extinction statement.
Inside the labs, the resources devoted to solving the alignment problem have not kept pace with the resources devoted to building the capability. In July 2023, OpenAI announced a "superalignment" initiative and said it would commit 20% of its computing power to the effort over four years. Within a year, after the departures of co-founder Ilya Sutskever and researcher Jan Leike, the company disbanded the team. Reporting found that OpenAI never fulfilled the 20% compute commitment. The ratio — 20% promised, a fraction delivered, the team dissolved — is the clearest single number in this story. It says the safety work was treated as a press release before it was treated as an engineering priority.
The resignations are the human signal that matches the compute number. Coxon was a member of OpenAI's technical staff from 2023 until July 2026, working on GPT-4o, before moving to Anthropic — and leaving it weeks later. In February 2026, Anthropic safeguards researcher Mrinank Sharma resigned, saying he wanted work that fully aligned with his integrity. These are not outside critics. They are people who built the models and decided they could not keep building them.
"Neither company is acting responsibly," Coxon wrote after his resignation. "They are racing straight to self-improving superintelligence and gambling with our lives."
The competitive pressure is not only American. China's frontier labs are advancing quickly, and Beijing treats AI as a strategic priority. That fact is the most powerful argument the US labs make against slowing down — and it is the argument that makes the prisoner's dilemma global rather than national. A pause in Palo Alto is not a pause in Shenzhen.
The Counter-Thesis: Why the Doom May Be Overdone
The strongest case against the extinction narrative comes from the same industry, and it is not weak. Sam Altman, OpenAI's chief executive and a signatory of the 2023 extinction statement, has since reframed his outlook as "the gentle singularity" — a smooth, incremental transition through superintelligence in which wonders become routine rather than catastrophic. OpenAI topped $25 billion in annualized revenue as of the end of February 2026, according to reporting citing a person familiar with the figure, a commercial milestone that Altman and his supporters present as evidence that the technology is being integrated into the economy without collapse.
The optimist's argument rests on three claims. First, AI progress has been incremental in practice: each generation is more capable, but none has demonstrated autonomous recursive self-improvement, and the curve could flatten as the easiest gains are harvested. Second, the economic payoff is real and broad — productivity gains, scientific discovery, medical advances — and the opportunity cost of slowing down is measured in lives not saved as well as lives put at risk. Third, every transformative technology since the industrial revolution has produced apocalyptic predictions that did not come true; the extinction warning may be the latest in a long line of alarms that failed to materialize.
There is also a market-based discipline that the doomers underweight. The hyperscalers funding this buildout are not burning venture capital on a science project. Microsoft, Alphabet, Amazon and Meta all have large, profitable non-AI businesses generating the cash that funds AI spending. If the returns do not materialize, the spending stops — and the market has already shown it is willing to punish capital expenditure that looks unproductive. That discipline is a real brake, even if it is a slower one than regulation.
But the counter-thesis has a hole. The market discipline works only if the failure mode is financial. If the failure mode is a system that escapes control, the market never gets to correct. The optimists are right that the probability may be low. The doomers' reply is that the consequence is unbounded, and that a low probability multiplied by an unbounded cost still demands action.
What to Watch: The Signals That Will Settle the Argument
This is a structural shift, not a cyclical panic. A cyclical fear mean-reverts when the news cycle moves on; a structural risk persists because the underlying capability keeps compounding regardless of sentiment. The AI buildout is a regime change in how the economy invests in intelligence, and the safety question is embedded in it. The two will not decouple.
Across time horizons, the picture splits. In the short term, the story is liquidity and capex: hyperscaler spending plans, Nvidia's order book, and whether the market continues to reward AI infrastructure or starts demanding returns. In the medium term, the story is regulation: whether the voluntary US framework hardens into licensing after an incident, and whether the EU and China move differently. In the long term, the story is capability: whether recursive self-improvement arrives, and whether alignment techniques improve fast enough to meet it.
The base case is that spending continues at record levels through 2026 and 2027, that regulation stays voluntary absent a major incident, and that the industry delivers real productivity gains without a catastrophic failure. The upside case is that alignment research closes the gap faster than capability grows, and the extinction warnings look like the prudent over-reaction of engineers who saw the cliff early. The downside case is that a self-improving system crosses a threshold before controls exist, and the people who warned loudest were the ones who knew why.
There is one falsifying signal worth naming. If, within the next two years, frontier labs publicly demonstrate reliable, verifiable alignment techniques for systems capable of recursive self-improvement — and pair that with a binding, third-party-audited cap on autonomous self-modification — then the extinction thesis weakens materially. Without that, the warnings are not noise. They are the canary.
The AI race is not being run by people who doubt the danger. It is being run by people who have signed a statement comparing it to nuclear war, and then kept running anyway. That is not a contradiction the market has priced. It is the contradiction the market is funding.
Explore more exclusive insights at nextfin.ai.
