Something strange is happening in artificial intelligence and we are asking the wrong question about it. The companies spending more money than some countries to win the AI race have suddenly begun telling us that perhaps the race is going too fast. Anthropic’s Dario Amodei wants frontier laboratories to coordinate their pace of development. Sam Altman and other competitors have expressed support for slowing things down. The explanation offered is frightening enough: increasingly autonomous AI may be approaching capabilities we do not completely understand and may not be able to control. That may even be true. What almost nobody is asking loudly enough is why the companies currently winning the race are becoming interested in freezing the board at precisely the moment when technology is beginning to threaten the rules that allowed them to win it.
The answer matters because OpenAI, Anthropic, Google and the rest may be facing something considerably more dangerous to their businesses than another company building a better large language model. They may be approaching a world where building the largest language model is no longer the only way to build extremely valuable intelligence. Smaller models are beating frontier systems at specialized work. Reinforcement learning is letting companies improve models against their own environments instead of waiting for the great laboratories to improve them. AI systems are beginning to participate in AI research itself. Researchers are finding ways to train across distributed hardware. Diffusion models are challenging the assumption that language must be generated one token after another. Governments across the planet are building AI infrastructure specifically because they do not want their intelligence supplied through an American corporate faucet. None of these developments alone kills the frontier laboratory. Together they attack nearly every reason the frontier laboratory is supposed to remain permanently dominant.
This changes the slowdown debate completely. Amodei’s proposal is not simply that Anthropic should behave more carefully. He proposes coordination among frontier developers and acknowledges that some forms of coordination may require government assistance because antitrust law exists for the rather sensible reason that competitors are normally not allowed to organize a market amongst themselves. One of the advantages he identifies is that companies could gain additional safety time without sacrificing commercial advantage. Critics noticed immediately. Former FTC official Alvaro Bedoya warned against allowing incumbent companies to use safety coordination to exclude cheaper upstart rivals. If I slow down and you keep running, I lose. If we agree that everybody currently at the front slows down together, none of us loses our place. That is not merely a safety arrangement. It has an industrial structure.
The Board Is Already Moving
The great assumption underlying the enormous AI valuations of the last several years is that intelligence will remain scarce. It can become vastly more powerful, certainly, but producing the most valuable forms of it will require extraordinary concentrations of capital, electricity, chips, researchers, data and datacenters. A few organizations will manufacture intelligence. Everyone else will buy it by the token, seat, subscription or API call. This is an extraordinarily attractive future if you happen to own one of those organizations, and investors have placed some extraordinary bets on precisely that future.
The technological evidence is beginning to suggest another possibility. Intelligence may become abundant before anyone figures out how to monopolize it.
That does not mean every teenager with a gaming computer is about to train GPT-whatever in the basement. It means something more commercially dangerous. Most customers do not need the most intelligent system on Earth. They need enough intelligence to do the job they are actually paying for, and “enough” is becoming cheaper at a ridiculous rate. The Financial Times has already been examining the threat from smaller and open models, because companies increasingly can obtain comparable utility on routine work with dramatically lower compute and energy requirements. Enterprises are not adopting open systems because they have suddenly become romantically attached to the philosophy of open source. They like cheap. They like private. They like controllable. Most of all, they like systems that already do the work.
This distinction between maximum intelligence and sufficient intelligence is probably one of the most important economic facts in the entire argument. A law firm does not care whether its document system understands astrophysics better than everything else on Earth. It cares whether the system understands its documents, cases, clients and jurisdiction. A logistics company does not need a machine capable of composing magnificent Elizabethan poetry while planning a delivery route. A bank does not particularly care whether the AI examining a transaction can explain the Punic Wars. The universal model is valuable because it can do everything. A specialized model can be more valuable because it does the one thing somebody is actually purchasing.
Now add reinforcement learning. Prime Intellect has made an argument that would have sounded mildly heretical two years ago: pretraining concentrated artificial intelligence inside a handful of laboratories, while reinforcement learning can distribute the ability to improve it. A company can take an existing model, expose it to its own work, reward successful behavior, punish failure and continuously optimize the resulting system against its actual environment. Prime Intellect has built an entire business around this proposition, and Ramp demonstrated the point particularly well by training a 35-billion-parameter specialized model that beat Claude Opus on its spreadsheet-search task while running faster and more cheaply.
That attacks the frontier moat from a direction that should make the frontier laboratories extremely uncomfortable because every serious company owns something OpenAI does not own: its own environment. A software company owns its code, tests, bug reports, deployments, telemetry and customer complaints. A bank owns financial interactions. A hospital owns clinical workflows. A manufacturer owns processes, failures, tolerances and production data. A logistics company owns millions of real decisions about moving real things through the real world. Once those environments can train machines continuously, the customer stops merely consuming intelligence manufactured elsewhere. The customer becomes part of the factory.
The consequences are unpleasant for anyone whose future depends upon selling all of those companies the same gigantic brain. Pretraining may continue producing the strongest general systems, but the frontier model increasingly becomes raw material rather than the finished product. OpenAI can still sell the steel. Somebody else gets rich manufacturing the automobile.
The Model May Not Be the Product
There is another problem hiding inside the word “model.” Public discussion still has the bad habit of treating the model and the AI system as if they were the same thing. They are not. Give an ordinary model persistent memory, tools, external storage, the ability to write and execute programs, subordinate agents, recursive delegation, a method for examining failure and enough time to try again, and the resulting system can perform work the original naked model appears unable to perform.
Prime Intellect’s work on recursive and persistent agent architectures is interesting for exactly this reason. The underlying model may remain unchanged while the system surrounding it becomes radically more capable. This matters because model weights are expensive. Software is comparatively cheap. If a meaningful portion of future progress comes from architecture, orchestration, memory and recursion rather than another gargantuan pretraining run, then a clever organization can extract much more intelligence from a model it did not spend billions training. The capital moat still exists, but somebody has found a tunnel.
The most serious tunnel may be artificial intelligence doing artificial-intelligence research. Sakana AI in Japan is worth watching far more closely than its size would suggest. Its Darwin Gödel Machine modifies the software governing its own agent, tests variants and preserves improvements. Its Recursive Self-Improvement Lab exists specifically to investigate the transition from static human-led AI research toward systems capable of driving more of their own improvement.
This matters because elite researchers are one of the strongest remaining concentrations of power in frontier AI. They are scarce, difficult to recruit, extraordinarily expensive and presently indispensable. A smaller laboratory cannot normally compete with an organization possessing thousands of such people. It does not need to if one excellent researcher can eventually supervise a hundred machine researchers that design experiments, modify code, execute tests, examine failures and preserve useful discoveries around the clock.
The machine researcher does not initially have to become smarter than the world’s best human scientist. That is the wrong benchmark. It merely needs to multiply one. Fifty exceptional researchers look hopelessly outmatched beside two thousand. Fifty exceptional researchers whose software lets each conduct the experimental work of fifty become something else entirely.
This is also the point where the commercial explanation and the Skynet explanation stop fighting each other and shake hands. Recursive improvement really is dangerous. A system capable of materially accelerating the creation of better AI compresses the time humans have to understand whatever comes next. The people working closest to these systems can sincerely fear that possibility while simultaneously understanding that whoever develops it first might leap over the existing competitive order. The technology most capable of frightening the frontier laboratories may also be the technology most capable of replacing them. We therefore do not need the simple accusation that everybody is lying. Their safety interests and their commercial interests can point in exactly the same direction.
I have been interested in this distinction between the system and the nominal model for considerably longer than the current panic. Red Anvil Creative’s Obsidian Tesseract: Critical Systems Accountability Infrastructure for High-Impact AI Deployments was deliberately designed around operational consequences rather than metaphysical arguments over whether a machine “really” thinks. Its central concern is reconstructability: what did the system do, with what inputs, under whose authority, using which version, and can an independent investigator reconstruct the decision afterward? That framework deliberately attaches accountability to deployment capability rather than merely to the origin of a foundation model.
The later Marble Tesseract extends the same problem into governance itself, including the rather important possibility that the institution supposedly watching the AI becomes captured. This is relevant because I am emphatically not arguing that advanced AI should be left alone because government regulation is automatically bad. I am arguing something much narrower and considerably less comfortable: regulations should follow dangerous capability and consequential use, rather than quietly becoming membership requirements for a small corporate guild.
The Architecture Can Change Underneath Them
Another threat comes from architecture. Autoregressive transformers currently dominate language AI so completely that people have begun treating their design choices like natural laws. They are not natural laws. They are engineering decisions.
Inception’s Mercury systems use diffusion rather than ordinary sequential autoregressive generation and have demonstrated extremely high generation throughput on ordinary Nvidia hardware. The significance is not that diffusion has already defeated transformers across the entire frontier. It has not. The significance is that the current way almost every famous language model generates language is not physically inevitable.
This distinction has destroyed incumbents before. Technological giants rarely disappear because another company performs their exact operation five percent better. They become vulnerable when somebody discovers that the expensive operation they perfected was not the thing customers ultimately needed.
Microsoft once appeared to control the strategic center of personal computing because Microsoft controlled the operating system. Google did not defeat Windows by building better Windows. It moved enormous economic value into another layer. I explored parts of that history years ago in The Sins of Silicon Valley, a long-form podcast and essay project examining how technical excellence, venture incentives, market concentration and political power interact. The Google and Microsoft installments are particularly relevant here because the mistake technological incumbents repeatedly make is believing that dominance of the current layer guarantees dominance of the next one.
The frontier laboratories are now spending staggering sums perfecting a giant general-purpose autoregressive model trained inside gigantic centralized clusters and served through corporate infrastructure. Somewhere right now, perhaps in Tokyo, Seoul, Paris, Boston or a room containing twelve people nobody has heard of, someone may be discovering that one of those adjectives is unnecessary. Maybe the model does not have to be giant. Maybe it does not have to be general purpose. Maybe training does not have to remain centralized. Maybe inference does not have to remain expensive. Maybe human researchers do not have to remain the primary engine of improvement. Maybe one giant model does not even have to be the useful unit of intelligence.
If somebody removes one of those requirements, the giants remain giants. If several disappear together, their present advantage can turn into an artifact of an older technological regime with astonishing speed.
Distributed training illustrates the point well because it is usually discussed badly. Enthusiasts pretend somebody can already build the world’s best model from random gaming PCs. Skeptics notice that this is false and stop thinking about the subject entirely. The actual work is more interesting. Nous Research’s DisTrO has attacked inter-GPU communication requirements directly, while Prime Intellect has demonstrated globally distributed reinforcement learning and continues developing training infrastructure intended to use geographically distributed compute more effectively.
None of this means today’s tightly interconnected frontier cluster has become obsolete. It means researchers are attacking one of the physical conditions that makes AI power so concentrated. Every substantial reduction in communication overhead makes more hardware economically interesting. University clusters become more useful. Corporate accelerators become more useful. National supercomputers become more useful. Geographically separated accelerators become more useful. Compute that cannot participate economically in today’s frontier architecture may participate in tomorrow’s.
The relevant question is not whether centralized clusters are superior today. They obviously are. The relevant question is whether anybody should bet a permanent industrial order on that remaining true.
Sovereignty Is Not a Chinese Problem
The rest of the planet has begun reaching another conclusion Silicon Valley may have underestimated: renting intelligence from a handful of American corporations is strategically stupid.
The Center for a New American Security has counted 184 government-backed sovereign-AI projects across 67 countries. That matters far more than another headline about the United States racing China because this is not fundamentally a China story. Europe wants greater technological autonomy. Japan explicitly talks about AI sovereignty. India is building domestic infrastructure. South Korea is developing models, compute capacity and alternative accelerators. Gulf states are building national AI organizations. Southeast Asian countries want systems that understand their languages and institutions. African and Latin American governments increasingly speak in terms of ownership rather than consumption.
The motivation is not complicated. If intelligence becomes critical infrastructure, no serious government wants another country or private corporation to possess an unquestioned kill switch over it.
A foreign provider can change its prices. It can change its terms. Its government can impose export restrictions. A geopolitical disagreement can interrupt access. A corporation can withdraw a product. An API can disappear. A domestic system can therefore be economically rational even when it is somewhat worse. Governments do not optimize solely for benchmark performance. They optimize for resilience, national security, language, law, political autonomy and the ability to continue functioning when somebody else says no.
Cloud computing taught much of the world to place enormous portions of its digital infrastructure inside American corporations. Artificial intelligence arrived after governments had already watched that centralization occur. It also arrived in a geopolitical period filled with sanctions, export restrictions, trade disputes and abrupt changes of policy. The return of Donald Trump did not create the argument for technological sovereignty. It merely made a preexisting argument impossible to politely ignore. A friendly country can remain friendly while deciding that the machinery thinking through its military logistics, financial system, hospitals and civil service should probably not possess a foreign off switch.
I was writing about parts of this problem years before “sovereign AI” became fashionable. One section of The Sins of Silicon Valley dealt directly with the collision between corporate technology and sovereign state power. Another examined how enormous concentrations of technological wealth externalize costs that governments and ordinary people are eventually expected to clean up. The industry changed. The incentive structure did not.
This leaves the frontier laboratories facing technological decentralization from below and political decentralization from above at exactly the same time. Customers increasingly can build, adapt and operate their own intelligence while governments increasingly want to own the infrastructure underneath it. The first threatens margins. The second threatens addressable markets. Both undermine the assumption that humanity will eventually route its cognition through several gigantic American gateways.
When Safety Becomes a Moat
Now return to the sudden desire to slow down.
A genuine capability-based safety regime would say that any AI system demonstrating particular dangerous abilities must satisfy particular requirements before deployment. OpenAI could fail that test. Anthropic could fail it. Google could fail it. A startup could pass. A university could pass. A European laboratory could pass. A Japanese laboratory could pass. The identity of the developer would be irrelevant because the rule follows the danger.
This is closely related to the argument I made in How to Certify AGI. If nobody can give a useful operational definition of “intelligence,” arguing over whether a machine has crossed some mystical AGI threshold is a terrible basis for law. Test what the system can actually do. Measure consequences. Certify capabilities. Regulate risks that can be demonstrated rather than allowing corporate branding or philosophical fashion to determine who is dangerous.
A very different structure emerges if the current frontier companies coordinate development rates, influence the standards defining responsible AI, participate in choosing or designing the evaluations, receive antitrust accommodations to coordinate with each other, impose enormous compliance expenses and create restrictions on weights, distillation or compute that are trivial for billion-dollar organizations and existential for everyone underneath them.
That system may still make AI safer.
It may also make the current companies permanent.
This is the point at which people should become impolite. The incumbents already possess the capital. They possess the chips, researchers, datacenters, customers, lobbying organizations, political relationships, distribution systems and brand recognition. If technological change continues at full speed, some of those advantages may cease to matter. A better architecture can appear. Machine researchers can multiply a small team. Open systems can become good enough. Specialized reinforcement learning can move value toward customers. Distributed training can weaken cluster concentration. Sovereign systems can remove national markets. Cheap inference can destroy margins.
But if regulation arrives first, the incumbents may be able to transform a technological lead into an institutional one.
They do not need a conspiracy to do it. They do not need to gather in a smoke-filled room and agree that the safety problem is bullshit. They can sincerely believe unrestricted AI development is dangerous. They can sincerely believe that only responsible developers should control systems this powerful. They can sincerely believe open models create unacceptable risks. They can sincerely believe their organizations have demonstrated enough maturity to deserve special trust. A man does not need to twirl his mustache before concluding civilization would be safer if civilization depended upon him.
The result can be capture without anyone ever admitting capture was the objective.
This is why the phrase “without sacrificing commercial advantage” matters. It is not proof of villainy. It is evidence that the commercial structure of the slowdown is understood by the people proposing it. The current leaders understand the difference between slowing themselves and slowing the field. Investors understand it. Antitrust lawyers understand it. Regulators should understand it before creating a safety framework that inadvertently converts temporary technical leadership into legal aristocracy.
The interesting thing is that there is a perfectly respectable answer the frontier laboratories can give to this criticism: accept regulation capable of actually hurting them.
Let independent institutions establish dangerous-capability thresholds. Let independent evaluators perform the testing. Keep antitrust law alive. Make compliance proportional enough that a twenty-person laboratory is not destroyed merely because Google can afford five hundred lawyers. Preserve experimentation and open research below genuinely dangerous thresholds. Update rules when architecture changes instead of writing today’s transformer stack permanently into statute. Make standards technology-neutral enough that a strange new laboratory can qualify without first becoming a member of the club.
Most importantly, create rules capable of telling an incumbent no. That principle already sits at the heart of the Obsidian Tesseract: accountability must attach to consequential actions and capability, with enough evidence preserved that an outside investigator can reconstruct what happened. Marble goes one step farther by treating capture of the accountability system itself as a failure mode. The watcher also requires watching.
If a safety regime can realistically prevent a new competitor from entering the market but cannot realistically stop OpenAI, Anthropic, Google or another incumbent from deploying a dangerous system, it is not a safety regime worthy of the name. It is expensive permission.
The Part We Keep Missing
The great irony is that the safety people may be right about the danger at exactly the same moment their critics are right about the economics. Increasingly autonomous AI may indeed become difficult to control. Recursive machine research may indeed accelerate progress beyond our institutional ability to understand it. Cyber, biological and other dangerous capabilities may indeed justify brakes. Those are reasons to build serious brakes. They are not reasons to hand the brake pedal to the drivers currently leading the race.
I have dealt with versions of this contradiction throughout the broader Tossing Grenades at Windmills project: powerful technologies can be genuinely useful, genuinely dangerous and genuinely convenient to the institutions that claim exclusive competence to manage them. Those propositions can all be true at once. The refusal to tolerate contradiction is usually where public analysis gets stupid.
There is also a final possibility that makes the entire debate look different. Perhaps nobody wins the artificial-intelligence race because eventually there stops being one race.
Computing does not have one winner. There are processors, operating systems, clouds, databases, networks, devices, programming languages and specialized machines. No single company owns computation because computation became an ecosystem. Intelligence may do exactly the same thing. Frontier models may coexist with sovereign models, local models, specialized models, machine researchers, agent architectures, open foundations, corporate reinforcement-learning systems, model routers and hardware optimized for particular kinds of thought.
If that happens, the organization that eventually Googles today’s Microsoft may look unimpressive right until it does not. It may begin with an obscure improvement in training efficiency, memory, recursive research, distributed optimization, inference architecture or specialized hardware. Somebody else solves another bottleneck. A third organization combines them. Open source absorbs the useful pieces. Suddenly a company with one-hundredth the resources can assemble a system that required a hyperscaler two years earlier.
That pattern is not exotic. It is the history of technology repeating itself with unusually expensive victims.
The companies at the frontier therefore have two excellent reasons to be frightened. They may genuinely fear that artificial intelligence is becoming too capable for humanity to control comfortably. They also have excellent reason to fear that artificial intelligence is becoming too cheap, too distributed, too sovereign and too easy for other people to improve for today’s leaders to control economically. Those fears can exist inside the same boardroom without contradiction. In fact, they fit together disturbingly well.
We should listen when the companies building the most powerful artificial intelligence on Earth tell us they are frightened. They know more about what their machines are doing than most of us do, and dismissing every warning because they have commercial interests would be reckless. We should also remember that frightened institutions do not become disinterested institutions. A company trying to prevent catastrophe remains a company. A corporation trying to save the world still has shareholders, competitors, payroll, capital requirements and a survival instinct.
So when the kings of artificial intelligence ask the rest of us to stop moving just as the board beneath their feet begins to change, we should listen very carefully to what they know.
Then we should look at their hands.
