The Token Apocalypse & Agentic Ecosystem Design

How can simple agents and small models can create a more reliable form of agentic intelligence than continued reliance on frontier models for everything

The path towards reliable agentic AI infrastructure in the enterprise continues to offer up new surprises, and the recent panic around token costs is a case in point that has focused minds on the cost-benefit analysis of different models. But perhaps model choice is not the only answer.

“Something has gone completely wrong”

Palantir CEO Alex Karp last week blasted the frontier model providers for seeking to reduce their own deficits between compute cost and revenue by effectively increasing token costs for most customers. Karp made the point to CNBC that “Something has gone completely wrong” with the frontier model approach, and argued that businesses he speaks to are secretly furious about the unpredictable cost increases they have faced.

Michael Spencer believes this explosion in costs combined with recent US political interference in AI model availability has created the conditions in which more companies will explore open models as the basis for their agentic AI systems:

The Token Apocalypse is that crucial moment where companies realize switching to Chinese open-weight models or open-source models made in the West has become a business necessity to get the most of AI agents without breaking the bank. I believe July, 2026 is that moment. If I’m right, at the scale we are going to see this trend, it could have a material impact on slowing down the growth of AI behemoths and model makers like Anthropic, OpenAI, Google, SpaceX, Meta and others. That is also a big deal for the AI bubble.

Open-source inference is now a billion-dollar infrastructure category in its own right, with enterprises cutting inference costs up to 60x versus closed alternatives.

Research analysed by Optimum Partners shows that 73% of enterprises report AI costs exceeded projections, even though token prices dropped by a similar proportion year-on-year. The reason is that agentic workflows multiply token usage 5–30x per task, and a large proportion of total AI cost now sits outside the model bill, for example in orchestration, retrieval, retries and observability. So this is not just a question of model cost inflation, but also a system design problem.

Frontier model providers like Anthropic have started responding to token anxiety with cost control measures that seek to provide re-assurance to enterprise leaders. But there are other reasons beyond concerns about vendor lock-in and token cost are leading enterprises to consider building at least a proportion of their agentic infrastructure using open, possibly self-hosted models, such as better cost and operational control (including security), model fine-tuning, and minimising latency for multiple round-trip reasoning steps.

Jaroslaw Wasowski recently wrote about how to decide between open and frontier models for certain tasks, and concluded that it is foolish to pose this as a binary question. Instead the real test was to ask “which task class, at what volume, measured on my data,” quoting economic analysis by MIT and Georgia Tech from November 2025 to remind us that:

Open models today achieve roughly 90% of the quality of closed models on real-world tasks, at approximately 87% lower cost per token. And yet closed models capture close to 96% of revenue on inference platforms.

Hybrid routing and fine-tuning

To optimise this kind of hybrid strategy - using the right models for the right tasks - requires intelligence at the router level, which directs tasks to agents and models it decides are most appropriate, and also needs to have escalation options if tasks prove harder than they look.

LLM routing is now a mature tooling category, with purpose-built solutions that route on cost, latency, and task complexity in real time, and many enterprises are running five or more models in production.

Frontier models are still the best solution where complex multi-stage reasoning is needed, or where long-running supervisor agents need very low error rates; but for simple, repetitive tasks run by a sub-agent, open models can save a lot of money and offer more opportunity for fine-turning based on proprietary data and knowledge. But for self-hosted models, even if a company has the skills and infrastructure needed, the front-loaded costs are high, meaning the break even point where the upward slope of frontier model API costs exceed the flatter curve of self-hosting costs depend on model size and infrastructure. It could be as low as 1m tokens per month for serverless GPU hosting and small models, but several orders of magnitude higher for a sophisticated setup with larger models.

The open model route is not for the faint-hearted. Recently released research from VentureBeat found that 45% of firms surveyed who are doing their own fine-tuning report falling into a Sandbox Graveyard (25–75% success with fine-tuning) that ultimately proved too costly or complex and got stuck in development, with only 27% reporting high success (>75%).

There is clearly a need for better ways to fine tune agents and the models they use in enterprise settings, but it is not yet clear whether this needs the hard work of model training, or whether specialist AI data platforms can achieve good-enough results.

Agentic ecosystem design

So what is a CIO to do when faced with big decisions that have long-term cost implications? Intelligent routing and keeping an eye on all the model variants and their token costs is advisable. And perhaps a middle ground of using cheaper cloud-hosted open models for simple workflows, reserving frontier models for the tasks that really need them, is a good way to assess whether self-hosted models might be an appropriate scaling option.

But model selection is the wrong problem to be solving. The real issue is ecosystem design.

In nature, and in other areas of technology, small, single-purpose ‘agents’ dedicated to their own fitness function and survival are one of the most powerful forces of evolution when they exist within ecosystems that allow them to cooperate and compete to maximise collective outcomes.

If we design our enterprise AI agentic capabilities with this in mind, and use the smarter ‘brain’ agents to direct and coordinate simpler ‘autonomous’ functions, then we can do a lot with very little compute.

If we assume a realistic level of error and output decay is inherent in the way models work, we could use multi-agent systems to check and challenge each other’s work with a view to creating a kind of ‘quantum’ error correction. Just as quantum computing achieves reliable computation from ensembles of unreliable qubits, well-designed multi-agent systems can aggregate uncertain individual outputs into robust collective decisions — but only if the architecture is built for it from the start.

The real lesson from the token panic isn’t to find a cheaper frontier model, but to stop treating agentic AI as a model-selection problem. A better frame is ecosystem design. In nature, powerful collective behaviour emerges not from increasingly powerful central intelligence but from simple, specialised agents acting within rules that make cooperation and competition productive. Enterprise agentic systems work the same way: route simple, high-volume tasks to fast, cheap, fine-tuned models; reserve frontier intelligence for genuine complexity and long-running coordination; and design your multi-agent architecture so that agents check each other’s work rather than any single answer being trusted implicitly.

The CIO’s job is not to find the best model. It’s to design the right ecosystem.