Since the arrival of ChatGPT in late 2022, the dominant AI technology narrative was that large, general-purpose “foundation models” could serve many use cases: everything from writing software code to creating marketing plans, summarizing meetings, analyzing contracts, and more.
But as we approach 2026, the tide is shifting, and it is becoming apparent that to serve targeted business use cases optimally, AI models work best when they are at least somewhat specialized. What’s more, even providers of state-of-the-art AI models are actually delivering their products and services as “mixtures of experts” (MoEs): collections of task-specialized models hidden behind a unified front-end, where each request (prompt) is routed to the specialized model that fits best.
According to IDC’s 2026 AI and Automation FutureScape, by 2028 70% of top AI-driven enterprises will use advanced multi-tool architectures to dynamically and autonomously manage model routing across diverse models.
The shift: From model selection to model orchestration
Before the rise of GenAI, choosing the best AI model architecture was a key step to success in driving an effective outcome. Even now, in GenAI implementations, many teams spend significant time trying to find the “best model” to serve a given use case. Teams scour model benchmarks, run tests, compare outputs, select a winner, and optimize around it.
However we are currently seeing an explosion of innovation and engineering advances in AI models, and what might be “best” today might very well not be best in six months (or even next month). What’s more, agentic AI systems demand flexibility. Different tasks that agents are being called on to execute are likely to require different kinds of capabilities.
Model routing enables teams to build systems that evaluate incoming requests and automatically route them to the model best suited to the job; or even combine models in sequence to produce an optimal outcome.
Why this matters: Performance, cost, and trust
The value of model routing is about more than insulation from technology entropy; it’s also about optimizing performance, cost and trust.
- Performance: Model routing enables systems to boost accuracy and reliability by dynamically selecting the most context-appropriate model rather than forcing a generalist to handle every request. In addition, models can be selected based on where they run – at the edge, on premises, in a public cloud, for instance – due to the impact on latency as well as cost.
- Cost control: With routing, workloads can be distributed intelligently between premium proprietary models, where needed, and efficient open-source alternatives.
- Governance and trust: Enterprises can enforce compliance and sovereignty by ensuring certain data types are always processed by approved, region-specific, or private models.
How leaders should prepare
So what does all this mean for leaders trying to put model routing into practice? It starts with shifting mindset, strengthening oversight, and designing for flexibility from day one.
- Adopt a multi-model mindset. Stop optimizing around a single model and start designing architectures that can host and switch between many.
- Invest in AI governance and observability. Model routing introduces another layer of technology, and you will need monitoring systems that track system performance, quality, and cost across every route and over time.
- Explore blends of open and proprietary models. Understand that state-of-the-art proprietary models can deliver great results, but cost and flexibility can suffer. Open models – which are massively easier to specialize, and offer deployment flexibility – may fit individual use cases very well.
IDC perspective: Routing is the road to scale
For businesses, delivering AI value at scale is about using a variety of levers to optimize results – it’s not about leveraging ever-larger models. Model routing is one of the main levers that will become increasingly important.
As businesses confront the crosscurrents of data sovereignty, compute costs, and model diversity, routing architectures provide a key tool to navigate complexity. Model routing helps organizations treat AI-powered automation as a distributed, orchestrated capability, rather than a monolith. Those who master routing will move faster, spend less, and innovate more safely. Those who don’t will watch their single-model strategies stall under the weight of their own limitations.
By 2028, 70% of top AI-driven enterprises will use advanced multi-tool architectures to dynamically and autonomously manage model routing across diverse models, orchestrating complex processes.
Download IDC FutureScape 2026: AI & Automation Predictions to explore how routing, orchestration, and agentic architectures will redefine the enterprise technology stack.
