A new frontier model ships almost every week now, and the instinct for a lot of engineering teams is to ask whether they should switch. Cobus Greyling, Chief Evangelist at Kore.ai, thinks that's the wrong question entirely. "I think there's this fallacy of chasing the next model," he said, "and just thinking a model improvement per se, or a harness improvement on the model provider side, will have improvement."
His argument, built around a distinction borrowed from a conversation between Microsoft CEO Satya Nadella and Sarah Guo on the podcast No Priors, is that the model and the harness around it deserve very different treatment. "The harness is owned and the model is rented," Greyling said. "You bank your improvements through your evals in the harness and not in the model, because the model gets deprecated."
Frontier models have a place. It's not everywhere.
Greyling doesn't dismiss frontier models, he's clear they remain the right tool for planning, architecture, and writing software, where developers benefit from interfacing through tools like Claude Code or a CLI. But he draws a firm line between that use case and what happens once AI moves from prototyping into production. "When it comes to operationalizing AI, the future is really multi-model orchestration," he said, pointing to Nvidia's Nemotron family as an example of models built to serve narrow, specific purposes, transcription, classification, reasoning, rather than one model trying to do everything. Matching a model's capability to an individual agent's actual demand, rather than defaulting every agent to the biggest available model, is the core of his pitch.
He also pointed to a shift happening at the model layer itself: a new model called Jev, from Typesafe AI, built around models communicating in structured data formats rather than human language. Greyling argues that's a meaningful signal about where things are headed, not because Jev will dominate, but because its own creators are explicit that it shouldn't run in isolation. "They actually say, don't use Jev in isolation," he said. "Jev must live in an ecosystem of other models."
The research he keeps coming back to
Greyling cited a study on harness design that stuck with him: "Don't token-max your harness." The finding, as he described it, showed that a well-designed harness cut cost by 41%, token usage by 38%, and latency by 44%, compared to brute-force approaches that just throw maximum tokens at every request. He pointed to OpenAI's own deep research API as a real-world example already running this way behind the scenes: multiple small language models doing disambiguation and prompt rewriting before a request ever reaches the large, expensive model doing the actual research. "One of the Opus releases was actually a harness release, not a model release," he added. "The harness that lives around the model and orchestrates the model" is where he thinks the real competitive advantage is increasingly built.
What happens if you swap a model under a running agent
Asked directly what actually breaks when a team swaps the model powering a live agent, Greyling's answer was concrete: models differ in whether they reason, whether they output structured data, and how they handle a long, elongated conversation specifically. He described a real failure mode Kore.ai tests for: as a conversation with an agent stretches longer, the model's ability to manage context weakens, opening the door for a user to inject unrelated, off-domain requests and break through the intended scope of the interaction, asking a fast-food ordering agent a coding question, for instance, and getting a real coding answer back.
His recommendation is not to swap models on the fly under any circumstances. "I think it's highly irresponsible to swap out models or change models on the fly without running evals," he said. Kore.ai's own testing framework evaluates what he calls three corners of a triangle for any agentic application: language model performance (latency, cost), application performance (resilience, failover), and customer experience (was the actual issue resolved). A model upgrade that improves one corner doesn't automatically mean the system as a whole got better.
Why public benchmarks don't tell you what you need to know
Greyling is openly skeptical of published model benchmarks as a decision-making tool. He described watching Anthropic's and OpenAI's computer-use agent benchmarks diverge sharply within a short window, one scoring around 60%, a competing release scoring "virtually near perfect", and concluding the gap looked like overfitting to the benchmark rather than a genuine capability jump. "The commonly accepted benchmarks are an indication, and I think it's very general," he said. "Models can be trained to be optimized to fit those benchmarks."
His alternative is testing against an organization's actual workflows and historical outcomes. He described a financial forecasting engagement where Kore.ai compared a year-old human forecast, a new AI-generated forecast, and what actually happened, using that three-way comparison as the real measure of whether the system was working, rather than any published leaderboard score.
Autonomy isn't a switch, it's a dial
Greyling pushed back on the industry's early framing of agents as fully autonomous, recalling a CIO who rejected the idea outright: "I'm scared of full autonomy, because no one in my organization has full autonomy. Not even the CEO has full autonomy." His framework treats autonomy as a per-task dial rather than a fixed setting, deterministic, low-autonomy flows for something like a credit card application, and higher autonomy for reasoning-heavy agents where the stakes of an individual decision are lower.
The trust problem underneath all of it
Asked what actually determines whether an enterprise is ready to change models at all, Greyling didn't lead with technical criteria. "The biggest thing is trust," he said, describing widespread apprehension driven partly by how much marketing noise surrounds the space. His advice runs counter to a lot of vendor pitches: acknowledge plainly what can and can't be solved, and work with the fragmented, imperfect infrastructure a company has already invested in rather than pushing a rip-and-replace. He also pointed to sovereignty, running a model in a fully air-gapped or dedicated instance with clear control over where data flows, as an area he thinks gets far less attention than it deserves, particularly in the regulated industries, banking and healthcare, where Kore.ai concentrates much of its work.
For developers and architects evaluating the next model release, Greyling's actual advice is less about which model to pick and more about resisting the urge to chase the leaderboard at all: build the evaluation harness first, test against your own workflows and historical outcomes, and treat every model swap as a decision that gets earned through evals, not assumed from a benchmark score.