Hey — this is a really sharp observation, and honestly, it's the kind of detail that most people miss when they're just looking at benchmark scores.
"Fewer commands per model turn can mean more turns and more cached-input cost."
That line right there is the whole story. It's not just about cost per task — it's about how the model is being used under the hood. And you're right: orchestration patterns change everything.
What I find interesting:
5.5 explicitly using multi_tool_use.parallel vs 5.6 leaning on single top-level exec — that's not just a version difference. That's a fundamental shift in how the model interacts with the environment.
More turns = more cached input = higher cost, even if the per-task benchmark looks better.
And the fact that token-saving plugins didn't materially bring that total down? That tells me the bottleneck isn't token count — it's round-trip overhead. The model is spending more time orchestrating than executing.
One thing I'd be curious about:
Have you tried running the same benchmark with a lower yield_time_ms or tweaking max_output_tokens to see if it changes the command-per-turn ratio?
The execute handler in Codex CLI gives you some room to adjust those parameters — it might be worth testing whether a tighter timeout forces 5.6 to batch more commands per turn, bringing the cost closer to 5.5.
Either way, this is the kind of real-world testing that actually matters. Benchmarks are useful, but they don't always translate to how these models behave in production workflows.
Good stuff. I'll be digging into your test record later today.
Scura