DeepSWE's harness comparison made me curious, so I tried mini-swe-agent myself on a matched set of debugging tasks with GPT-5.6 SOL at High reasoning.
The current numbers surprised me:
- Codex CLI High: 151.91M tokens, 60% pass rate
- plain mini-swe-agent: 70.33M tokens, 67%
- the same mini harness plus macro execution, without changing the prompt: 60.59M tokens, 78%
So plain mini-swe-agent used about 54% fewer tokens than Codex and scored 7 percentage points higher. In the smaller ablation, changing only execution cut another ~14% of tokens and added 11 points.
That does not prove it is universally better. It is one matched benchmark slice, and DeepSWE itself notes that prompt tuning may explain part of the harness gap. Still, for debugging, the very small “bash + linear history” setup looks unusually strong.
Has anyone used it for day-to-day bug fixing or feature work? What breaks first outside a benchmark: repository setup, long-running tests, permissions, or patch quality?
Disclosure: I maintain Tura and ran the benchmark above.