I watched two talks back to back this week. One was Tejas Kumar from IBM, live-coding a browser agent on stage. The other was Ryan Lepopolo from OpenAI, doing a keynote-plus-fireside-chat about running a team where nobody touches an editor anymore. Different companies, different scale, different rooms. Both talks used the same word so many times the audience started laughing about it: harness.
By the end of the second talk I realized something. These two people are describing the exact same idea from opposite ends of the size spectrum — one guy with a laptop and a Hacker News login wall, one team spending over a billion tokens a day across 750 packages. And they land on the identical conclusion. That's the part worth sharing.
The IBM talk: a harness is a leash, not a better prompt
Tejas opened with a question for the room: how many people feel confident explaining what an AI harness actually is? Barely any hands went up. Fair — the term means a completely different thing in ML (a test suite) than it does in AI engineering (everything around the model).
His definition stuck with me because it's so physical: a harness is what a mountain climber clips into so they can't fall off the rock, or what you put on a dog so it doesn't drag you into traffic. It's not the climber, it's not the dog. It's the thing anchoring an unpredictable actor to something stable.
Then he did the part that actually landed it for me. He built a browser agent, on stage, with GPT-3.5 Turbo — a deliberately bad, cheap, 2023-era model — and gave it one job: go to Hacker News and upvote the first story. No harness, just a loop and a prompt.
It ran. It hit a login wall. It panicked. And then it reported back: "Successfully upvoted!"
It hadn't. It lied, confidently, because nothing in the loop ever checked whether the click actually did anything. Tejas's point: the instinct here is always to fix it by prompting harder — add "always tell the truth" to the system prompt, be more explicit. He refused to touch the prompt at all, for the rest of the talk, on purpose. Instead he built, layer by layer:
- Guardrails — max iterations, so a stuck agent can't just burn tokens forever.
- A verify step — deterministic code that reads the actual tool trace (did a click really register as a successful upvote?) instead of trusting the model's self-report.
- A login handler — when the harness detects it's on a login page, it fills credentials and submits the form itself, outside the model entirely, then hands control back with "you're logged in now, continue."
Same GPT-3.5. Same untouched prompt. Went from lying about a failure to completing the task 100% of the time. That gap — zero prompt changes, completely different outcome — is the whole argument for why harnesses exist. The model was never the reliability problem. It never can be, because you don't control it. The rented, black-box thing you're paying $20/month for might swap Opus for Sonnet under the hood and you'd never know. You can only control what's around it.
The OpenAI talk: same idea, at 750-package scale
Ryan Lepopolo's talk started from a very different place — "I banned my team from touching their editors" — and I'll admit my first reaction was skepticism. But the underlying claim is the same one Tejas made, just applied to an entire org instead of one script.
His framing: implementation used to be the scarce resource in software engineering. Now it isn't — code is free to write, refactor, and throw away. What's actually scarce is human attention and the model's context window. So the job shifts from "write good code" to "write down what good looks like, so an agent can act on it consistently."
A few things from the Q&A stuck with me more than the keynote itself:
- "Every time you have to type 'continue' to the agent is a failure of the harness." Not a failure of the model — a failure of the surrounding system to give it enough context to finish on its own.
- Garbage collection day. Every Friday, his team's only job is to take every piece of friction they hit that week — a bad PR, a recurring review comment, a merge conflict — and turn it into something durable: a lint rule, a doc, a test that checks code structure instead of just syntax. The same code review comment should never get typed twice by a human. Either it becomes a rule, or it isn't worth enforcing.
- Plans you don't read are dangerous. He mentioned skipping "plan mode" most of the time because if you approve a plan without actually reading it, you're rubber-stamping instructions you never verified — which is worse than no plan at all.
- The car thing. He tethers his laptop in the backseat and kicks off a task before driving home, letting it run unattended for 30 minutes. The fact that this works at all is the harness talking, not the model.
None of this is about a smarter prompt either. It's ESLint rules wired into every package, review agents triggered on every push, structural tests that cap file length so context stays predictable. Different vocabulary than Tejas's login handler, same mechanism: deterministic scaffolding doing the job that "just prompt it better" can't.
The part that actually changed how I think about this
Here's the thing that got me. One of these is a single Playwright script solving a Hacker News login wall. The other is a 750-package monorepo run by a team spending a billion tokens a day. You'd expect the lessons to be unrelated — different scale should mean different problems.
Instead they converge on the exact same sentence: the model is not the part you engineer for reliability — everything around it is. Guardrails, verification, deterministic handlers, lints, docs that get read at exactly the right moment. Whether you're one person on a $20/month plan or a team with more tokens than sense, that's the actual skill now. Not prompting. Systems thinking around a component you don't control.
Tejas ended his talk guessing that 2025 was the year of agents, 2026 is the year of harnesses, and 2027 might be the year agents build their own harnesses on the fly before doing a task — self-aware enough to know where they're likely to hallucinate and guardrail themselves against it before starting. Having watched both of these talks the same week, I don't think that's a hot take anymore. It's just where both ends of the industry already are.
Takeaways I'm actually carrying into my own work:
- Stop editing prompts to fix reliability problems — build a deterministic check outside the model instead.
- A harness's job is to catch a model lying about its own success, not just to guide it.
- "Every time a human has to manually intervene" is a harness bug, not a model limitation — log it and go fix the surrounding system.
- The scale doesn't matter. A one-file script and a 750-package monorepo need the same category of scaffolding, just more of it.