Skip to content
Writing
Thread

The harness is the biggest lever, not the model

Models set the ceiling on performance; your harness determines how close you get to it. With the current generation we're already working inside an aircraft hangar and most people are only getting a few feet off the ground.

AIBuilding7 posts
01

With Gemini 3.5 Pro and Opus 5 launching soon, we're about to enter another model hype cycle. Everyone will focus on marginal benchmark gains while continuing to ignore the biggest lever for improving model performance: their harness.

Because, while models set the ceiling on performance, your harness determines how close you get to it. And with the current generation of models we're already working inside an aircraft hangar and yet most people are only getting a few feet off the ground.

02

Harnesses matter more than model now because the quality of the model determines how good your output is on a single turn (one input, one output). But work isn't single turn, it's iterative. Authors don't sit down and write in one go, programmers don't code by opening an IDE and typing until the app is done. You take a few steps forward, look at the result, backtrack and edit before moving forward again in a slightly different direction. Harnesses take models through that same journey.

03

Harnesses determine how close you get to that ceiling because they drive how the model actually navigates through the task space. How it breaks the problem down, how it checks its work. What it does when it hits a wall. If you ever switch Claude Code into manual permissions approval from Auto, you realize how much it backtracks when it hits a dead end. How it iteratively discovers a solution instead of landing on an answer. Right now, that's where the magic is.

04

And the difference in performance is dramatic. It's why, if you've ever tried Opus in Copilot, it feels like an entirely different model from Opus in Claude Code.

The benchmarks make that tangible. Terminal bench shows a 18.4 point difference in performance across harnesses for one model, going from 58% pass rate to 76.4%. On the extreme end, one iOS benchmark found a 6x difference in benchmark performance for the same model. Same model, different harness.

05

From my own testing, with the right scaffolding (a well broken down set of tasks), I can't reliably tell the difference between Sonnet 5 and Opus 4.8, or even between Sonnet 5 and GPT 5.6 Sol. So if you're waiting for the next model release to unlock performance, stop. You probably have a massive ceiling above you and you're just not getting close to it. Look at your harness instead and you'll see much greater gains than switching to the latest and greatest model.

06

Now, that's not to say models don't matter at all, they obviously still matter. But it's the inverse of how people treat it: the weaker your harness, the more model choice matters, because you're asking it to one-shot the whole thing. Fix the harness and the gap between frontier models gets a whole lot smaller.

07

Model choice can also make a big difference within a harness. If you already have a well tuned harness then paying attention to how different models perform within it is important. A few percentage point difference in performance can compound dramatically over time as the work horizon gets longer. But, that step should come after optimizing your harness, not before it. Otherwise you're missing the most powerful lever in your arsenal and optimizing at the margin.

Originally on Threads