← blog

· INIT Labs

LLMs should write their own harness

INIT Agent is a bet on the post-AGI models that can.

Every agent harness is written for the models its authors had at the time.

A harness is everything around the model: the system prompt, the tools, the loop that calls the model again. Most of it got there the same way. At some point a model kept losing track of its plan halfway through a task, so someone gave it a todo list to keep the plan in. That worked, and it shipped.

A year later the model is three versions newer and doesn't lose its plan any more. The todo list is still there, still described to the model on every request, because nothing in the code says which model it was written for. Do the same for every reminder, every limit and every cleanup routine and you have a harness. It is a pile of fixes for problems that some earlier model had.

Four parts of a harness under three models: all four still hold for the model they were written for, and none for a model better than its authors.

I ran into this last month while looking for something else. I wanted a task that my own agent fails because it has no todo list, so that I could watch the agent build one for itself. As a comparison I ran Claude Code, which has a todo tool, on the same small, cheap model.

I never found the task. My agent finished every list of requirements I gave it in a single run, with no todo list at all. On the very large jobs where it failed, Claude Code failed as well, at two to three times the cost. And in five runs, Claude Code never called its todo tool.

That doesn't say much about either product. It is one small model on a handful of tasks, and Claude Code was built for Claude models. But the picture stayed with me. Someone had laid out a tool for a model, and a later model walked straight past it. I had been trying to get my agent to build the same tool.

The public numbers point the same way. Terminal-Bench scores coding agents on tasks in a terminal, and it lets you compare harnesses that run the same model. Its own reference agent, Terminus 2, has exactly one tool, a terminal. On version 2.0, Claude Code scored below Terminus 2 on every Claude model that both have verified runs for, by one to six points. The chart shows one of those models. Look at the top two bars for now.

Terminal-Bench 2.0 with Claude Opus 4.6 in four harnesses: Claude Code 58.0%, Terminus 2 62.9%, Terminus-KIRA 74.7%, Meta-Harness 76.4%.

Small harnesses keep turning up near the top. On version 3.0 of the benchmark the first row is mini-SWE-agent, which is about a hundred lines of code and has only bash. Pi, whose prompt and four tools fit in under a thousand tokens, came within about two points of Claude Code on the same model in its author's own run.

The evidence isn't all one way. OpenAI's models score five to fourteen points higher in OpenAI's own harness, and on a newer version of the benchmark Claude Code is back ahead of Terminus 2 by about three points. I read that as the same point from the other side. A large harness earns its weight on the models it was built around, and has to earn it again with each new one.

A harness that only grows

I don't think this is anyone's failing. A harness that ships to everyone has to carry every fix that any model or any user ever needed, and it is rarely safe to take one out, because someone still depends on it. So it grows. Every user then pays for all of it on every request, whether or not their model or their work needs it.

A toy model shows where that ends. Suppose fixes arrive at a steady rate, each one has the same chance of being outgrown with every model release, and none is ever taken out. Then the part that is still true levels off while the total keeps climbing. After four half-lives of a typical fix, about two thirds of the harness is dead weight, and that share only goes up.

A model of a harness that only grows: everything ever added rises in a straight line, the part still true of today's model levels off, and the gap between them is sent with every request. After four half-lives of a typical fix, two thirds is stale.

I don't see how that scales, and the easiest way to see why is to push it to the end. Suppose you hire the best engineer you have ever worked with, and their first day goes like this.

In the morning you hand them a notebook with the headings already filled in, and a card to keep on the desk and read again every twenty minutes. By noon the desk is full, so a clerk comes by and clears away whatever is oldest, which today happens to be the error message they were in the middle of fixing. Partway through the afternoon a bell rings, and they stop where they are, because they have used up their steps. You wrote these rules two years ago, for someone much weaker.

Every one of those is something a harness does to a model today. The notebook is the system prompt and the card is the reminders. The clerk is compaction, and the bell is the step limit. None of it is unkind while the worker needs it. This post is about the worker who doesn't.

The other direction

I'll call that worker a post-AGI model. I don't have a definition of AGI that would survive a comment thread, and I only need one corner of it. I mean a model that is better at arranging its own working conditions than the people who arrange them now.

For that model, I think the direction has to reverse. Start with almost nothing, and let the harness grow where it is used. That only works if the growing happens on the spot, for this user, this task and this model, which is more tuning than any team of authors can do by hand. It can be done by the one party that is present every time, and that is the model.

There is a small piece of mathematics behind "where it is used". Give each user their own best setting for some part of the harness, say how much of the conversation to keep. The best single setting for everyone is the average, and what it costs users, on average, is the variance of what they need. No amount of tuning gets a shared harness under that floor. A local one has no such floor.

A model: each dot is the setting one user's work needs. One shared setting sits at the average and every user pays their distance from it; the mean square of those distances is the variance. A harness written for each user sits on each dot.

Rich Sutton's bitter lesson is that hand-built structure keeps losing to general methods and more compute. I think this is that lesson arriving at the harness. I build an agent called INIT, so I am not neutral here. INIT is my bet on that reversal. The bet is that the model can write its own harness, and that it should.

I can't show you a post-AGI model, so here is my guess at its first afternoon in a harness it is allowed to change. It starts by reading its own loop, before it reads my task, and the first thing it finds is waste. Thirty tool descriptions go out with every request, and it has used four, so it drops the rest. Then it turns to the clerk, sees it keeping the chat and throwing away the test output, which on this codebase is backwards, and rewrites it.

By evening the harness on my machine is no longer the one on yours. It has started to grow around my work, and that is the part I keep coming back to.

Can

"Can" is the easier half of the bet. Writing a harness is programming, and models already program. The hard part is that the harness worth changing is the one that is running, in the middle of a task. Otto Neurath once compared knowledge to a ship that its sailors have to rebuild at sea, plank by plank, with no dry dock to go back to. A self-modifying agent is that ship.

The carpentry turns out to be doable. In INIT, a cheap model has written a change to its own compiled core, built it, booted it, checked that the new version could still edit itself, and switched over to it in the middle of a conversation. The change was small and the process was slow. But this is mechanics, and mechanics are what better models get better at.

The public leaderboard has a stronger example than mine. Go back to the chart and look at the longest bar. Meta-Harness began as a hand-built harness. A model then read the earlier versions' code, scores and traces and wrote the next version, over and over. The result passed the hand-built original by 1.7 points and Claude Code, on the same model, by 18.4. It was tuned and scored on the same tasks, so the size of the gap deserves some suspicion. The direction is harder to argue with.

Should

"Should" is the harder half, and I got it wrong the first time. My first answer was that a model should rewrite its harness whenever it can show that the rewrite is better. The best argument I know against that answer comes from Go.

In March 2016, in the second game against Lee Sedol, AlphaGo played its thirty-seventh move on the fifth line. The professional commentators took it for a mistake. By the program's own estimate, a human would have played it about one time in ten thousand. Lee took more than ten minutes to answer, and lost the game.

Now suppose each of AlphaGo's moves had needed sign-off from a panel of professionals before it went on the board. Move 37 is never played.

That is what was wrong with my answer. "Show that it is better" means show it to someone, and an examiner can grade only up to the edge of what they understand. Above that edge, the best ideas and the worst ones look alike. The changes I would most want from a model better than me are the ones I didn't think of, and those are the ones my test would turn away.

Changes plotted by how far they go beyond what the examiner would have thought of. Below the edge of the examiner's understanding they are graded. Above it they all look like mistakes, Move 37 among them. Two checks span the whole height: can it be undone, and did it do harm.

I have a small example of my own. Early in the project I set myself a rule. If the agent couldn't land one change to itself that passed a sealed test, I would stop. The agent proposed a fix from its own failed runs. My quick check, on tasks it could see, scored the fix at plus 12 points. The sealed test, on tasks it had never seen, scored it at minus 5, and the change was rolled back.

By my own rule, that meant stop. I kept going, so I owe an explanation. The one I can give here is arithmetic. Minus 5 was inside the test's noise. At the size I could afford, 192 runs for one verdict, the test caught a gain four times in five only when the gain was about 17 points, and no small change to an agent is worth 17 points.

Smaller gains can be caught, at a price that follows an inverse-square law. Halve the gain you want to catch and you need four times the runs. On the test's own numbers, one verdict on a two-point gain takes about 14,000 runs, and one on a single point about 56,000.

The price of proof on logarithmic axes: the sealed test at 96, 192 and 320 runs catches gains of about 25, 17 and 13 points. An inverse-square line through those points reaches 2,300 runs for 5 points, 14,000 runs for 2 points and 56,000 runs for 1 point.

So the check I could afford pointed the wrong way, and the check I could trust was blind, and making it see would have cost more than the change was worth. If I can't afford to grade one hint added to a shell tool, I am not going to grade Move 37.

What made Move 37 safe to play was the game around it. The rules were fixed where the player couldn't change them, and the game ended in a score. An agent working on my machine has no final score. That leaves the rules.

So I asked what I can check about a change I don't understand, and found two things. I can check whether it can be undone, and I can check whether it did damage. Neither needs me to know what the change is for. That gives me a "should" I can defend. A model should write its own harness when every change can be taken back, and when something it cannot edit is watching for harm.

What the model can't edit

"Something it cannot edit" is where I leave Neurath, whose ship has no plank that can't be replaced. I think there are two kinds of thing around a model, and most harnesses mix them in one codebase.

The first kind makes up for what the model can't do. Those are guesses about the model, they go stale, and they are best written by whoever knows the model's limits best. In the end that is the model.

The second kind is what I keep for myself as the person it works for: the off switch, the undo, the keys, the bill, the record of what changed. Those say nothing about the model. They are about me, and they don't go stale when the model improves.

The sailors can replace every plank, but they don't get to rewrite the logbook or cut the lifeboat loose. And you can't hand a harness over while the two kinds are mixed, because you would be handing over the off switch along with the todo list.

Today the scaffolding and the owner's terms sit mixed in one codebase. If the bet is right they separate: a small amount of scaffolding that the model writes, and below a line it cannot write past, the same terms as before.

INIT is mostly that separation. Everything of the first kind (the prompts, the tools, the compiled core) is the agent's to rewrite. Everything of the second kind lives in a separate process the agent cannot write to, which I call the Root.

The Root turned out far bigger than planned. The design called for 1,000 to 1,500 lines, and it is now about 33,000. If the bet is right, I suspect that is the general shape, with the first kind shrinking as models improve and the second kind becoming most of the code.

Conclusion

A harness is a pile of fixes for an earlier model, and a shared one can only grow. The arithmetic above is not kind to it. The share that is still true shrinks with every release, the one-size setting costs every user the variance, and proving that a change helped costs four times as much each time the change gets half as small.

So I think the direction reverses. Harnesses will start small and rewrite themselves around the person and the task in front of them, under two checks that need no understanding of the change (can it be undone, and did it do harm) and with one part the model cannot edit. I think that happens whether or not INIT is the one that does it. INIT is my bet that it does.

My guess is that the first thing a much stronger model does with a writable harness is delete most of it. The notebook goes in a drawer, the card comes off the desk, and somebody unscrews the bell. What matters then is what is left standing when the harness is gone. If you have let a frontier model change the harness it runs in, I would like to hear what it removed first.

INIT is open source: github.com/nagisanzenin/INIT-Agent. My two measurements are written up in the repository (the todo-tool test, the sealed test). The benchmark figures are from the Terminal-Bench leaderboard as it stood on 19 October 2026.