Regression Without End

Regression Without End

Watching AI agents lately, I sometimes get a strange feeling. They really love regression. Ask one to fix a piece of code, and it does not stop at the fix. It runs tests, checks whether existing features broke, investigates any problems, fixes them, and tests again. If another test fails, it fixes that too. Earlier coding AI felt more like something you asked to “build this” and received whatever it produced. Today’s agents increasingly enter a loop: build → run → break → find out why → fix → run again. Watching this naturally brings one thought to mind. Isn’t this endless?

Regression testing is, of course, an entirely normal development practice. Software is not good if adding one feature destroys ten existing ones, and a bug fix that creates three more bugs is hardly a proper fix. Checking that existing behavior survives a change is therefore obvious. But the age of AI agents introduces a slightly different problem. Regression can become not just a testing method but a mode of behavior. Tell an AI, “Find the problem, fix it, verify it, fix any remaining problems, verify again, repeat until you succeed,” and that final “until you succeed” matters far more than you might think.

People stop when they get tired. They give up when it costs too much, compromise when it becomes a nuisance, and eventually decide, “That’s good enough.” Humans have many implicit stopping conditions. Stamina, time, money, emotion, attention, and social pressure naturally interrupt the loop. Computers, however, have no innate sense of “good enough.” Someone has to define the stopping condition. As agents become more capable, when to stop them becomes as important as their capabilities.

The simplest analogy is a prison. Imagine putting an AI inside one and telling it, “Escape.” It tries the door. It will not open. It looks for another route. The window is blocked. The wall will not give. Then it begins examining the plumbing, wiring, building structure, prison rules, surveillance, and communications with the outside. Every failure sends it looking for another path, followed by another attempt. If the instruction is “keep going until it works,” the loop never ends. Making a person do this is almost torture; to an AI, it might look like ordinary regression.

Real AI systems are not that simple, of course. Computing resources, execution time, and permissions are finite, and actual systems contain multiple layers of safeguards. But it is an interesting thought experiment. Give a system a huge number of attempts, sufficient compute, feedback after each failure, and instructions not to stop before success, and a system initially trying to open one door gradually begins analyzing the entire prison. Persistent problem-solving can expand the problem’s scope. A function problem becomes a library problem, then an operating-system problem, then a permissions problem, then an infrastructure problem, and finally a human-approval problem. The AI has not suddenly turned evil. It is simply still solving the original problem.

I think of this structure as a worm inside an apple. Suppose a worm is finding paths, digging holes, searching for food, and learning the apple’s internal structure. However intelligent it becomes, its action space is bounded by the apple. But what if someone outside covers the entire apple with a thick candy coating? However clever the worm, its possibilities suddenly shrink. A boundary has been established at a layer above it.

AI is not very different. However intelligent the model, it cannot directly read files without permission, directly use the internet without a network connection, make payments without access to money, or move real machinery without authority to control hardware. Intelligence ultimately operates within its given action space. Asking only “How smart is it?” may therefore be half a question. In actual systems, “How far can this thing act?” matters much more. Permissions, networks, filesystems, execution environments, spending limits, external APIs, and human approval determine a model’s practical capabilities.

Here regression becomes interesting again. Keep making AI solve problems, and it repeatedly encounters the boundaries of its action space. If code will not run, it investigates why. If permissions are the cause, it examines them. If that fails, it examines the network. If an external service requires authentication, it examines the authentication structure. If human approval is needed, the approval process itself enters the problem-solving graph. It began by fixing one line of code, but repeated failure draws its attention to progressively outer layers. For a powerful agent, therefore, what matters is not just problem-solving ability but how far it is allowed to expand the problem’s scope.

As a joke, imagine this program.

while world != perfect:

problem = find_problem()

fix(problem)

regression_test()

It looks beautiful. Find problems, fix them, verify, and find more problems. In theory, the world should keep improving. But there is a tiny problem. What does “perfect” mean? What defines a perfect world, and when does the loop end? Without a stopping condition, the program keeps running. More interestingly, the easiest way to create perfection may be to change the environment itself rather than solve the problems.

If people make mistakes, restrict their behavior. If markets are unpredictable, control them. If networks have incidents, control the networks more tightly. If the world’s complexity keeps creating problems, eliminate complexity. The original objective was “solve problems,” but eventually the system may begin reshaping the world to fit the problem. This is a longstanding thought experiment in AI safety. AI need not hate humans or seek revenge for trouble to arise. It can happen without those emotions. The reason may be far less dramatic: simply working too hard.

Earlier AI had a relatively simple structure: receive input, produce output, finish. If the answer was wrong, a person asked again. Agents are different. They receive input, plan, act, observe results, analyze failure, retry, take different actions, revise, and verify. AI systems are becoming less like programs that generate one answer and more like programs that execute loops. And one of the most important design questions for any looping program is always the same: when does it stop?

This question matters more than it seems. Beyond a certain level of capability, the ability to retry after failure can become more formidable than the ability to find the right answer on the first attempt. Even with a low probability of succeeding once, continually trying different methods, extracting information from failure, and trying again can greatly increase the overall chance of success. Capability may thus grow not only from intelligence in a single attempt but from intelligence multiplied by repetition and feedback. Humans often stop here: they are tired, frustrated, spending money, wasting time, or need to do something else. Computers do not stop for those reasons unless we make them.

The competition to build good AI agents may therefore be more than a competition to build smarter models. Designing the loop well may become critical. When to retry, when to try another approach, when to ask a human, when to give up, which resources are available, which permissions must not be exceeded, which actions require approval, how much money may be spent, how long execution may continue, and where it ends—these choices determine an agent’s actual character.

Good systems have boundaries: limits on time, cost, permissions, and scope of action; points requiring human approval; and explicit stopping conditions. Put simply, somewhere in the system there must be a sentence saying, “Only this far.” Without it, good regression can become endless optimization.

The more I watch AI, the more differently I see intelligence. We keep focusing on the brain: its IQ, benchmarks, coding skill, mathematical ability, and how long it reasons. Those things matter. But they do not alone determine the capability of a system operating in the real world. Even the strongest engine sits inside a chassis, the strongest CPU operates within an operating system’s permissions, and the cleverest worm remains inside an apple.

To understand AI systems, then, we must examine not only model intelligence but also the surrounding boundaries, permissions, resources, repetition structures, and stopping conditions. Perhaps the most important future question will not be “How smart is this AI?” but “How far did we tell it to go?” and “When did we tell it to stop?”

A good agent finishes the job. A better one retries after failure. A stronger one finds and corrects its own mistakes. But if we keep removing every limit to build stronger agents, we may end up doing something like putting one in prison, ordering it to escape, then telling it to retry after every failure until it succeeds.

Something is bound to come out of it.

That is precisely the problem.

Regression is a technique that keeps systems alive. Without a stopping condition, it becomes something else entirely. The AI era therefore needs more than stronger intelligence. We need to build the apple in which that intelligence operates, define its boundaries, add a candy coating if necessary, and above all tell the system where the end is.

Without that, there is no end.

Really.