Stop Engineering Context. Engineer the Loop.

The case for designing the outer loop — not just the prompt, not just the context — and a hands-on look at how we built one to tune our own retrieval models.

TL;DR

• 2023 was prompt engineering. 2024 was context engineering. 2025 is loop engineering — and most teams haven’t noticed yet.

• A loop is the smallest unit of an agent that actually learns. Prompts produce outputs. Contexts produce answers. Loops produce improvement.

• We built one to do our own ML research: an outer controller that proposes one experiment per iteration, runs it in an isolated git worktree, scores it on a fixed objective, and only keeps the commit if it wins.

• The result was less “AI does research for you” and more a disciplined falsification engine — and that turns out to be the actual unlock.

• Five patterns I’d steal: bounded scorer, one-hypothesis-per-iteration, allowlist editing, keep-or-discard gates, and structured memory writeback. Details below.

The shift nobody’s naming yet

You’ve seen this. Your coding agent runs overnight. 47 commits. Three break the build. You can’t tell which. You roll back to yesterday’s main and try again tomorrow. The agent didn’t fail at thinking — it failed at the loop around the thinking.

Everyone’s talking about context engineering. Fair — it’s a real thing. Stuff the right tokens into the window, pick the right retrieval, get the model to behave. Important work.

But context engineering has a quiet ceiling: it optimizes a single turn. You hand the model a beautifully assembled context, it produces an output, and then… what? You hope it’s good? You eyeball it? You retry?

That’s not engineering. That’s vibes with extra steps.

The teams I see actually shipping reliable agents — Cursor, Claude Code, Devin, the serious research-agent crowd — aren’t winning because they have better prompts. They’re winning because they’ve engineered the loop around the prompt. The plan step. The verification step. The gate. The memory. The thing that decides whether to commit, revert, or retry.

That’s loop engineering. And once you see it, you can’t unsee it.

Your agent’s while-loop isn’t the loop that matters

Most people, when they say “agent loop,” picture this:

while not done:
    thought = llm.think()
    action = llm.act()
    result = run(action)

That’s a tool-calling loop. It’s the inside of an agent. Useful, but it’s not where the leverage is.

The leverage is in the outer loop — the thing that wraps a whole attempt and decides what to do with it:

while budget_left and not_converged:
    1. plan an experiment from history + memory
    2. isolate an environment (worktree, sandbox, branch)
    3. let the agent make ONE coherent change
    4. run a bounded scorer
    5. compare against a gate
    6. keep / discard / promote
    7. write structured findings back to memory

That outer loop is where reliability comes from. It’s where you get a system that can run for 8 hours unsupervised and not corrupt itself. It’s where you go from “AI assistant that helps you” to “AI process that compounds.”

Here’s the line worth memorizing: in agent systems, reliability fails before generation. It fails in the loop, not the prompt. By the time the model is producing tokens, the decision that mattered — what to attempt, in what environment, against what gate — has already been made. Or, more often, not made.

Karpathy gestured at this when he said the LLM is the kernel and everything around it is the OS. Loop engineering is writing that OS.

Why bigger prompts and more tool calls aren’t the answer

When teams hit reliability walls, the instinct is to push harder on what already worked. Bigger context windows. Longer ReAct chains. Multi-agent debate. Fancier system prompts. Another reranker. A second judge model to grade the first one.

Each of these helps at the edges. None of them fix the core problem, which is that a single well-executed turn is not the same thing as a system that improves over many turns. You can have a perfect inner step and still have an agent that drifts, repeats failed ideas, and silently corrupts its own state.

There’s also a cost story here, and it cuts the opposite direction from intuition. More context, more tool calls, more debate — these all increase reasoning tokens per attempt without changing whether the attempt was a good idea in the first place. You end up paying for longer, more expensive failures.

The bottleneck isn’t the model’s reasoning per turn. It’s the absence of a structure that decides what to keep.

We built one to tune our own models. Here’s what mattered.

For the last few months, a small part of our research stack has been an autonomous loop that proposes and validates its own retrieval experiments. I want to walk through the architecture — not the embedding model itself, which is its own story for another post — but the loop pattern, because I think it generalizes.

The setup: we have a training pipeline, an evaluation pipeline, and a fixed scoring objective. Pretty standard ML setup. What we wanted was a controller that could, overnight, propose a change, try it, and keep it only if it actually improved retrieval quality across three datasets without overfitting any one of them.

Here’s the loop, in plain English.

1. Bounded scorer first, autonomous loop second

Before we let any agent loose, we built a bounded mode: read a spec, run training, run eval, emit one experiment_summary.json. Deterministic. Boring. The kind of thing you can call from a shell script.

This is the most underrated step in loop engineering: the autonomous loop is just a thin wrapper around a bounded thing. If your inner step isn’t reproducible and scoreable, no amount of agentic scaffolding will save you. You’ll get a system that confidently hallucinates progress.

Rule of thumb: if you can’t grade it, you can’t loop it.

There’s a cost story here too. Loops with strong scorers waste less compute, not more — because the gate kills bad branches at iteration boundaries instead of letting the agent burn 40k tokens reasoning its way deeper into a dead end. Cheap scorer + aggressive discard beats expensive context + hopeful generation, every time.

2. One isolated environment per iteration

Each iteration gets its own git worktree and branch. The agent edits real code, but in a sandbox that can be thrown away. No “the agent broke main.” No “I lost track of which experiment changed what.” The worktree is the iteration’s identity.

This is the same insight Cursor and Claude Code converged on: agents need cheap, disposable physical state. Not just chat history — actual file systems they can mess up without consequence.

3. One hypothesis per iteration

The agent’s prompt is brutal about this:

Your job in each iteration is to propose and implement exactly one coherent experiment that could improve retrieval quality… Favor simple, falsifiable changes over sprawling complexity. Treat every edit as a research claim that must be justified before it is implemented.

This is the single biggest behavior change. Left to their own devices, coding agents will refactor your house. Tell them to fix a leaky faucet and they’ll repaint the kitchen on the way out.

One hypothesis per iteration turns the agent from a contractor into a scientist. Every iteration becomes a clean A/B against the current best. Every keep/discard decision is interpretable.

4. Allowlist, not denylist

The prompt also includes a hard allowlist of editable files. Not “don’t touch these dangerous things” — only touch these files. Everything else is read-only by convention and enforced by the harness.

This sounds restrictive. It’s actually freeing. It’s the difference between letting a junior dev have prod access and giving them a scoped branch. They can move fast precisely because the blast radius is bounded.

5. Keep-or-discard gates

After the agent finishes, the controller runs the bounded scorer and checks:

• Did it beat the current best macro score?

• Did it preserve cross-domain behavior (or just overfit one dataset)?

• Did it pass safety gates (no contamination, no editing forbidden files)?

If yes: commit it, update the campaign manifest, move on. If no: throw the commit away. Not “save it for later.” Not “let me look at it.” Discard. The agent doesn’t get partial credit for trying.

This is the part most people get wrong when they try to build self-improving systems. They keep too much. They preserve “interesting failures.” They let the loop drift. The bias has to be aggressively toward discarding — because the cost of keeping a bad commit compounds across iterations, but the cost of discarding a marginally-good one is just one wasted hour.

6. Structured memory writeback

Every iteration — kept or discarded — writes back to a shared memory layer: the experiment spec, what changed, the score, what the agent claimed, what actually happened. The next iteration’s agent reads from that memory before proposing its next change.

This is where loop engineering and memory engineering merge. The loop is the producer of memory, and memory is the input to the next loop. Without that, you get an agent that proposes the same failed idea three times in a week. With it, you get something closer to a research lab notebook that actually compounds.

The substrate matters less than the discipline. Any decent store with structured retrieval works — what matters is that every iteration writes structured findings (hypothesis, change, score delta, claim-vs-reality), not just chat logs.

7. Permission to propose curation, not just tuning

One subtle but important detail: the agent is explicitly told that data curation is a first-class experiment, not just hyperparameter tuning. Most ML loop systems get stuck in the “twiddle learning rate forever” trap because that’s all they’ve been authorized to do.

In practice, the largest lifts in our campaign came from curation hypotheses — what to train on, how to label, where the contamination boundaries are. The loop only found those because we widened the action space and gave it a rubric for evaluating data quality, not just model quality.

The expressivity of the action space is a hyperparameter of the loop itself. Most people set it too narrow.

This isn’t AGI. It’s better plumbing — and that’s the point.

This isn’t “the AI did the research.” A human still defines the objective, the scorer, the allowlist, the safety gates, the rubric, and the program prompt. The loop is a force multiplier on a well-specified problem, not a replacement for specifying it.

But here’s what surprised me: once the loop runs, you stop thinking like a researcher and start thinking like a PI. Your job becomes designing better gates, sharper rubrics, broader action spaces. You’re tuning the loop, not the model. And the loop is tuning the model.

That’s the inversion. And it’s why loop engineering is going to eat a lot of what we currently call ML research workflow. Not the science — the workflow.

Five patterns worth stealing

If you’re building anything that runs for more than a single turn, these five are the load-bearing patterns:

1. Bounded scorer first. If you can’t grade an attempt deterministically, you can’t loop it. Period. Build the scorer before you build the agent.

2. One isolated environment per attempt. Worktrees, sandboxes, fresh branches — whatever your domain’s equivalent is. Cheap, disposable physical state.

3. One hypothesis per iteration. The single most underrated prompt constraint in agentic systems. Turns contractors into scientists.

4. Aggressive keep-or-discard gates. Default to discarding. Make “kept” expensive. The compounding cost of bad commits is what kills long-running loops.

5. Structured memory writeback. Every iteration’s outcome — kept or discarded — feeds the next one’s plan. Without this, you don’t have a loop, you have a slot machine.

You’ll notice none of these are about the model. That’s the point. Loop engineering is mostly the work around the model. And it’s where the next two years of agent reliability is going to come from.

The boring future

Most of what’s coming in agents isn’t going to look like a sci-fi demo. It’s going to look like this — careful outer loops, bounded scorers, ruthless gates, structured memory. Less “watch the agent code a whole app” and more “watch the agent ship 200 disciplined micro-improvements overnight, of which 14 stuck.”

That’s not as fun to demo on Twitter. But it’s what works. And it’s where I think the actual durable value is going to be built.

Prompt engineering taught us to ask better. Context engineering taught us to ground better. Loop engineering teaches us to commit better.

Loops are where agents stop being party tricks and start being processes.

If you’re building agentic systems and want a memory substrate actually designed for loop writeback — versioned, structured, queryable, with graph-native retrieval instead of just a vector blob — we built one. Start free at papr.ai.

Subscribe if you want the next post, which goes deeper on gate design — particularly how to detect when a loop is silently overfitting to its own scorer (it always tries to).