Loop Engineering: The Karpathy Method That Made AI Workflows 5 Better

What Is Loop Engineering?

Loop engineering is the discipline of building an AI workflow that keeps working toward a defined goal on its own, instead of requiring a person to prompt every individual step.

A single prompt is one instruction: you ask, you get an answer, you decide what happens next. A loop is different. You define the goal once, and the system discovers what needs doing, plans the work, does it, checks the result against a real standard, and — if the result isn’t good enough — feeds that result back in and tries again.

Three components make a loop actually work:

• A verifier. This is what turns repetition into real progress. Without an objective check on the output, the system is just agreeing with its own work on repeat. A verifier can be a test that passes or fails, a metric that moves up or down, or a build that compiles or breaks.
• State. This is what lets the loop learn between cycles. A small record of what’s already been tried — and what failed — means each run resumes instead of starting from zero.
• A stop condition. Every working loop needs two exits: the goal is met, or a hard limit says “stop after N attempts and report back.” Without one, a loop runs until it succeeds, breaks, or exhausts its budget.

Do You Need Loop Engineering? A Four-Question Test

Loop engineering only pays off when all four of the following are true:

1. The task repeats at least weekly. Anything less frequent doesn’t recover its setup cost — a one-off job is usually better handled with a single well-written prompt.
2. Verification is automated. A test suite, type checker, linter, or build process needs to be able to fail the work without a person reviewing it. Without that, you’re back to reading every output manually — the exact job a loop is meant to remove.
3. Your token budget can absorb retries. Loops re-read context and retry work, which consumes tokens whether or not a given run produces anything useful. This is why loop engineering looks obvious on a large budget and reckless on a limited one.
4. The agent has real tools. It needs logs, a reproducible environment, and the ability to run its own code and observe what breaks. Without that, the loop is iterating blind.

Loop engineering is a genuine capability, but most people don’t need its most intensive form yet. On a limited token budget, a heavy loop will hit a rate limit or a cost ceiling before it delivers a productivity gain.

The Karpathy Loop: How Loop Engineering Became a Movement

In March 2026, Andrej Karpathy released a GitHub repository called AutoResearch — three files, about 630 lines of code. Within a month it had passed 66,000 stars, and Fortune gave the underlying technique a name: the Karpathy Loop.

The setup has three parts:

• train.py — the training script, and the only file the agent is allowed to modify.
• prepare.py — the evaluator that scores the model. The agent cannot touch this file; if it could, it would make the test easier instead of making the model better.
• program.md — instructions describing what the agent should explore and which constraints to respect.

The loop itself runs a simple cycle: read the code, propose a change, train for five minutes, check whether the result improved, commit if it did, roll back if it didn’t, and repeat. The human writes program.md once and never touches train.py — the agent handles execution while you’re away from the keyboard.

What happened when Karpathy ran it: he pointed the loop at a model he’d already hand-tuned over two decades of experience and let it run for two days. The agent ran 700 experiments and found 20 improvements he had missed — including a missing scalar multiplier in the attention mechanism that was making attention too diffuse across heads. It wasn’t a bug a fuzzer would catch; it was a subtle optimization a careful human could have found but hadn’t, because the agent doesn’t get tired after experiment number twelve the way a person does.

Shopify CEO Tobi Lütke tested the same approach overnight on an internal model. He woke up to a 19% quality improvement — in a model half the size of the original, because the agent optimized for the hardware instead of defaulting to “bigger is better.”

Karpathy’s core insight: if you have an objective metric, you shouldn’t be the one running the experiments yourself. You’re the bottleneck. Removing yourself from the loop is what lets it run.

The Five Building Blocks of a Working Loop

Every functioning loop — whether built in Claude Code, Codex, or a bash script — is assembled from five components:

1. Automation — the heartbeat that fires the loop on a schedule, an event, or a trigger. Without this, a script that runs once isn’t a loop.
2. A skill — stored project knowledge (conventions, build steps, past incidents) written once in a file and read by every run, so the agent isn’t re-deriving context from zero each cycle.
3. Sub-agents — separating the agent that writes the work from the agent that checks it. The model that produced the code tends to grade its own homework too generously; a second agent with different instructions catches what the first one talked itself into.
4. Connectors — the ability to act inside a real environment: reading an issue tracker, opening a pull request, updating a ticket. This is the difference between an agent that reports a fix and a loop that ships one.
5. A verifier — the gate that automatically rejects work that doesn’t meet the standard. This is the piece that makes the rest of the system meaningful; without it, the loop is just paying for an agent to agree with itself overnight.

Bilevel Autoresearch: Applying Loop Engineering to the Loop Itself

In March 2026, two researchers published a paper titled “Bilevel Autoresearch: Meta-Autoresearching Itself.” Their starting question: if autoresearch is itself a form of research, can autoresearch be applied to autoresearch?

Their approach adds a second loop on top of Karpathy’s original:

• Inner loop — functions exactly like the original Karpathy Loop: propose a change, train, evaluate, keep or discard.
• Outer loop — observes the inner loop’s work and traces, identifies where its search process is getting stuck, and writes new code that changes how the inner loop searches. That code is injected, and the inner loop runs again.

On Karpathy’s GPT pretraining benchmark, this two-level approach produced a 5x improvement over the standard single loop (-0.045 versus -0.009 val_bpb) — not 5%, five times. Both loops used the same underlying model, so the improvement came from the architecture of the system, not from a smarter model.

What the outer loop actually found was that the inner loop kept falling back into the same search patterns — the model’s priors about what to try next, even after those priors stopped working. The outer loop broke that pattern by forcing exploration in directions the model’s instincts avoided. As the paper puts it: if autoresearch can meta-autoresearch itself, it can in principle meta-autoresearch anything with a measurable objective.

How to Try Loop Engineering Right Now, No Tools Required

You don’t need Claude Code or Codex to experience the core mechanic. Paste a prompt like this into any LLM:

You will work in a loop until the task meets the bar.

TASK:
[describe exactly what you want produced]

SUCCESS CRITERIA (be strict):
- [criterion 1]
- [criterion 2]
- [criterion 3]

LOOP PROTOCOL, repeat every turn:
1. PLAN   - state the single next step.
2. DO     - produce or improve the work.
3. VERIFY - score the result 1-10 on each criterion.
            Be brutally honest. List exactly what is still weak.
4. DECIDE - if every criterion is 8+, print FINAL and stop.
            Otherwise print ITERATING and go again, fixing
            the weakest point first.

RULES:
- Never call it done until every criterion is 8 or higher.
- Each pass must fix the weakest score from the last VERIFY.
- Do not ask me questions. Make a sensible assumption
  and keep going.

Begin.

The model will draft, grade its own work against your criteria, identify the weakest point, revise, and repeat until every criterion clears the bar. That’s a working loop, built from a single paragraph.

It’s a limited version — you’re still the one triggering it, there’s no schedule, and no state persists once you close the tab. But it demonstrates the core mechanic. Turning this into a full autonomous loop just means adding automation, a state file, and a verifier.

The Limits of Loop Engineering

Loop engineering changes the shape of the work — it doesn’t remove the need for judgment. Two problems become more pronounced as a loop gets better, not less:

• Comprehension debt. The faster a loop ships code no one has personally written, the wider the gap grows between what exists in a codebase and what the team actually understands. That gap compounds like interest — the day someone has to debug a system nobody has read costs far more than the tokens the loop ever used.
• Cognitive surrender. When a loop runs itself, it’s tempting to stop forming an opinion and simply accept whatever it returns. Designing a loop with judgment is what makes it valuable; using it to avoid thinking is what makes it a liability. The same action produces opposite outcomes depending on which one is happening.

Two people can build an identical loop and get opposite results: one uses it to move faster on work they understand deeply, the other uses it to avoid understanding the work at all. The loop itself can’t tell the difference — that judgment still belongs to the person running it.

Frequently Asked Questions