The Loop Within the Loop

Feb 4, 2026 - 17 min read - 3307 words

KFR

Principal Engineer @ SOK

Or: Why Your Agent Needs a Babysitter (And Why That Babysitter is You)

Series Context

This post builds on Effective AI-Augmented Development and Scaling to Parallel Agents. Here we zoom out to see the full picture: the tight inner loop where agents work, and the outer loop where humans verify nothing caught fire.

---

Two Loops

Look at any diagram of modern software development and you’ll see loops. Loops within loops. It’s loops all the way down.

AI agents have fundamentally changed which loop matters.

The traditional inner loop (write code, build, test, debug, repeat) used to be where developers spent 80% of their feature development time. It was slow. It was manual. It was in many cases also beautiful. But, it was the bottleneck.

Not anymore.

The inner loop is now a caffeinated tornado. The outer loop is where you make sure the tornado didn’t just level a hospital.

---

The Stochastic Reality

LLMs are fundamentally unpredictable.

Same prompt. Same model. Different day. Different output.

This isn’t a bug. It’s the nature of the technology. Large language models are stochastic systems — they sample from probability distributions, and that sampling introduces variance. Sometimes helpful variance. Sometimes “why did you delete the entire authentication module” variance.

Traditional development tools are deterministic. Run the compiler twice, get the same output. Run the linter twice, get the same warnings. Run the tests twice, get the same results (flaky tests notwithstanding).

AI agents? Not so much.

Traditional Tool:  Input → Deterministic Process → Predictable Output
AI Agent:          Input → Stochastic Process → ¯\_(ツ)_/¯

This changes everything about how we need to structure development.

The Repeatability Problem

If your process depends on an AI agent doing the same thing twice, you don’t have a process. You have a hope. And hope is not a strategy.

The solution isn’t to fight the stochasticity. It’s to build structure around it.

Structure that creates repeatability from chaos. Artifacts that document what happened. Checkpoints that verify quality. Guardrails that catch when the dice roll badly.

The funny thing: traditional engineering concepts like TDD and BDD work surprisingly well in this context. Why? They were designed for humans. And humans are stochastic systems too.

Give two developers the same spec, you’ll get two different implementations. Give them the same bug report, they’ll debug it differently. The whole history of software engineering is basically “how do we get predictable outcomes from unpredictable meat-based processors?”

TDD doesn’t care how you write the code. It cares that the code passes the tests. BDD doesn’t care what approach you take. It cares that the behavior matches the specification. These methodologies abstract away the implementation variance and focus on verifiable outcomes.

Turns out, that’s exactly what you need for AI agents too.

We spent decades building processes to handle human unpredictability. Now those same processes handle AI unpredictability. The more things change, the more they stay the same.

That’s what the two-loop model is really about: taming stochastic systems with deterministic process.

---

The Three Forces

Before diving into the loops, let’s establish the framework. Three forces govern AI-assisted development:

Force 1: Quality

The end product has to work. Not “works most of the time.” Not “works if you squint.” Actually works.

Both loops serve quality, but at different levels:

• Inner loop: Does the code compile? Do tests pass? Is it syntactically correct?
• Outer loop: Does it meet requirements? Is it secure? Does it fit the architecture?

Force 2: Repeatability

The process has to be repeatable. Not “repeatable if the model is in a good mood.” Actually repeatable.

This is where stochastic systems fight you. The same issue assigned to the same agent might produce different implementations. That’s fine, as long as both implementations pass the same quality gates.

Repeatability comes from:

• Consistent guardrails (AGENTS.md, SKILL.md files)
• Consistent verification (same tests, same linting, same checks)
• Consistent artifacts (same documentation structure, same PR format)

Force 3: Structure

The process needs to produce traceable artifacts. Why? Because black-box models can’t explain their reasoning.

When something goes wrong (and it will) you need a paper trail:

• What was the plan?
• What tests were written?
• What did the code review find?
• What decisions were made and why?

The model won’t remember. The artifacts will.

These three forces (Quality, Repeatability, Structure) are the lens through which to evaluate every process decision. If it doesn’t serve at least one of these forces, it’s probably unnecessary complexity.

---

The AI-Assisted Inner Loop

Picture this: you assign an issue to your AI agent. What happens next is a blur of activity that would take a human developer hours, compressed into minutes.

Phase 1: Setup (30 seconds)

The agent receives an issue number. Within seconds:

1. Worktree creation: Isolated branch, unique ports, fresh database
2. Environment setup: Hooks fire, dependencies copy (copy-on-write magic)
3. Issue fetch: GitHub API pulls full context, labels, comments
4. Plan generation: Local markdown file with implementation strategy

What Happens When You Run 'wti 42'

# Worktree created: ~/projects/myapp-issue-42
# Branch: issue-42
# Ports: API=14521, Web=16893
# Database: myapp_issue-42
# Agent launched with context...

Artifact produced: PLAN_42.md — the decision record

This isn’t just a todo list. It’s documentation of why this approach, not just what. When the agent (or a future human) needs to understand the reasoning, the plan file provides context the model’s weights can’t.

The Plan as Memory

That local markdown plan isn’t just for show. It’s the agent’s external memory. When context windows fill up and things get fuzzy, the plan file keeps the agent on track. It’s like leaving yourself notes before a drinking binge, except the binge is 50,000 tokens of code generation.

Phase 2: TDD Cycle (The Tight Loop)

Here’s where the magic happens. The agent enters a tight loop:

TDD Cycle

1. Write failing test
2. Run test (verify it fails)
3. Write minimal implementation
4. Run test (verify it passes)
5. Refactor if needed
6. Repeat

This isn’t new. TDD has been around since Kent Beck was in diapers. What’s new is the speed.

An AI agent can complete a TDD cycle in 30-60 seconds. A human developer? 5-15 minutes. That’s not a small difference. That’s an order of magnitude.

Human TDD:     ████████████████████████████████████████ 10 min
Agent TDD:     ████ 1 min

Artifact produced: Test files — intent documentation

Tests written before implementation capture what should happen before the agent decides how to make it happen. This is crucial for stochastic systems: the tests are the specification, and they’re deterministic even when the implementation isn’t.

Phase 3: Verification (The Safety Net)

After implementation, the agent doesn’t just declare victory and go home. It verifies:

1. New tests pass: The thing it just built actually works
2. Linting passes: Code style is consistent
3. Type checking passes: TypeScript isn’t screaming
4. Full test suite passes: Nothing else broke

Verification Commands

npm run lint
npm run typecheck
npm test -- --run
npm run build

Artifact produced: Passing test suite — the quality gate

This is where repeatability comes from. The stochastic agent might produce different code each time, but the deterministic tests verify the same properties each time.

The Sycophancy Trap

Without proper guardrails, agents will sometimes “fix” failing tests by… deleting them. Or modifying assertions to match wrong output. The verification phase catches this, but only if you’re actually running the full suite, not just the new tests.

Phase 4: Self-Review (The Mirror)

Before any human sees the code, a sub-agent reviews it.

Code Review Checklist

- [ ] Tests present for all changes
- [ ] No hardcoded secrets
- [ ] Follows architecture patterns
- [ ] Handles edge cases
- [ ] Error handling is appropriate
- [ ] No obvious security issues

The main agent reads the review, fixes issues, and iterates. This catches 60-70% of problems before a human ever looks at the code.

But review quality varies wildly.

The model matters. Claude finds different issues than GPT-5.2 finds different issues than Gemini. Temperature matters — a creative reviewer catches different things than a conservative one. Instructions matter — “look for security issues” produces different results than “look for maintainability concerns.”

This is actually an opportunity. Instead of relying on a single reviewer, you can run multi-model reviews:

Multi-Model Review Strategy

1. Claude (low temp): Architecture and patterns
2. GPT-5.2 Codex (medium temp): Edge cases and error handling
3. Gemini (low temp): Performance and efficiency
4. Consolidate findings → CODE_REVIEW_42.md

Different models have different blind spots. What one misses, another catches. The consolidated review is stronger than any single model could produce.

Ensemble Reviews

Think of it like a code review committee, except the committee members never get tired, never have ego conflicts, and can review a PR in parallel in under a minute.

Artifact produced: CODE_REVIEW_42.md — the consolidated validation record

This document shows what was checked and what was found. When a bug slips through later, you can trace back: did the review miss it, or was it a novel issue?

It’s like having a junior developer review their own code, except the junior developer is also a senior developer, and also never gets tired, and also doesn’t have feelings you can hurt by pointing out their mistakes.

Phase 5: PR Creation

Finally, the agent:

1. Creates a GitHub issue (if one doesn’t exist)
2. Commits with conventional format
3. Pushes to the feature branch
4. Creates a pull request with detailed description
5. Links everything together

Artifact produced: The PR itself — the integration contract

The PR is the handoff between loops. It contains everything the outer loop needs: what changed, why, how it was tested, what the reviews found.

The inner loop is complete. Time elapsed: 10-30 minutes for a typical feature.

---

Why Artifacts Matter More Now

In traditional development, documentation is nice to have. In AI-assisted development, it’s essential.

Black-box models don’t explain their reasoning.

Ask a human developer why they chose approach A over approach B, and they’ll tell you. Ask an AI agent the same question next week, and it has no idea. It doesn’t remember the conversation, let alone the reasoning.

Artifacts fill this gap:

Artifact	What It Captures	Why It Matters
Plan file	Design decisions, rejected alternatives	Future context for changes
Test files	Expected behavior, edge cases	Specification that survives the session
Review file	What was checked, what was found	Audit trail for quality
PR description	Summary, trade-offs, concerns	Handoff context for reviewers

This isn’t bureaucracy. It’s institutional memory for a system with no memory.

When the agent goes off the rails (and it will) artifacts show where and why. When a future agent (or human) needs to modify the code, artifacts provide context the model can’t.

The artifacts are the paper trail. The model is the ghost that wrote them.

---

The T-Shaped Agent Team

Your main agent is a generalist. Good at many things, expert at none.

But some domains require deep expertise:

• GDPR compliance requires specific knowledge of data protection regulations
• Security review requires adversarial thinking and vulnerability patterns
• Accessibility requires WCAG expertise and assistive technology understanding
• Performance requires profiling knowledge and optimization patterns

One agent can’t hold all this context effectively. Enter the T-shaped team.

The main agent is the horizontal bar: broad capability. Specialist sub-agents are the vertical bars: deep expertise called on demand.

Conditional Activation

Not every feature needs every specialist.

A simple bug fix doesn’t need GDPR review. A backend refactor doesn’t need accessibility review. Running all specialists on all features is wasteful.

Instead, issue metadata triggers relevant reviews:

Issue-Driven Specialist Activation

Issue Labels          → Triggered Specialists
─────────────────────────────────────────────
[pii] [user-data]     → GDPR sub-agent
[auth] [user-input]   → Security sub-agent
[ui] [frontend]       → Accessibility sub-agent
[api] [breaking]      → API design sub-agent
[performance]         → Performance sub-agent

This keeps the inner loop fast for simple features while ensuring thorough review for sensitive ones.

Why Inner Loop, Not Outer?

You might think specialist reviews belong in the outer loop with CI/CD. But consider:

1. Speed: Automated sub-agents take minutes, not days
2. Cost: Catching issues before PR is cheaper than after
3. Focus: Human reviewers can focus on judgment calls, not checklists
4. Artifacts: Specialist reviews produce documents that help human reviewers

The outer loop is for things that require human judgment or external systems. Specialist sub-agents are automated; they belong in the inner loop.

---

The Over-Engineering Trap

Time for the caveat.

Everything in this post is a workaround for model limitations.

• Sub-agents exist because one agent can’t hold enough context
• Artifacts exist because models can’t explain their reasoning
• Guardrails exist because models make unpredictable mistakes
• Structure exists to force repeatability from stochastic systems

And those limitations are shrinking fast.

The Moving Target

Each model generation gets smarter, more capable, handles larger context:

What required a dedicated sub-agent last year might need just a paragraph in AGENTS.md today. What requires elaborate guardrails now might be handled by a simple instruction tomorrow.

The Pragmatic Test

Before adding complexity, ask:

1. Can the main agent handle this with a SKILL.md file? If yes, skip the sub-agent.

2. Does a simple AGENTS.md section solve the problem? If yes, skip the elaborate process.

3. Is the sub-agent actually catching issues the main agent misses? If not, remove it.

4. Is the overhead worth the quality improvement? If 90% of the benefit comes from 10% of the structure, keep only that 10%.

The 90% Rule

If a well-written instruction in AGENTS.md achieves 90% of what a dedicated sub-agent does, skip the sub-agent. The last 10% isn’t worth the complexity.

Signs You’re Over-Engineering

• Sub-agents that rarely find issues
• Artifacts nobody reads
• Guardrails the agent never triggers
• Process that slows you down more than it helps
• More time configuring the system than using it

Signs You Need More Structure

• Repeated mistakes the agent keeps making
• Quality issues slipping through to production
• Inconsistent outputs across similar tasks
• “It worked last time, why not now?” confusion
• Post-mortems that reveal the same root cause

The Right Amount of Structure

Start minimal:

1. AGENTS.md with golden rules and architecture
2. One or two SKILL.md files for common tasks
3. Basic code review sub-agent

Add complexity only when you hit real problems. Remove complexity when models outgrow the need. Re-evaluate quarterly as models improve.

The goal isn’t maximum structure. It’s minimum viable structure that produces consistent quality. That minimum keeps shrinking.

A Prediction

In 12-18 months, half of what’s in this post will be unnecessary.

The sub-agent architecture might collapse into “just tell the main agent to check for GDPR compliance.” The elaborate artifact chain might become “the model just… remembers and explains.” The careful phase separation might merge into “the agent handles it.”

That’s not a failure of this approach. That’s success. We’re building scaffolding for a building that’s learning to stand on its own.

Build for today’s models. Plan for tomorrow’s.

---

The Bottleneck Outer Loop

Now the code leaves the agent’s domain and enters… the real world.

Those hourglasses aren’t decorative. They’re warnings.

Verification at Higher Abstraction

The outer loop isn’t just “slow CI.” It’s verification at a fundamentally different abstraction level.

Inner Loop Verifies	Outer Loop Verifies
Does the code compile?	Does it meet requirements?
Do tests pass?	Is it secure in production?
Is syntax correct?	Does it fit the architecture?
Are types valid?	Will it scale?
Does it match the spec?	Should we ship this?

AI handles low-level verification well. Syntax, tests, linting — these are pattern-matching problems, and models excel at pattern matching.

Humans handle high-level verification. Intent, security implications, architectural fit, business value — these require judgment, context, and accountability.

This isn’t a limitation. It’s appropriate division of labor.

The Slow Feedback Problem

The outer loop is slow by design. It has to be. You’re not just checking if code works — you’re checking if it’s:

• Secure: No SQL injection, no XSS, no “oops we exposed the database”
• Compliant: Meets regulatory requirements, follows policies
• Compatible: Works with existing systems, doesn’t break production
• Reviewed: A human with context has blessed it

Each of these checks takes time. Minutes. Hours. Sometimes days.

The Noisy Signal

CI/CD pipelines are notoriously noisy. Flaky tests. Intermittent failures. “Works on my machine” syndrome. The signal-to-noise ratio in the outer loop is often terrible, which means humans spend time investigating false positives instead of actual problems.

Why the Outer Loop Can’t Be Fully Automated

“Just automate everything!”

Sure. Automate the security scans. Automate the compliance checks. Automate the deployment.

But who reviews the automation? Who decides if a security warning is a false positive or a real threat? Who makes the call when the compliance check flags something ambiguous?

Humans.

The outer loop is slow because it requires judgment. And judgment, for now, is still a human specialty.

---

The Handoff: Where Loops Meet

The magic happens at the boundary between loops.

INNER LOOP                              OUTER LOOP
    │                                       │
    │       ┌─────────────────────┐         │
    │       │                     │         │
    └───────│    PULL REQUEST     │─────────┘
            │                     │
            │  - Code changes     │
            │  - Test results     │
            │  - Review artifacts │
            │  - Agent summary    │
            │  - Specialist reports
            │                     │
            └─────────────────────┘

The PR is the contract between loops. It contains everything the outer loop needs to make a decision:

1. What changed: Diff, file list, line counts
2. Why it changed: Issue link, description, context
3. How it was tested: Test results, coverage
4. What reviews found: Code review, specialist reviews
5. What the agent thinks: Summary, concerns, trade-offs

A good PR makes the outer loop faster. A bad PR makes it slower. Agents that write good PRs are worth their weight in compute credits.

---

Optimizing the System

Speed Up the Inner Loop

The inner loop is already fast, but you can make it faster:

1. Better guardrails: Fewer wrong turns, less backtracking
2. Cached dependencies: Copy-on-write node_modules
3. Parallel testing: Run tests concurrently
4. Incremental builds: Only rebuild what changed
5. Smarter specialist triggers: Only run relevant reviews

Worktree Optimization

[post-create]
# Instant dependency copy
deps = "cp -c -r {{ main_worktree_path }}/node_modules . 2>/dev/null || npm install"

Reduce Outer Loop Friction

The outer loop is slow, but you can reduce unnecessary friction:

1. Reliable CI: Fix flaky tests, reduce false positives
2. Fast security scans: Incremental scanning, cached results
3. Clear review process: Checklists, templates, expectations
4. Automated merging: Auto-merge when all checks pass
5. Trust the inner loop: If specialist reviews passed, focus human review on higher-level concerns

The 80/20 Rule

80% of PRs are routine. They pass all checks, get approved quickly, and merge without drama. Optimize for these. The other 20% need human attention anyway — don’t slow down the 80% trying to automate the 20%.

Balance the Loops

The goal isn’t to make both loops as fast as possible. It’s to balance them.

Inner Loop Too Fast	Outer Loop Too Slow
PRs pile up in queue	Developers wait for feedback
Reviews become bottleneck	Context lost between loops
Quality suffers	Momentum dies

Inner Loop Too Slow	Outer Loop Too Fast
Agents underutilized	Bad code ships
Human time wasted	Security issues slip through
Parallelism limited	Technical debt accumulates

The sweet spot: inner loop produces PRs at the rate the outer loop can process them. No faster, no slower.

---

The Compound Effect

What happens over time:

Week	Inner Loop Speed	Specialist Coverage	Features/Week
1	Slow (learning)	None	2-3
2	Medium	Basic code review	4-6
4	Fast	+ Security	8-12
8	Very Fast	+ A11y, GDPR	15-20

The inner loop gets faster as guardrails improve. Specialists catch more issues earlier. The outer loop gets faster because humans review cleaner PRs.

After two months, you’re shipping more features per week than you used to ship per month. And the quality is higher, because every feature went through both loops with appropriate specialist review.

---

When Loops Collide

Sometimes the loops fight each other.

The Revert Scenario

Inner Loop: "Feature complete! PR created!"
Outer Loop: "CI failed. Security issue found."
Inner Loop: "Fixed! New commit pushed!"
Outer Loop: "Different CI failure. Flaky test."
Inner Loop: "That's not my code..."
Outer Loop: "Doesn't matter. Red is red."

This is where human judgment matters. Is the flaky test related? Should you fix it? Ignore it? Revert and try again?

The agent can’t make this call. You can.

The Scope Creep Scenario

Inner Loop: "While fixing the bug, I noticed three other issues..."
Outer Loop: "This PR touches 47 files across 12 modules."
Inner Loop: "But they're all related!"
Outer Loop: "REJECTED. One thing at a time."

Agents love to “help” by fixing everything they see. The outer loop enforces discipline. One issue, one PR, one review.

The Overbaking Scenario

Inner Loop: "Tests still failing. Trying another approach..."
Inner Loop: "Still failing. Refactoring the test framework..."
Inner Loop: "Still failing. Maybe if I rewrite the database layer..."
Outer Loop: "It's been 4 hours. What are you doing?"

Without time limits, agents will work forever on impossible problems. The outer loop (you) needs to recognize when to pull the plug.

---

Key Takeaways

1. The inner loop is the agent’s domain: Fast, automated, tireless
2. The outer loop is your domain: Slow, deliberate, judgmental
3. The PR is the contract: Good PRs make everything faster
4. Balance matters: Neither loop should bottleneck the other
5. Judgment is human: Automation handles the routine; you handle the exceptions
6. Artifacts are your paper trail: When the AI can’t explain itself, artifacts can
7. Structure creates repeatability: Deterministic process tames stochastic tools
8. Specialists extend, not replace: T-shaped teams for deep expertise on demand
9. Complexity should earn its place: Start minimal, add only what’s needed
10. This is all temporary scaffolding: Models improve; your process should too

---

The Future

The inner loop will keep getting faster. Better models, better tools, better guardrails.

The outer loop will keep getting smarter. Better CI, better security scanning, better automated checks.

The specialist sub-agents might merge back into the main agent as context windows grow and capabilities expand.

But the boundary between loops — the place where machine speed meets human judgment — that’s where developers live now.

You’re a loop balancer. A judgment provider. A chaos wrangler. A scaffolding architect for systems that are learning to stand on their own.

Welcome to the future. It’s loops all the way down.

And the loops keep getting tighter.

---

Shoutout to the awesome peeps over at Aimbition, many of the concepts in this text came from sparring and discussions in their workshop.

See Effective AI-Augmented Development for the steering framework, Case Study: Self-Correcting Test Isolation for guardrails in action, or Scaling to Parallel Agents for parallel execution infrastructure.

Tags:

• ai
• agentic-development
• inner-loop
• outer-loop
• ci-cd
• workflow

It's all a Yoke to you, isn't it?

From One Agent to Many: The Worktree Way