Skip to content

Beyond Prompt Engineering: Welcome to the Age of Harness Engineering

Still tweaking your prompts trying to squeeze more out of your AI agent? It might be time to zoom out.

OpenAI, Anthropic, LangChain, and Martin Fowler have all been circling the same idea lately: the thing that determines how good an AI agent is isn't the model itself — it's everything wrapped around it.

Same model. Better harness. Dramatically better results. Here's what that means and why it matters.

The Evidence Is Hard to Ignore

LangChain ran an experiment where they kept the model completely unchanged and only improved the surrounding system — the "harness." The result? Their Terminal Bench 2.0 score jumped from 52.8% to 66.5%, pushing them from outside the top 30 all the way into the top 5.

Over at OpenAI, three engineers paired with a Codex Agent over five months and merged 1,500 pull requests — roughly one million lines of code — with zero hand-written code. About ten times the output of a traditional team.

Same horse. Different saddle, reins, and bit. Completely different race.

From Prompt Engineering to Harness Engineering

If you've been in the AI space for a few years, you've watched this story evolve in real time:

  • Prompt Engineering (2022–2024): The art of asking a model a well-formed question. Valuable, but limited in scope.
  • Context Engineering (2025): Focused on giving the model better inputs — richer, cleaner, more relevant context.
  • Harness Engineering (2026): Encompasses everything Context Engineering covers, and adds tool orchestration, middleware, evaluation systems, feedback loops, and safety guardrails.

Context Engineering manages what the model sees. Harness Engineering also manages what the model can do, how its outputs get verified, and what happens when things go wrong.

The leap isn't just incremental. It's a shift in how you think about building with AI.

What Is an Agent Harness?

An Agent Harness is all the non-model code that wraps a language model. The model provides the intelligence. The harness makes that intelligence usable in production.

Think of it like this: the model is the horse. The harness — reins, saddle, bit — is what lets a rider actually direct it. Without the harness, the horse just goes wherever it wants.

A well-designed harness has six components:

1. System Prompt — Defines the agent's identity, rules of engagement, and what "done" looks like. Think CLAUDE.md or AGENTS.md — configuration files that act as the agent's operating manual.

2. Tools — The external capabilities the agent can call: MCP servers, Bash commands, APIs. The richer and more reliable these are, the more the agent can actually do.

3. Middleware / Hooks — Deterministic logic injected before and after each tool call. For example: automatically running a linter before every commit, regardless of what the model decided to do.

4. Context Architecture — Active management of context quality. As conversations grow longer, context degrades. Techniques like lazy loading and sliding windows keep quality high across long sessions.

5. Persistent Memory — Lets the agent remember important information across sessions. Tools like MEMORY.md or git history serve as long-term recall.

6. Verification Loop — Checks whether the agent's output is actually correct. This includes automated tests, lint checks, and cross-validation by sub-agents. The agent doesn't just produce — it verifies.

None of these pieces are individually new. But designed as an integrated system, they compound. Missing any one of them will limit the whole thing.

How LangChain Built Their Harness

LangChain structured their harness as a composable middleware pipeline with four layers:

  • LocalContextMiddleware — Scans the directory structure and environment variables at startup so the agent always knows where its tools live. Dramatically reduces environment-related errors.
  • LoopDetectionMiddleware — Detects when an agent is spinning its wheels doing the same thing repeatedly and forces a strategy switch.
  • ReasoningSandwichMiddleware — Allocates more reasoning compute at the planning and verification stages, and normal compute during execution. (They called it the "reasoning sandwich.")
  • PreCompletionChecklistMiddleware — Forces a verification checklist before the agent is allowed to declare the task done.

One finding from this experiment is counterintuitive but important: maxing out reasoning compute made performance worse. The highest reasoning setting ("xhigh") scored only 53.9% — because it kept timing out. The "high" setting, at 63.6%, was the sweet spot.

The lesson: how you allocate reasoning budget matters more than how much of it you have.

How OpenAI Built Their Harness

The team behind Codex Agent distilled their approach into four core principles:

1. If the agent can't see it, it doesn't exist. All knowledge must live inside the codebase. Their AGENTS.md is about 100 lines long — but it acts as an index pointing to design docs, architecture diagrams, and quality scores, all version-controlled.

2. Constraints must be encoded, not just documented. A rule in a wiki ("please don't make cross-layer calls") is optional. A structural test that enforces a dependency order — Types → Config → Repo → Service → Runtime → UI — and breaks CI if violated? That's unavoidable. Agents can't work around it.

3. Agents must be able to complete tasks end-to-end. From reproducing a bug to writing code to running tests to opening a PR, the agent handles everything. The human spends about one minute reviewing. That's it.

4. Minimize merge friction. If tests fail intermittently, auto-retry them. Don't let flaky tests block progress.

They also run background Codex tasks on a schedule — scanning for code drift, updating quality scores, and opening refactor PRs automatically. Like a garbage collector, but for technical debt.

AGENTS.md Is Not a Static Document

Here's one of the most powerful ideas in the Harness Engineering model: your configuration file isn't a one-time setup — it's a dynamic feedback loop.

The agent reads AGENTS.md at the start of a task. If it hits a failure, it writes the root cause and the solution back into AGENTS.md. The next agent that spins up reads the updated version, and won't make the same mistake.

Every failure makes the harness smarter. The system learns from its own errors.

You're Already Doing This (Just Not Systematically)

If you've used any AI coding tool, you've already touched Harness Engineering without calling it that:

  • Written a CLAUDE.md or .cursorrules to set project conventions? That's a System Prompt.
  • Connected an MCP server so your agent can access a database or external service? That's Tools.
  • Set up a hook to run lint before committing? That's Middleware.
  • Used a MEMORY.md to carry key decisions across sessions? That's Persistent Memory.

Harness Engineering isn't a brand-new technology. It's a shift in mindset: taking all these practices you're already doing and designing them as a coherent, intentional system.

Three Things You Can Do This Week

You don't need to redesign everything at once. Start here:

1. Write a real configuration file. Document your project structure, naming conventions, common commands, and things the agent must never do. If the agent can't read it, it doesn't exist.

2. Turn your acceptance criteria into executable commands. Don't write "code should be clean." Write npm run lint && npm test. Make "done" something a machine can verify.

3. Update your config every time the agent makes a mistake. Record what went wrong and how to avoid it. This is the minimum viable feedback loop — and it works.

These three changes alone — no new model required — will meaningfully improve what your agents can do.

The Changing Shape of Engineering Work

The day-to-day workflow is already shifting for engineers who've embraced this model.

Before: Read requirements → Write code → Write tests → Open PR → Wait for review → Merge.

After: Clarify the goal → Define acceptance criteria → Configure the agent's environment → Launch → Review the PR → Update the config if something failed → Repeat.

The bottleneck is no longer how fast you can write code. It's how clearly you can describe what success looks like, how well you can automate verification, and how thoughtfully you design the feedback loop.

The engineer's job is evolving from "I'll write it" to "I'll define what right looks like — and let the agent execute."

The models will keep improving. But right now, the biggest gains aren't coming from model upgrades. They're coming from engineers who understand that the harness is where the real work is.

Build a better harness.