Harness Engineering: Why Agent Projects Fail and How Sovereign Is Built to Survive

Most agent failures are harness failures, not model failures. The harness is everything around the model: what it may touch, what it remembers, how it proves it is done, who approves what, and what gets recorded. Prospectr Sovereign is built on the premise that the harness is the moat and the model is a replaceable part, so every Sovereign deployment ships with action boundaries, context discipline, deterministic verification, approval gates, and traces in the customer’s own cloud.

This article adapts a July 2026 talk from Microsoft’s internal MLADS community, “A Hitchhiker’s Guide to Harness Engineering,” into the way we actually run agents for Prospectr and for our Sovereign customers. The framing, the examples, and the implementation notes are ours. The research lineage and the statistics belong to the people cited below.

Every failed agent shares three failure modes

Ask anyone who has run an agent in production whether it has ever “finished” a task it did not finish. The answer is always yes, and the failure almost always takes one of three shapes.

Wrong file. The agent modified a path with a similar name instead of the one requested. Nothing in the environment stopped it, because nothing in the environment declared what it was allowed to touch.

Forgotten context. The constraint set in step one was gone by step four. Long tasks dilute instructions. Without a budget for what stays in view and what gets re-read, the agent is reasoning from a stale picture.

Premature done. The agent declared success with no verifiable outcome. “Done” was the model’s opinion of its own work. Nobody asked the system of record.

It is tempting to call these hallucinations and wait for a smarter model. That reading is wrong. In each case the model did what it was asked with the information it had. What was missing was structural: an action boundary, a context budget, and a verification loop. Those are harness components, and each one has an engineering fix that does not depend on the next model release.

The industry signal

In June 2025 Gartner published a forecast that more than 40 percent of agentic AI projects will be canceled by the end of 2027. The stated pressures were escalating costs, unclear business value, and inadequate risk controls.

Read the list of canceled-project symptoms against the three failure modes and the pattern is obvious:

• No action constraints, so the agent touches things it should not.
• Context that degrades across long sequences, so constraints are lost.
• No verification, so “done” is the model’s word.
• No observability, so failures are silent until a customer notices.
• No governance, so every model upgrade changes behavior in ways nobody planned.

The structural thesis of this article, and of how we build: every one of those failure modes has a fix in the harness, not in the model.

Agent = Model + Harness

The industry has moved through four framings in roughly eighteen months. First it was prompt engineering: write a better instruction. Then context engineering: curate what the model sees. Then harness engineering: build the environment and the feedback loop. Now the frontier is intent engineering: freeze the outcome and the acceptance check, and let the system discover the path.

The equation that survived is simple. Agent = Model + Harness. The model provides capability. The harness creates the environment, enforces the boundaries, and closes the loop between what was asked and what was proven.

This is not a vendor idea. It was named by practitioners responding to operational failures at scale.

Contributor	Contribution	When
Tobi Lütke (Shopify)	Originated “context engineering” publicly	Jun 2025
Andrej Karpathy	Popularized context engineering	Jun 2025
Mitchell Hashimoto	Named “harness engineering”	Early 2026
LangChain	Formalized Agent = Model + Harness	2026
Birgitta Boeckeler	Defined the guides-and-sensors taxonomy	Apr 2026

The five layers of a harness

Field practitioners converge on five layers. They operate together, not in sequence, and the useful audit question for each is not “is the model smart enough” but “is this layer explicit, testable, and owned by someone.”

1. Tool orchestration (action). Declares which tools exist, with what permissions and schemas, in what order, routed how. The agent acts only through capabilities the harness exposes.
2. Verification loops (truth). Replaces self-review with tests, builds, linters, scans, and end-to-end assertions. Done is an external signal.
3. Context and memory (state). Curates instructions, task history, retrieval, long-term memory, and token budgets so the right information survives the whole run.
4. Guardrails (control). Enforces risk tiers, cost ceilings, rate limits, sandbox boundaries, and human approval before anything irreversible.
5. Observability (signal). Captures traces, evidence, cost, latency, drift, and outcomes so a person can reconstruct and govern any run.

Boeckeler’s vocabulary makes the audit concrete. Guides constrain behavior before action: tool manifests, context injections, routing rules. Sensors observe after action: test runners, build checks, trace exporters, drift monitors. A layer with guides but no sensors is flying blind. A layer with sensors but no guides is logging chaos. For each of the five layers, can you name the guide and the sensor?

Self-review is not verification

Asking a model to check its own output is asking a student to grade their own exam. The replacement is deterministic signal.

Check	Self-review says	Harness says
Tests pass	Opinion	Exit code
Build succeeds	Assumed	Verified
Lint clean	Estimated	Measured
No secrets leaked	Hoped	Scanned
End-to-end behavior correct	Inferred	Asserted

Every PASS from self-review is a bet. Every PASS from the harness is a fact. This single distinction is the one we would ask any operator to adopt first, before buying anything.

Same model, different scaffold, different rank

The counterintuitive evidence is that the scaffold, not the model, often determines measured performance.

• LangChain documented a run on Terminal-Bench 2.0 that moved from outside the top 30 to the top 5 by adding a structured action space and verification layers around the same model. The weights did not change.
• ISO-Bench showed one scaffold scoring 46 percent and ranking first on one inference server, then 27 percent and last on another, while a second scaffold moved the opposite direction. Same evaluation family, inverted order.
• The April 2026 Agentic Harness Engineering paper (arXiv 2604.25850) held the model constant and reported a solve rate rising from 69.7 to 77.0 percent after iterating only on the harness, with 12 percent fewer tokens and a 75.6 percent transfer result on an adjacent task domain. That is laboratory evidence, not permission to let an agent rewrite a production control system, and we treat it as such.

The implication for anyone evaluating a model upgrade: benchmark comparisons are harness comparisons. Before you approve the next model, benchmark your harness.

Where does your business sit?

The talk lays out a five-level adoption ladder. Each step up is not “more agents.” It is a new set of harness capabilities that the previous level did not require. We have rewritten the ladder in operator terms, because that is who we deploy for.

Level	Name	Scale	Your role	What slows you down	What moves you up
0	Gated	0 agents	Access seeker	Legacy approval workflows and hosting constraints	A vetted toolchain, budget caps, a fast sandbox path
1	Assisted	About 1 agent	Pair operator	Human attention: inspecting every output	Spend limits, written policy rules, tracing on every run
2	Parallel	About 10 agents	Orchestrator	Reviewing several output streams at once	Self-verification, isolated work areas, automated review
3	Supervised autonomy	About 100 agents	Manager of managers	Trust and decision throughput	Durable context, loops and routines, automatic review, sandboxing
4	AI-native	1,000 or more	Intent steering	Finding domain automation opportunities, governing exceptions	Cost controls, model selection policy, human approval by risk tier

A quick self-assessment:

• If a person reads every output before it is used, you are at Level 1 regardless of how many tools you have connected.
• If agents run in parallel but you have no automated review, you are at Level 2 with Level 1 controls, and the gap is where incidents come from.
• If you cannot show the trace from the last ten runs of your most important workflow, you are not yet at Level 3, whatever the org chart says.

Most enterprises believe they are at Level 2 and operate at Level 1. The honest conversation about which level you are actually on is the first harness decision, and it tells you what to build next. If you are at Level 1, do not build for Level 3. Build the harness that moves you reliably to Level 2.

How Prospectr Sovereign implements the harness

Sovereign is a deployed agent running in the customer’s own cloud account. It is the same architecture Prospectr runs internally across live client accounts, and the four-layer design (orchestrator, skills, state, guardrails) maps directly onto the five harness layers above. Here is what each layer looks like in practice.

Harness layer	What Sovereign does	Guide	Sensor
Action boundaries	Each skill is a single-purpose silo with its own least-privilege IAM scope and input contract. Design-locked files, production credentials, and send paths are off-limits unless the brief grants them explicitly. Every sub-agent brief names the exact files and systems it may touch.	Skill manifest, locked-file list, brief scope	Scope diff in the report: files touched versus files allowed
Context budget	Sub-agents start with isolated context and a complete written brief, not a copy of the parent conversation. Measured on our own workload, isolated briefs ran at roughly one tenth the tokens of forked context at half the runtime with equal output quality. Domain memory loads on trigger, not wholesale.	Brief template, memory silo index	Token and runtime per task, logged
Deterministic verification	No result is relayed on the agent’s say-so. Site work is checked by a verifier against the live domain, never the build folder. Ledger writes are read back by ID. Drafts are fetched by ID. Deploys that report success but serve a 404 count as failures. A validation script gates every sensitive report before a human sees it.	Definition of done in every brief	Exit codes from the verifier, the 200-gate, and the report validator
Observability	Every run leaves a dated memory record, a commit, a job ID, and a rollback job. Daily drift monitors re-check live properties for banned terms, missing schema, and broken routes. Break-glass elevation to cloud infrastructure is logged to an audit file with the session name.	Report format, memory schema	Drift watch, audit log, git history
Governance and model policy	Customer-facing sends, deletes, journal entries, data pulls, and production changes stop at a human approval gate. Models are routed by risk tier: judgment work (client copy, accounting, strategy) runs on the strongest tier; mechanical work (deploys, config reads, git) runs on a cheaper tier with the right skills loaded. Model choice is never used as a safety control.	Approval policy, routing rules	Approval records, cost reports

Three practices deserve a closer look, because they are where the talk’s thesis becomes operating procedure.

The skill library is the guide layer

Prospectr’s skills are version-controlled procedures, one folder per skill, each with a trigger, a numbered procedure, a verification section that checks the system of record, a failure-modes table, and a named owner. They are the harness equivalent of Boeckeler’s guides. When a deploy went wrong in September 2026, the root cause was not the model: a lightweight context setting had hidden the deploy gate skill from the sub-agent. A stronger model with the same missing skill would have made the same mistake. The fix was to make the skill load mandatory, not to upgrade the model.

Deterministic rules, regression tests, and a clean test environment

Following an architecture review with an outside engineer in late September 2026, Sovereign’s roadmap adds three harness components that come straight from the “sensors” column. First, a deterministic rules engine in the harness, so that policy checks (banned terms, locked files, approval requirements, secret scanning) are code with exit codes rather than instructions the model is asked to remember. Second, regression tests written before any code moves, so that the harness can prove a refactor did not change behavior. Third, a dedicated test environment with zero customer data, so that agents can be exercised against licensed test instances of every third-party integration without touching production. The sequence is unit, then regression, then the next increment. No code moves without the test in place first.

A validation layer in front of the human

The owner of a business should not be the agent’s tester. Sovereign’s “validation layer” is a harness step that reads the brief, reads the agent’s report, and checks that what was built, what was output, and what was expected all line up before anything reaches the person who approves it. The gates are concrete: system-of-record evidence pasted, scope matched to the brief, client-facing quality confirmed (no banned terms, brand facts correct), per-skill criteria met, and every URL or commit reference verifiable. The maturity path is crawl, walk, run: build the validation agent, rely on it more as it earns trust, then let it close the loop on its own for low-risk tiers while high-risk tiers keep the human gate.

Powerful and disposable

The architecture principle that ties this together: the harness should be a workhorse, not a monument. Build it to do significant work. Build it to be replaced when a better model or a changed task makes part of it obsolete.

• Thin. The minimum viable harness for the task. Resist scaffolding everything.
• Disposable. Model-agnostic interfaces. Expect to swap the model without rewriting the harness.
• Observable. Every action traced and checkable. A silent failure is a harness defect.
• Seed, do not specify. Declare the constraint and the verification target. Let the agent find the path.

Before every model upgrade, run an assumption registry: list every harness component that depends on current model behavior and classify it REMOVE (the new model handles it), KEEP (still needed regardless, which is always true of verification and identity), or ADD (a new capability creates a new action risk). Done with the registry, an upgrade is a migration. Done without it, an upgrade is an incident.

Three decisions for this week

1. Run the assumption registry on whatever you have running now. Classify every component before you approve the next model.
2. Add one deterministic checker to your highest-volume agent workflow. Replace one self-review step with a test, a build, or a read-back from the system of record.
3. Place yourself on the ladder honestly. Build the harness that moves you one level, not three.

The model will improve again in ninety days. The harness you build this week will still be running.

One question to leave with

For your most important agent workflow right now: Can you name the action boundary? Can you name the verification gate? Can you show the trace from the last ten runs? Can you describe what changes when the model changes?

If any answer is no, that is where to start. Not with a new model. With the harness.

If you would rather start with a harness that already answers all four, that is what Sovereign Agent is. Read the security baseline for the hardening detail, the skills catalog for the guide layer, or talk to Travis about where your business sits on the ladder.

Frequently asked questions

What is harness engineering?

Harness engineering is the discipline of building the environment around an AI model: the tools it may call, the context it is given, the checks that decide whether its work is done, the policies that gate risky actions, and the traces that record what happened. The model supplies capability. The harness supplies boundaries, memory, verification, and governance. An agent is the model plus the harness.

Why do most agentic AI projects fail?

Gartner forecast in June 2025 that more than 40 percent of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. In practice the canceled projects share the same gaps: no limit on what the agent can touch, context that decays across long tasks, no external definition of done, no observability, and no policy for what happens when the model changes. Each gap is a missing harness component.

What are the three classic agent failure modes?

Wrong file: the agent edits something with a similar name instead of the thing requested. Forgotten context: a constraint set at step one is gone by step four. Premature done: the agent declares success without a verifiable outcome. None of these is a hallucination in the usual sense. They are what happens when there is no action boundary, no context budget, and no verification loop.

Is a better model the fix?

Usually not. Published results show the same model moving from outside the top 30 to the top 5 on a coding benchmark after only the scaffold around it changed, and another study showing rank order inverting when only the inference server changed. Before approving a model upgrade, it is worth asking whether the gain is intrinsic to the model or dependent on the harness. If it is harness dependent, the harness is where the investment belongs.

Where does my business sit on the adoption ladder?

Level 0 means AI access runs through ticket queues. Level 1 means one assistant whose every output a person inspects. Level 2 means a handful of agents working in parallel with an orchestrator reviewing streams. Level 3 means supervised autonomy with automatic review and sandboxing. Level 4 means intent steering over a fleet with risk-tiered human approval. Most organizations believe they are at Level 2 and operate at Level 1. The honest answer tells you which harness to build next.

How does Prospectr Sovereign implement the harness?

Every Sovereign deployment ships with explicit action boundaries per skill, isolated context for each sub-agent brief, deterministic verification against the system of record before any result is reported, approval gates for sends, deletes, money, and production changes, model routing by risk tier, and a versioned skill library that acts as the guide layer. The customer owns the cloud account, the traces, and the policy.

What is a deterministic verification gate?

A check with an exit code, not an opinion. Tests pass or fail. A deployed URL returns 200 with the expected title or it does not. A journal entry read back from the ledger matches the request or it does not. Sovereign treats a self-report from the agent as a claim and a deterministic gate as evidence. Only evidence clears the gate.

Can I run this on my own cloud?

Yes. That is the point of Sovereign. The harness, the skills, the state, and the audit trail live in your AWS, GCP, or Azure account. Prospectr deploys it, transfers the operating knowledge, and does not hold your data. See the four-layer architecture and the security baseline for the specifics.