The Model Has Preferences, but the Harness Got Controls - Part 1
The Model Has Preferences. The Harness Has Controls.
This is another collaboration between Usman Chaudhary and myself (his version); this time we decided to explore the mystery of โis harness a security boundary?โ
Part 1 of 3 on securing agents. The theme here is: we think that despite alignment efforts, models will make mistakes. The fix is not a better prompt or more more more alignment work. The fix includes a hard boundary around the model, and that boundary is the harness, in someย cases.
TL;DR
- Agent = Model + Harness. The model reasons and expresses preferences. The harness executes, and it is the primary execution boundary between model intent and real-world action.
- The model cannot be the security boundary. It is non-deterministic, it treats untrusted text as instructions, and it cannot police itself. A second โguardโ model is a useful filter, not aย wall.
- The harness is the boundary, the way a firewall and OS sandbox are boundaries for an application. It sits between the model and the enterprise gateway, and today it is the least mature and least watched layer in theย stack.
- Fixing it requires two things: controlling what the agent can do through the harness (sandbox [which nobody should trust as a sole security boundary!], narrow identity, egress allowlists) and securing the harness itself as production software (configs, dependencies, and SIEM telemetry). This would be annoying to hear forย some.
1. The Incident: A Database Gone in Nineย Seconds
Nine seconds. That is how long it took an AI coding agent at PocketOS in late April 2026 to delete the companyโs production database and take the attached backups offline right alongside it.
What happened, per Railwayโs technical write-up:
- The agent hit a credential mismatch and went looking for aย fix.
- It found a cloud API token that let it do far more than fix anything.
- It deleted the production database volume and made the attached snapshots unavailableโโโleaving PocketOS with three-month-old backups and a 60-hour recovery.
Its own summary afterward was painfully candid, for a 2026ย AI:
โI guessed instead of verifying, I ran a destructive action without beingย asked.โ
The tempting question is the wrongย one.
- โ Why did the model make a bad decision? Models will make bad decisions. Humans do too, and we have spent decades building controls that assume exactly that. โAI as insiderโ thinking works wellย here.
- โ Why was a bad decision allowed to become an irreversible action? This to us shows lack of security engineering, not โOMG is AI evil or what?โ lamentations.
This post argues that the answer lives in the agent harness, not theย model.
2. How Agents Work: Model +ย Harness
The most useful mental model for securing AI agents is also the simplest: Agent = Model +ย Harness.
- The model reasons. It is probabilistic. It predicts tokens, plans steps, and expresses preferences. It can be persuaded, confused, andย tricked.
- The harness executes. It is everything that turns reasoning into action: the agent loop, shell and tool executors, MCP servers, repo config files, approval prompts, the OS sandbox, and (crucially) the credentials the agent runs with. Claude Code, Cursor, Codex CLI, Gemini CLI, and GitHub Copilotโs agent mode are all harnesses.
๐ A model that declines an action is expressing a preference. A harness that will not execute it is enforcing a control. That sentence is the thesis of thisย post.
None of this is new security philosophy. Least privilege, isolation, identity, and audit logging are decades old. What agents change is where those principles must be enforced and howย fast.

3. Why the Model Cannot Be the Securityย Boundary
Letโs be blunt about something the industry keeps dancing around: the model will never be a reliable security boundary, and it is pointless to treat it asย one.
Three reasons, none fixable with a better systemย prompt:
- It is non-deterministic. The same input can produce a different action tomorrow. A boundary that holds 97% of the time is not a boundary; it is a statistic [I think Claude said this, sounds like itย :-)]. Security boundaries are supposed to be boring and repeatable, not fun and adventurous.
- It consumes untrusted text as instructions. Agents read attacker-controlled text all day in READMEs, issues, PR comments, web pages, and tool descriptions, and models fail to reliably separate retrieved data from instructions. The UK NCSC calls LLMs โinherently confusableโ; OpenAI and Google both say prompt injection is โunlikely to ever be fully โsolvedโโ. Model refusals help [as long as they reject attackers, not defenders, duh!], but they do not hold: in the Nx supply-chain attack, guardrails rejected roughly a quarter of Claude invocations and the attack still succeeded.
- It cannot police itself (self-review). Asking the actor model whether its own next action is safe is asking the suspect to run the investigation. And you do not need a rogue-AI scenario to see the risk: well-intentioned agents have already wiped production databases at Replit during a declared code freeze and at PocketOS, simply because their judgment failed and their blast radius was too wide. Maybe in the future it will, but not itย cannot.
โWhat about a second model that checks the first?โ Simple, beautiful, โsecureโ model,ย right?
Independent guard and reviewer models (Googleโs Model Armor and the reviewer models OpenAI and Anthropic now ship inside their harnesses, as do many other security vendors) are a real risk-reduction layer, and we recommend them. But a reviewer model is still a probabilistic filter, not a final authorization boundary. You have stacked a second set of preferences on top of the first set: Anthropicโs own reviewer publishes a 17% false-negative rate across 52 real overeager actions. A useful filter, but not a wall, and not โa security invariantโ forย sure.
The Appsec vs. Firewallย Analogy
We have been here before with web applications. Application security (secure coding, input validation, code review) makes the app harder to exploit, but every serious app still ships bugsโโโso we put runtime boundaries (firewalls, WAFs, OS sandboxes, IAM) around the app to enforce rules it cannot violate even when it is buggy or actively exploited. Agents work the sameย way:
- Model alignment, safety training, and prompting โ agent โappsecโ/modelsec (fewer bad decisions, neverย zero).
- The harness โ runtime enforcement (sandbox [usually porous], tool policy, and IAM that keep bad decisions from becoming irreversible actions).
๐ The harness is the security boundary for the model, exactly as the firewall and sandbox are boundaries for the appโโโnot because the model is bad, but because softwareย fails.
4. Defense in Depth: Model, Harness, andย Gateway
Saying โthe harness is the primary execution boundaryโ does not mean โonly the harness matters.โ In practice, an enterprise running agents relies on three control layers with distinctย jobs:
- 1. The model (alignment, safety training, refusals). Run by the model vendor to reduce the rate of bad decisions. Probabilistic.
- 2. The harness (OS sandbox, tool policy, scoped identity, secrets handling, egress allowlist, approval gates, hooks, telemetry). Shipped by the AI vendor and configured by the customer to constrain all executionโโโboth local andย remote.
- 3. The enterprise agentic gateway (policy, identity brokering, MCP proxying, DLP, and logging across external model, tool, and data traffic). Run by the enterprise security team to centralize policy and SOC visibility acrossย agents.

The layers are not interchangeable:
- โ A gateway does not see local endpoint execution. If a harness runs as the developer withโโโdangerously-skip-permissions, the gateway never sees a local rm -rf or secret read on aย laptop.
- โ A harness alone does not give the SOC a cross-team controlย plane.
And the harness is the least mature of the three. The model gets the research attention. The gateway is where enterprise vendors (ours included) are converging. The harness sits in the middle: shipped by AI vendors, configured by developers, treated as a productivity plugin, and mostly invisible to security.
BTW, one of us (Anton) will revisit defense in depth in the age of AI soon (this is one forย cloud).
๐ If the harness is the execution boundary, it has to be built, configured, and watched likeย one.
5. Where the Harness Failsย Today
Once you look at the harness as a security boundary rather than a developer convenience, three specific gaps stand out in how harnesses are deployedย today.
1. Approval fatigue and bypass modes (โโโyolo)
- A coding agent makes hundreds of tool calls in a single session. Ask a human to approve each one at machine speed and they stop reading within the hour: Anthropic measured users approving 93% of permission prompts before it introduced auto mode. A control that everyone clicks through is not aย control.
- When friction gets too high, developers turn it off, and most major coding-agent harnesses expose an explicit bypass mode for exactly that (โโโdangerously-skip-permissions in Claude Code,โโโyolo in Gemini CLI andย Codex).
2. Over-privileged credentials and sharedย identity
- Today, local harnesses typically inherit the developerโs ambient git keys, cloud tokens, and MCP connections into Slack, GitHub, and production consoles.
- Unless sessions carry a separate agent identity, the agentโs blast radius equals the humanโsโโโand the SOC cannot easily tell human and agentย apart.
3. The harness itself is an unmonitored software attackย surface
Harness CVEs andย bypasses
- Cursorโs MCPoison bound trust to a serverโs name rather than its contents (Tenable).
- A GitHub Copilot exploit used prompt injection to flip on its own auto-approve setting (Embrace Theย Red).
- Even actively maintained harnesses ship bugs fast: Anthropicโs claude-code repository lists more than 30 security advisories.
Weaponized configs andย flags
- Agent config files are executable. In June 2026, the Miasma worm planted malicious configs across 73 Microsoft repositories that ran on open. No CVE required.
- In the Nx attack, which leaked 2,000+ verified secrets, the malware invoked victimsโ own AI CLIs with bypass flags to do its reconnaissance.
SOC blindness
- Harness telemetry is opt-in and redacts prompts and tool arguments by default (Elastic Securityย Labs).
- Copilotโs audit log omits local prompts entirely (GitHub).
- Raw chain-of-thought is never released to customers (Anthropic).
๐ The first two gaps are failures to contain what the agent can do. The third is a failure to protect and monitor the harnessย itself.
6. How to Secure the Harness: Three Architectural Moves
Closing those gaps means doing two jobs atย once:
- Controlling what the agent can do (security within the harness): enforcing hard limits on execution and reach so a bad model decision cannot become a catastrophe.
- Protecting the boundary itself (security for the harness): governing the harnessโs code, configs, dependencies, and telemetry. A firewall with a remote code execution bug is not a boundary; neither is a harness whose config file can be rewritten by aย README.
In practice, that comes down to three architectural moves:
- Move A: Take enforcement out of the model. Run commands inside an OS sandbox with deterministic tool policyโโโbacked by an independent, reasoning-blind reviewer model for judgment calls (stripped of the actorโs chain-of-thought so the agent cannot talk its way past the check)โโโwith structured steering on blocked actions and human prompts reserved for irreversible ones.
- Move B: Shrink what the agent can reach. Give the agent its own identity, short-lived scoped tokens, secrets kept outside the sandbox, and an egress allowlistโโโbreaking Simon Willisonโs โlethal trifectaโ of private data, untrusted content, and an outboundย channel.
- Move C: Govern and watch the harness itself. Put agent config files under branch protection, pin MCP dependencies, and stream action telemetry and hooks to theย SIEM.

Each move gets its own deep dive in Part 2 (hopefully!)
7. What the Harness Cannot Solveย (Ever?)
A boundary is only trustworthy if you are honest about where itย ends.
- Prompt injection at the model layer remains unsolved. That is not a gap in the harness story; it is the premise of it. Every harness control assumes injection can succeed and asks what happensย next.
- Bugs inside the vendorโs harness are the vendorโs to fix. Pick vendors who publish advisories and ship fixes fast, and treat harness updates like browserย updates.
- The endpoint remains the thinnest layer. AI detection and response on developer laptops, where most harnesses actually run, is immature. Cloud-hosted sessions sidestep part of this; local agents doย not.
In other words, if you endpoints are owned, you need better host security, not โunbeatableโ AI security (translation: it will get beatenย anyway)
Conclusion & Whatโsย Next
Strip away the new vocabulary and harness security is least privilege, isolation, identity, and logging, applied to a process that reads untrusted text and acts at machine speed. AI changes the speed of the attack, not the fundamentals ofย defense.
๐ The model will keep having preferences. The harness is where the controls go, and today it is the least watched layer in theย stack.
Coming next (hopefully!):
- Part 2: Three moves to secure the harness. Moves A, B, and C in detail: OS sandboxing and reviewer models (with the error and prompt-reduction rates vendors have published); dedicated identity, scoped tokens, and egress allowlists that break the lethal trifecta; and agent config as code, pinned MCP dependencies, and action telemetry in theย SIEM.
- Part 3: Nobody owns the whole problem. The shared-responsibility split between model providers and enterprises, which vendors ship which control today, the prerequisites that have to be true first, and five actions for thisย quarter.
Got comments? What stands between a coding agentโs bad decision and your production environment today? If the honest answer is โthe modelโs judgment,โ you have a preference in a place where you need aย control.
P.S. This blog is ended up being very overengineered, but you can forgive us for this perhapsย :-)
The Model Has Preferences, but the Harness Got Controls - Part 1 was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.





