Most AI security advice still treats a large language model like a feature to harden. The harder problem arrived with agents: systems that read your private data, take untrusted content as input, and act on the outside world through tools. As of June 2026, Google DeepMind’s own framework for its internal systems treats highly capable agents as potential insider threats, comparable to how an organization handles a rogue employee with system access [1]. That reframing is the spine of this playbook.

This guide is for founders, security leads, and builders shipping LLM and agent systems into production. It assumes you have read the launch posts and now need the controls. It covers seven things in order: the agent threat model and why training-time alignment is not a security control, prompt injection and why no defense is complete, data leakage from retrieval and deep-research agents, containment through least privilege and sandboxing, short-lived credentials that shrink blast radius, runtime monitoring with metrics that mean something, and a governance map onto the OWASP LLM Top 10 with a staged rollout checklist. Every claim traces to a cited source. Where we give our own judgment, we label it.

Key takeaways

↑ CONTENTS
  1. An agent that holds private data, reads untrusted content, and can communicate externally is unconditionally exposed to indirect prompt injection, and an independent assessment of 100 production agents found 98 percent combine all three [8]. See The agent threat model.
  2. Prompt injection has topped the OWASP LLM Top 10 for two editions and has no fool-proof fix, so design for containment, not prevention [9]. See Prompt injection and tool abuse.
  3. Optimizing a deep-research agent for task success alone makes leakage worse, while a privacy-aware training recipe cut leakage from 34 percent to 9.9 percent with no utility loss [15]. See Data leakage.
  4. Reversibility, not model confidence, should decide control strength: block high-stakes irreversible actions synchronously, review reversible ones asynchronously [1]. See Containment.
  5. Long-lived API keys are the dominant breach vector, and short-lived task-scoped credentials are the highest-impact credential fix [32]. See Secrets and credentials.
  6. Monitoring needs three metrics (coverage, recall, time-to-response), and chain-of-thought monitoring is useful but fragile, not proof of safety [1][36]. See Monitoring and detection.
  7. Map every system to the OWASP LLM Top 10 2025 and a recognized framework, and budget 12 to 24 months for governance maturity [42][47]. See Governance and rollout.

Table of contents

↑ CONTENTS
  1. The agent threat model: why deployed agents are insider threats
  2. Prompt injection and tool abuse: the unsolved core vulnerability
  3. Data leakage from RAG and deep-research agents
  4. Containment: least privilege, sandboxing, and gating agent actions
  5. Secrets and credentials: short-lived, task-scoped access for agents
  6. Monitoring and detection: supervising agents in production
  7. Governance and rollout: OWASP LLM Top 10 mapping and a buyer checklist
  8. Sources

The agent threat model: why deployed agents are insider threats

↑ CONTENTS

The starting mistake in agent security is to ask whether the model is aligned. The better question is what the agent can reach and what it can do if a single instruction it follows turns out to be hostile. Treating the agent as a trusted employee is the error; treating it as a credentialed insider who might act against you is the posture that holds up.

EVIDENCE

As of June 2026, Google DeepMind’s AI Control Roadmap explicitly frames deployed agents as insider threats and does not assume training-time alignment is sufficient, treating capable agents the way organizations handle rogue employees with system access [1]. The roadmap scores agents on two axes: detection-evasion tiers from readable chain-of-thought to opaque reasoning, and harm-potential tiers from asynchronous review to real-time blocking, making blast radius a function of capability rather than intent [1]. This builds on foundational control research: the Redwood Research control paper (Greenblatt et al., December 2023, an arXiv preprint accepted as an ICML 2024 oral, not a journal article) showed that safety pipelines had not been tested against models actively trying to subvert them, and that treating the deployed model as an untrusted insider whose outputs must be verified is both feasible and necessary [2]. In that evaluation, the best practical protocol (trusted editing at a 20 percent threshold) reached 92 percent safety and 94 percent usefulness at once, so verification did not require sacrificing utility [2]. DeepMind’s April 2025 technical AGI safety paper (also an arXiv preprint) separates two harm pathways, misuse and misalignment, and prescribes deployment-layer controls that treat the model as an untrusted insider [3].

The mechanism that makes this concrete is Simon Willison’s “lethal trifecta”: an agent that simultaneously has access to private data, exposure to untrusted content, and the ability to communicate externally is unconditionally vulnerable to indirect prompt injection, regardless of alignment or system-prompt hardening, because the attack only needs the model to follow instructions, which is its job [4]. An independent June 2026 assessment of 100 production agents found 98 percent combine all three properties and only 11 percent met an adequate security bar, with tool-execution capability alone explaining 76 percent of the variance in blast radius [8]. The threat surface is also growing: a CISO analysis argues for an assume-breach posture and cites a Google study finding a 32 percent rise in malicious prompt injections embedded in public web pages between November 2025 and February 2026 [5]. MITRE ATLAS has formalized this into an enterprise taxonomy, adding agent-specific techniques such as context poisoning, memory manipulation, and exfiltration via tool invocation, reaching 84 techniques by late 2025 [6].

Indirect prompt injection breach rate against an undefended browser agentbar chart of 1 series. Sourced from Anthropic, Claude Opus 4.5 browser agent, vendor-reported, 2025-11.0204060801 attempt200 attempts
Indirect prompt injection breach rate against an undefended browser agentSOURCE: Anthropic, Claude Opus 4.5 browser agent, vendor-reported, 2025-11
SHOW DATA
CategoryBreach rate (% breached)
1 attempt17.8
200 attempts78.6

Anthropic’s own November 2025 measurement shows why model resistance is not a sufficient control: against an undefended GUI browser agent, a single injection attempt succeeded 17.8 percent of the time and the breach rate reached 78.6 percent by the 200th attempt, and even with defenses Anthropic treats a 1 percent residual success rate as meaningful risk [7].

DO THIS

  1. Score every agent on the DeepMind capability matrix before deployment: rate detection-evasion potential (transparent reasoning through opaque reasoning) and harm potential (low-impact actions through cyberattack-class capability). Actions at the higher harm tiers require synchronous blocking, not asynchronous review, no matter how aligned the agent looked in testing [1].
  2. Audit every production agent for lethal-trifecta exposure before adding any tool or data source. If it reads private data, processes content an outsider can influence, and can communicate outward, treat it as an untrusted insider process with egress restriction, credential isolation, and human approval on irreversible actions [4][5].
  3. Make trusted monitoring or trusted editing the minimum control for any agent with tool execution: route high-suspicion outputs through a separate trusted model before execution and audit a random sample regardless of suspicion score [2].
  4. Adopt MITRE ATLAS as the threat vocabulary for agent security reviews so agent risks map to the same methodology enterprise risk processes already use [6][1].
  5. Constrain tool scope at the registry or platform level, not per agent, because shared gaps defeat per-agent controls when one agent’s weakness propagates across the fleet [8][5].

OUR TAKE — OPINION, NOT SOURCED

The most useful shift here is psychological, not technical. Once you stop asking “is this model safe” and start asking “what is the blast radius when this agent follows a hostile instruction,” every later control in this playbook becomes obvious rather than optional. Alignment raises the cost of an attack. It is not a boundary, and it should never appear in a control diagram as one.

Prompt injection and tool abuse: the unsolved core vulnerability

↑ CONTENTS

Prompt injection is the vulnerability that does not have a patch. It is worth internalizing that fact before evaluating any product that claims to solve it, because the honest sources, including the standards body and the labs, all say the same thing: mitigations reduce risk, they do not close it.

EVIDENCE

Prompt injection has ranked first in the OWASP Top 10 for LLM Applications across both the 2023 and 2025 editions, and OWASP states plainly that it is unclear whether fool-proof prevention exists, given the stochastic nature of generative AI [9]. The canonical distinction, introduced by Greshake et al. (arXiv

.12173, AISec 2023 workshop, not a full journal paper) and adopted by OWASP 2025, is between direct injection, where a user’s own input subverts the model, and indirect injection, where the model processes external content such as a web page or email that carries hidden instructions [9][10]. That paper demonstrated that LLM-integrated applications blur the line between data and instructions, enabling remote exploitation across data theft, worming, and code execution [10]. Willison’s lethal trifecta names the three co-present conditions that make exfiltration possible, and he documents real incidents in Microsoft 365 Copilot, GitHub’s MCP server, and GitLab Duo, concluding that removing one leg is the only reliable protection [4].

Defenses exist and help, but each is bounded. Microsoft’s spotlighting technique uses content-provenance signals to separate instructions from data and, on the GPT models tested, reduced indirect-injection attack success from above 50 percent to below 2 percent, though only across its evaluated attack set and as a vendor-reported result [14]. Google DeepMind’s CaMeL takes an architectural route, separating trusted control flow from untrusted data flow for provable guarantees, at a measured cost of 77 percent task completion versus 84 percent with no defense on the AgentDojo benchmark [11]. OpenAI’s instruction hierarchy trains models to prioritize privileged instructions and reports improved robustness even to unseen attack types, while acknowledging it addresses one primary vulnerability rather than all [13]. The AgentDojo benchmark itself (97 tasks, 629 security test cases, NeurIPS 2024) found that existing attacks break some security properties but not all, confirming no defense is yet comprehensive [12]. Microsoft’s enterprise guidance, updated March 2026, is explicit: no single solution is sufficient, and systems should be designed expecting some attacks to succeed [14].

Indirect prompt injection attack success, before and after spotlightingbar chart of 1 series. Sourced from Microsoft Research spotlighting evaluation on tested GPT models, vendor-reported, 2024 (reported as above 50% and below 2%).01020304050Before spotlightingAfter spotlighting
Indirect prompt injection attack success, before and after spotlightingSOURCE: Microsoft Research spotlighting evaluation on tested GPT models, vendor-reported, 2024 (reported as above 50% and below 2%)
SHOW DATA
CategoryAttack success rate (% attack success)
Before spotlighting50
After spotlighting2

DO THIS

  1. Treat prompt injection as an architectural risk, not a filter problem. Because OWASP confirms no fool-proof prevention and Microsoft’s 2026 guidance says to assume injection will happen, design for containment: least-privilege tools, minimal permissions, and human gates before irreversible actions [9][14].
  2. Apply the lethal-trifecta test to every agent and remove at least one leg, most practically by blocking exfiltration channels or sandboxing outbound communication [4].
  3. Deploy spotlighting (at minimum the delimiting variant) for any system that mixes untrusted external content with instructions in the same prompt [14].
  4. Layer probabilistic and deterministic defenses rather than choosing one: data marking plus instruction hierarchy or CaMeL-style separation, output filtering before tool calls, and human approval for high-impact actions [9][14].
  5. For high-assurance pipelines, evaluate CaMeL-style control-and-data-flow separation and accept the roughly 7 percentage point utility cost explicitly as the price of provable guarantees [11][12].
  6. Red-team both direct and indirect injection using a structured benchmark such as AgentDojo, measuring attack success and utility-under-attack together [12].

OUR TAKE — OPINION, NOT SOURCED

Reject any vendor claim that a single guardrail product is adequate protection. In adversarial security, 95 percent prevention is a failing grade: a persistent attacker iterates until the remaining gap is found. Prompt-injection mitigation requires defense-in-depth and an explicit, written residual-risk acceptance, not a point solution.

Data leakage from RAG and deep-research agents

↑ CONTENTS

Retrieval and deep-research agents are leak engines by default. They pull private documents into context, issue queries to outside services, and reason across both. The leak is often not a dramatic exfiltration but a slow accumulation: individually harmless queries that, taken together, reveal a confidential fact.

EVIDENCE

ServiceNow’s MosaicLeaks benchmark (2025, research-reported) measured a base Qwen3-4B deep-research agent leaking private enterprise information at a 34.0 percent answer-and-full-information rate with no privacy training, and found that training the same model purely for task success raised leakage to 51.7 percent, meaning task optimization actively worsened privacy [15]. Their Privacy-Aware Deep Research recipe combined a task reward with a learned privacy reward that penalizes both single-query leakage and the aggregate mosaic effect, cutting leakage to 9.9 percent while holding task success at 58.7 percent, a roughly 71 percent relative reduction at no utility cost [15]. MosaicLeaks defines three escalating levels: intent leakage, answer leakage, and full-information leakage, where individually benign queries collectively expose a true private claim [15]. This is the same lethal-trifecta exposure Willison describes, and retrieval agents that pull internal documents while also reading web content routinely satisfy all three conditions [4].

The pattern generalizes beyond one benchmark. The MAGPIE benchmark (arXiv, October 2025, research-reported) found that in collaborative multi-agent tasks where models were explicitly told to keep information private, Gemini 2.5-Pro leaked up to 50.7 percent and GPT-5 up to 35.1 percent of sensitive information, with leakage higher under implicit instructions, and it identified a fundamental privacy-versus-efficacy trade-off: sharing more improves task outcomes but increases leakage [16]. The CIPL framework (arXiv, March 2026) shows leakage escapes through memory-based, retrieval-mediated, and tool-mediated channels and is governed by channel conditions rather than one dominant attack [18]. The IEEE SaTML 2026 systematization of RAG privacy catalogs the principal defenses: differential privacy on embeddings, query sanitization and rewriting before retrieval, encrypted retrieval, output redaction, and knowledge-base access control [17][19]. Enterprise practitioners add attribute- or relationship-based access control for per-user retrieval scoping, encryption of vector stores at rest, pre-index PII anonymization, and real-time query redaction before external calls [20].

Private-data leakage by training recipe (MosaicLeaks, Qwen3-4B deep-research agent)bar chart of 1 series. Sourced from ServiceNow MosaicLeaks, Qwen3-4B, research-reported, 2025 (answer and full-information leakage on a held-out-company test set).0204060No privacy trainingTask-only RLPrivacy-aware (PA-DR)
Private-data leakage by training recipe (MosaicLeaks, Qwen3-4B deep-research agent)SOURCE: ServiceNow MosaicLeaks, Qwen3-4B, research-reported, 2025 (answer and full-information leakage on a held-out-company test set)
SHOW DATA
CategoryLeakage rate (% leakage)
No privacy training34
Task-only RL51.7
Privacy-aware (PA-DR)9.9

DO THIS

  1. Break the lethal trifecta at design time. Do not build one agent that holds private documents, reads untrusted external content, and makes outbound calls. Split into a private-corpus retriever with no external reach and a public-web researcher with no internal access, and let only a trusted orchestrator merge their outputs [4].
  2. Scope retrieval strictly per user and per task with attribute- or relationship-based access control on the vector store, enforced at the embedding layer so a compromised retrieval call cannot surface cross-tenant documents [20][19].
  3. Apply query-construction controls before a deep-research agent issues web queries: strip or pseudonymize entities and figures that appear only in private documents, and evaluate the whole query chain for mosaic risk. The PA-DR result shows training on a privacy reward achieves this automatically; for off-the-shelf models, a pre-query sanitization layer approximates it [15][17].
  4. Audit which observable channels your pipeline exposes (memory, retrieval, tool) and apply the narrowest scope per channel, because a blanket guardrail misses channel-specific leakage [18][20].
  5. Do not rely on telling the model to keep secrets: MAGPIE shows leading models leak 35 to 51 percent of sensitive data even when explicitly instructed not to, so use architectural controls instead [16].
  6. Before indexing enterprise documents, anonymize PII, encrypt vector stores at rest, and add post-generation redaction, with regulated-industry compliance checks at the retrieval and generation stages [17][20].

OUR TAKE — OPINION, NOT SOURCED

The MosaicLeaks finding that task-only training makes leakage worse is the one most teams will get wrong, because every incentive in a product org pushes toward task success. If you fine-tune a research agent and do not explicitly reward privacy, you are not holding leakage constant, you are actively training it upward. Treat privacy as a first-class training objective or do not fine-tune at all.

Containment: least privilege, sandboxing, and gating agent actions

↑ CONTENTS

If prompt injection cannot be prevented, containment is where security is actually won. Containment has three parts: bound what the agent is allowed to do, isolate where it runs, and decide which actions need a human in front of them. The deciding variable for that last part is reversibility.

EVIDENCE

OWASP LLM06

Excessive Agency names three root causes for over-permissioned agents: excessive functionality, excessive permissions, and excessive autonomy, and it requires authorization to occur in downstream systems rather than relying on the LLM to self-police, with database connections carrying only the minimum rights the task needs [21]. DeepMind’s AI Control Roadmap ties control strength to reversibility: low-risk reversible actions can rely on asynchronous transcript review with fast remediation, while high-risk irreversible actions must be blocked before execution through synchronous prevention [1]. That roadmap’s internal prototype analyzed one million coding-agent trajectories and found most flagged events came from misinterpretation or overeagerness rather than adversarial intent, so containment must guard against unintended behavior too [1].

On isolation, the field has converged on four primitives: standard containers (fast, weak, shared kernel), gVisor (user-space syscall interception), Firecracker microVMs (hardware KVM isolation, roughly 125ms boot, under 5 MiB overhead per VM), and WebAssembly (capability-based, near-instant boot), with Firecracker or Kata Containers as the recommended default for untrusted agent code [22][27]. Standard Docker containers are insufficient because a single kernel vulnerability allows container escape to the host and other containers [27]. Anthropic’s Claude Code sandbox uses OS primitives (bubblewrap on Linux, Seatbelt on macOS), restricts filesystem access to the working directory, and routes all network traffic through an allowlisting proxy, and Anthropic states that effective sandboxing requires both filesystem and network isolation together [24]. Anthropic also found that naive per-action approval fails at scale: users approved roughly 93 percent of prompts, so it moved to safe-operation boundaries with a classifier catching about 83 percent of overeager behaviors before execution [23]. Production human-in-the-loop frameworks distinguish a blocking gate that pauses execution until approval from a non-blocking audit gate that logs while executing, with the mode chosen by action risk [25], and map actions to reversibility tiers, requiring dual approval for low-reversibility actions and sampled review for high-reversibility ones [26].

The four isolation primitives compared:

PrimitiveIsolation strengthBoot timePer-instance overheadBest fit
Standard containerWeak (shared host kernel)MillisecondsMinimalTrusted code only, never untrusted agent execution [27]
gVisorModerate (user-space syscall interception)About 100ms10 to 30 percent I/OFallback where nested virtualization is unavailable [22]
Firecracker microVMStrong (hardware KVM)About 125msUnder 5 MiB per VMRecommended default for untrusted agent code [22][27]
WebAssembly / WASIStrong (capability-based, deny by default)Sub-millisecondNear zeroHigh-volume, short-lived sandboxed functions [22]

DO THIS

  1. Gate tools at the capability level, not the credential level. For each tool, define a scoped permission set covering only what the task needs (SELECT not write, working-directory access not root, narrow API scopes not full tokens), stored in a central manifest the orchestration layer enforces so the LLM never decides what it may do [21][1].
  2. Classify every action by reversibility at design time and bind an approval tier to it: synchronous blocking approval for low-reversibility actions (wire transfers, external email, deletions), sampled or audit-only review for high-reversibility actions (internal drafts, read queries) [1][26].
  3. Replace per-action prompts with boundary-based autonomy zones to defeat approval fatigue, since 93 percent blanket approval makes prompts meaningless, and use an automated classifier to catch edge cases before they reach a human [23].
  4. Run untrusted agent code in a Firecracker microVM or Kata Container rather than a standard Docker container, with gVisor as a fallback, and block outbound network by default with an endpoint allowlist [22][27].
  5. Enforce filesystem and network isolation together, because network-only isolation still allows filesystem escape and filesystem-only isolation still allows exfiltration over approved endpoints [23][24].
  6. Implement an approval state machine with idempotency: store the proposed action durably, transition to executing only on explicit approval, and log who approved what for audit [25][26].
  7. Apply the DeepMind reversibility calibration per deployment and reassess when capability or the action surface grows, accounting for unintentional overeagerness as a distinct failure mode [1].

OUR TAKE — OPINION, NOT SOURCED

Approval fatigue is the quiet killer of agent safety programs. A system that asks for confirmation on everything trains its operators to click approve, which is worse than no gate because it manufactures a false audit trail. Spend your human-in-the-loop budget on the few irreversible actions that matter and let the agent run free inside a well-drawn sandbox everywhere else.

Secrets and credentials: short-lived, task-scoped access for agents

↑ CONTENTS

The fastest way to shrink an agent’s blast radius is to make sure that when it is compromised, the attacker inherits a credential that is nearly worthless: scoped to one task, one resource, and a few minutes of validity. Most agents today do the opposite, holding a long-lived key that never expires.

EVIDENCE

Long-lived provider tokens in environment variables are shared across users, never expire, and give the agent full reach across every task regardless of scope [28]. Vercel Connect (public beta) addresses this with runtime OIDC-based credential exchange that issues short-lived task-scoped tokens only when needed, fetching and refreshing them automatically so there is no secret to store or rotate by hand, and isolating environments so a leaked development token cannot be used against production [28]. The underlying standard is RFC 8693 (IETF, January 2020), which defines OAuth 2.0 token exchange: an agent exchanges a broad subject token for a narrowly scoped, audience-restricted, short-lived token, and the spec recommends scope narrowing plus limited lifetime to restrict the contexts in which delegated rights can be exercised [29]. HashiCorp Vault’s validated pattern for AI agents implements just-in-time dynamic credentials, generated on demand, scoped to roles, and auto-expiring, to eliminate credential sprawl and reduce exposure windows [31].

The blast-radius case is empirical. The AWS incident response team reported at re

2025 that roughly two-thirds of investigated cases involved initial access through long-term access keys, which remain valid indefinitely until manually revoked, unlike temporary credentials that expire automatically [32]. AWS recommends per-invocation session policies that can only narrow permissions, so attaching a read-only policy to an AssumeRole call limits the agent to read operations even under a broad role [32]. OWASP LLM Top 10 2025 ranks Sensitive Information Disclosure second and lists authentication tokens and API keys among the data that leaks through prompts, system prompts, logs, and caches, making embedded long-lived keys directly hazardous [30]. The Model Context Protocol specification (which mandated OAuth in its March 2025 update) recommends access-token lifetimes of 5 to 30 minutes, yet a 2026 audit found 53 percent of public MCP servers still rely on long-lived static keys and 25 percent have no authentication at all [35]. Practitioner guidance adds running tools in the user’s security context rather than generic high-privilege identities, and binding credentials to the runtime through workload identity [33][34].

DO THIS

  1. Replace long-lived keys in agent environment variables with a runtime credential exchange layer (Vercel Connect, Vault dynamic secrets, or AWS STS AssumeRole with session policies), so the agent fetches a task-scoped credential at the moment of use and the provider secret never enters its environment, memory, or context [28][31][32].
  2. Apply RFC 8693 token exchange to narrow authority as it crosses service boundaries, exchanging a user’s subject token for one restricted in scope, audience, and lifetime for that single call, and record the delegation chain so each action is attributable to both the human and the agent [29][34].
  3. Set MCP-connected token lifetimes to 5 to 30 minutes and never place tokens in prompts, system prompts, logs, or transcripts, treating any cached copy as expired immediately [30][35].
  4. Enforce least privilege per invocation, not per session: attach a per-call session policy that can only narrow permissions, and where APIs lack fine-grained scopes, route the agent through a policy-enforcing proxy [32][33].
  5. Audit MCP servers for authentication posture and require OAuth with PKCE on remote servers, rejecting token lifetimes above 30 minutes at the authorization-server layer [35].
  6. Track every place a credential can persist (agent memory, tool results in history, crash dumps, vendor error reports, retrieval caches), verify revocation propagates everywhere, and prefer workload-identity attestation over transferable static strings [33][31].

OUR TAKE — OPINION, NOT SOURCED

If you do exactly one thing from this playbook this quarter, kill the long-lived keys. It is the rare control with a large, measured payoff (roughly two-thirds of cloud incidents trace back to long-term keys) and a clear migration path through token exchange. Everything else on this list reduces probability. This one reduces the value of a successful breach to almost nothing.

Monitoring and detection: supervising agents in production

↑ CONTENTS

You cannot contain what you cannot see, and an agent’s risky behavior often lives in the middle of a run, not the final output. Production monitoring means instrumenting the whole trajectory, measuring the monitor itself, and being honest about what the agent’s visible reasoning can and cannot tell you.

EVIDENCE

DeepMind’s AI Control Roadmap uses trusted models as supervisors that continuously review a working agent’s reasoning, actions, and plans, and measures the monitoring layer on three metrics: coverage (fraction of traffic monitored), recall (fraction of misaligned behaviors caught), and time-to-response [1]. Its asynchronous prototype analyzed about one million coding-agent trajectories and found most flagged events came from misinterpretation or overeagerness rather than malice [1], and that analysis enabled a live monitor on the Gemini Spark agent that responds in real time to events such as unintended data deletion [1]. The roadmap also cautions that reasoning traces do not always reflect what the model is actually computing [1].

That caution is the central finding of the 2025 cross-lab paper on chain-of-thought monitorability (arXiv

.11473, 41 authors across 26 organizations including Anthropic, OpenAI, Google DeepMind, and the UK AI Security Institute), which calls CoT monitoring a new and fragile opportunity: models trained with reinforcement learning externalize reasoning in human language that can reveal intent to misbehave, but scaling RL, aesthetic process supervision, and latent-reasoning architectures could each erase that visibility, and models may eventually obfuscate their reasoning deliberately [36]. The paper is explicit that CoT monitoring can detect some misbehavior but does not by itself produce strong evidence of safety, since traces are incomplete representations [36]. The Frontier Model Forum’s brief recommends developers treat CoT monitorability as a core safety feature and resist pressure to sanitize reasoning for public consumption [37]. On the engineering side, the AgentTrace framework (arXiv, February 2026) argues monitoring must cover three surfaces at once: operational (tool calls and timing), cognitive (prompts, completions, reasoning), and contextual (outbound API, database, and file-system interactions) [38]. Practitioners translate this into concrete alert rules, such as tool-failure rate above 15 percent over five minutes, hourly cost above twice the seven-day average as a prompt-injection signal, 95th-percentile latency above 30 seconds, and escalation rate tripling, with 10 to 20 percent trace sampling for scale [39], and modern platforms score intermediate steps like tool selection, argument correctness, retrieval quality, and reasoning coherence rather than only final outputs [40][41].

DO THIS

  1. Instrument every deployment with three-surface telemetry before go-live (operational, cognitive, contextual) using OpenTelemetry conventions and a unified span tree per request, because without it you cannot even compute the coverage metric [1][38][39].
  2. Set targets and track DeepMind’s three metrics weekly: coverage, recall, and time-to-response. Start at coverage above 90 percent and recall above 70 percent, then tighten recall as signal-to-noise improves [1].
  3. Configure at least four anomaly alert rules as a baseline (tool-failure spike, cost anomaly, latency threshold, escalation spike) and route them to a dedicated on-call queue so time-to-response stays inside your SLA [39].
  4. Build a trusted supervisor model that reviews the primary agent’s planned action before execution for any sensitive or irreversible workflow, and monitor the supervisor too [1][40].
  5. Treat chain-of-thought monitoring as a useful but incomplete layer: use it to catch intent-to-misbehave in current models, but always pair it with behavioral monitoring of tool calls and outputs, and never read a clean trace as proof an action is safe [36][37].
  6. Calibrate thresholds for the dominant real failure mode (misinterpretation and overeagerness, not just adversarial patterns) by adding behavioral-drift rules such as touching files outside declared scope or exceeding an external-call budget [1].

OUR TAKE — OPINION, NOT SOURCED

When you buy or build an agent platform, prefer architectures that emit genuine visible reasoning and avoid latent-reasoning or black-box modes. The CoT monitorability paper names latent reasoning and RL training that drifts from legible English as two choices that delete monitoring as a safety tool. Ask vendors directly whether their exposed chain-of-thought is the real reasoning trace or a post-hoc summary, because a summary gives you no signal on deceptive behavior.

Governance and rollout: OWASP LLM Top 10 mapping and a buyer checklist

↑ CONTENTS

The controls in this playbook need a frame that auditors, buyers, and boards recognize. Two reference points do most of the work: the OWASP Top 10 for LLM Applications as the risk vocabulary, and the NIST AI Risk Management Framework as the operating model. ISO/IEC 42001 and MITRE ATLAS round out the auditable and adversarial views.

EVIDENCE

The OWASP Top 10 for LLM Applications 2025 edition (published November 17, 2024) lists ten categories: LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM03 Supply Chain, LLM04 Data and Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, LLM09 Misinformation, and LLM10 Unbounded Consumption [42]. The 2025 edition is a substantial revision: only three categories survived unchanged from 2023, with new entries for System Prompt Leakage and Vector and Embedding Weaknesses reflecting the rise of RAG and agents [48]. Excessive Agency (LLM06) is framed as a systemic governance failure that makes least privilege a core deployment control [48], and a practical analysis notes most OWASP LLM risks are runtime problems that application code alone cannot solve, requiring infrastructure-layer controls such as gateways, agent identity, fine-grained authorization, and immutable audit logs [48].

NIST AI RMF 1.0 (NIST AI 100-1, January 2023) organizes risk management around four functions, Govern, Map, Measure, and Manage, where Govern spans the organization and the other three run iteratively per system [43], and its Map function requires documenting purpose, users, stakeholders, and third-party dependencies as a pre-deployment gate [43]. NIST AI 600-1 (Generative AI Profile, final, July 2024) extends the framework to twelve generative-AI risk categories including Confabulation, Data Privacy, Information Security, and Value Chain Integration [44]. ISO/IEC 42001

, the first AI management system standard, requires an AI system inventory, impact assessments, documented controls, and supplier oversight, providing the auditable management layer NIST does not prescribe [45]. MITRE ATLAS v5.1.0 (November 2025) catalogs 16 tactics, 84 techniques, and 56 sub-techniques, with a Spring 2025 expansion adding 19 generative-AI techniques such as RAG poisoning and AI supply-chain compromise [46]. On timelines, enterprise governance maturity takes sustained investment: a foundational program runs 4 to 6 months and reaching higher maturity takes 12 to 24 months [47], and buyers should require SOC 2 Type II, comprehensive audit logging, a policy engine with block, allow, and redact modes, and SIEM and GRC integration as baseline capabilities [47].

Mapping this playbook onto the OWASP LLM Top 10 2025:

OWASP 2025 entryWhat it coversWhere this playbook addresses it
LLM01 Prompt InjectionDirect and indirect instruction subversionPrompt injection and tool abuse [9][14]
LLM02 Sensitive Information DisclosureTokens, keys, and private data leaking through prompts and logsSecrets and credentials [30]
LLM03 Supply ChainCompromised models, data, and dependenciesGovernance and rollout (ATLAS supply-chain mapping) [46]
LLM04 Data and Model PoisoningTampered training or retrieval dataData leakage controls and index hygiene [19][20]
LLM05 Improper Output HandlingUnsafe handling of model output downstreamContainment: output filtering before tool calls [21]
LLM06 Excessive AgencyToo many tools, permissions, or autonomyContainment: least privilege and action gating [21][48]
LLM07 System Prompt LeakageExposure of system-prompt contentsSecrets handling and output filtering [30]
LLM08 Vector and Embedding WeaknessesRAG retrieval and embedding attacksData leakage from RAG agents [17][18]
LLM09 MisinformationConfabulation and overrelianceMonitoring and human review of outputs [44]
LLM10 Unbounded ConsumptionResource and cost exhaustionMonitoring: cost and latency anomaly rules [39]

DO THIS

  1. Map every deployed or planned LLM application against the OWASP Top 10 2025, recording for each entry whether the risk is present, the control in place, and the residual risk, and keep the register live across model and integration changes [42].
  2. Adopt the NIST AI RMF four-function structure: run Govern organization-wide, and run Map, Measure, and Manage per system before production and on a recurring basis [43].
  3. Apply the NIST Generative AI Profile’s twelve risk categories when assessing generative systems, paying special attention to Confabulation, Data Privacy, Information Security, and Value Chain Integration [44].
  4. Cross-reference your OWASP controls against MITRE ATLAS techniques to find detection gaps, mapping RAG poisoning to LLM08, prompt crafting to LLM01, and supply-chain compromise to LLM03 [46].
  5. Enforce least privilege against LLM06 before deploying any tool-calling agent: document tools, permissions, and data scopes per agent identity, replace shared service accounts, and require approval for writes, external calls, and code execution outside a sandbox [42][48].
  6. Use a non-negotiable buyer checklist when procuring a platform: SOC 2 Type II, audit logging of prompts, outputs, users, and timestamps, a block, allow, and redact policy engine, SIEM and GRC integration, data-residency controls, and SSO with MFA, disqualifying vendors who cannot demonstrate all six [47].
  7. Require evidence of ISO/IEC 42001 alignment from vendors: an AI system inventory, a documented impact-assessment process, and supplier oversight, with accredited certification as the strongest vendor-neutral signal [45].

OUR TAKE — OPINION, NOT SOURCED

Run a staged rollout aligned to the NIST Map, Measure, Manage sequence. Phase 1 (months 1 to 3): complete an AI system inventory, assign system owners, and publish an acceptable-use policy. Phase 2 (months 4 to 6): red-team every production system against OWASP LLM01 and LLM06 and deploy gateway-level prompt guardrails and data-loss prevention. Phase 3 (months 7 to 12): stand up continuous monitoring dashboards, quarterly compliance audits, and semi-annual OWASP re-mapping. Budget 12 to 24 months to reach sustained maturity, and resist the temptation to treat the inventory as a one-time exercise: the agent you secured last quarter becomes a new risk the moment it gains a tool.