Every material agent needs a known way to stop it. Most organisations find out during an incident that what they have is a belief rather than a mechanism — and the belief is almost always the same one: that disabling somebody’s account will do it.
Stopping is not one act. It is five: halt what is executing, disable the trigger so it does not restart, revoke the credential, remove the tools, and deal with the effects already in motion. A stop that achieves four of the five is the incident, not the resolution.
Plain-English explainer · ISO/IEC 42001:2023 · NIST AI RMF · EU AI Act Article 14 · Last reviewed August 2026
A kill switch is not a button. It is a set of five effects — the running task ends, the trigger stops firing, the credential stops working, the tools stop answering, and the actions already in motion are contained — produced quickly enough to matter and available to a person who is awake at 3 a.m. and did not build the thing. A circuit breaker is the automatic version of the same idea: a pre-agreed condition that stops the agent without waiting for a human to notice. The reason this deserves its own page is that the failure is so consistent. Ask an engineering team how they would stop an agent and the answer is usually “we would turn it off”, which on inspection means disabling one user account while the agent runs on a service credential, fires from a scheduler nobody owns, and has already submitted work to a third party. What follows is what a stop must actually achieve, the layers available and what each really costs, what the delay costs once you multiply it by the agent’s action rate, where the stop actually lives on each class of platform, the breakers worth building and how to choose a threshold, how handover works mid-workflow, the runbook a responder can use without calling the author, the control wording and test procedure an examination needs, and the populations an assessor will define before any of it counts as evidence.
Key Takeaways
- A kill switch has to produce five effects, not one: halt execution in flight, disable the trigger so nothing restarts, revoke the credential the agent acts with, remove its tool access, and contain the actions already set in motion.
- Disabling the user account is not a kill switch. The agent usually runs on its own identity, the token it already holds may stay valid until it expires, the trigger lives in another system, and work already accepted downstream completes regardless.
- Circuit breakers are the automatic counterpart — retry limits, consecutive-failure and error-rate trips, step budgets, spend ceilings, rate and blast-radius limits, anomaly triggers — and every one of them needs a defined safe fail state that holds work rather than releasing it.
- An agent should usually stop itself after repeated failures, and the threshold follows an asymmetry test: where the action is reversible and cheap, allow a few. Where it sends, pays, deletes or deploys, the defensible number is often one.
- Human takeover is a handover, not a notification. The responder needs the task, the state already committed, the tool-call record rather than a summary the agent wrote, the authority to complete or abandon, and a rule that nothing releases by timeout.
- An untested kill switch is a belief. Test it on a date, in an environment sharing the real trigger and the real identity, executed by an on-call responder who did not build the agent, and keep the record: layers exercised, measured time to effect, and what was still running afterwards.
What a stop must actually do
Write these five down for each material agent before you design anything. Most stop designs fail because they achieve one or two of them convincingly and leave the rest to an assumption nobody has tested.
| Halt what is executing now | The loop currently running has to end, including the tool call already in flight. An agent mid-task is not idle software waiting to be switched off. It is a process that will take its next action in a few seconds unless something ends the process, cancels the task in the orchestrator, or makes the next call fail. |
| Stop it starting again | Whatever caused the run — a schedule, a webhook, a queue consumer, an event subscription, a CI trigger, another agent — has to be disabled too. A stop that ends one run and leaves the trigger armed buys you the interval between runs and nothing else. |
| Revoke the credential it acts with | The agent holds an identity: a service principal, a workload identity, an API key, an OAuth grant. Revocation is what makes the stop hold at systems you do not control. It is also the layer where organisations discover that a token issued ten minutes ago may stay valid for the rest of its lifetime, which on some platforms is measured in hours rather than minutes. |
| Remove the tools it can reach | Tools and connectors frequently carry their own credentials and their own network paths. Disconnecting an MCP server, disabling a connector, or removing a function from the agent’s registered tool list is a separate act from revoking the agent’s identity, and on some platforms it is faster. |
| Prevent the effects already set in motion | Stopping is not undoing. Queued messages, a payment batch already submitted, a webhook a third party will retry, a downstream job that consumes the agent’s output, and a retrieval index that already copied the data all survive the stop. This is the step most runbooks omit and most incidents turn on. |
Two properties sit across all five. The stop has to be fast enough to matter, which means you state a target — the useful question is not “can we stop it?” but “how many actions will it take between the decision and the effect?” at the rate this agent actually works. And it has to be operable by whoever is awake: a named on-call rota with standing access and standing authority, not a vendor support ticket, not a break-glass procedure nobody has exercised, and not the engineer who wrote it and is on leave.
Why disabling the account is not a stop
Offboarding a person is a well-understood control and it works on people. Applied to software it addresses one of the five effects, partially, and creates a false sense that the incident is closed. Six reasons it does not hold, and each of them surfaces in a live test rather than in a design document.
| The agent is usually not running as that user | Unattended workloads run on service principals, machine accounts, workload identities or a vendor-hosted runtime holding its own credential. Developer-side agents are the mirror image: Claude Code, Cursor and GitHub Copilot typically act with a developer’s own tokens, in a local session that no agent platform’s control plane can reach, so disabling the directory account may leave that session and a cached credential working. Establish which of the two shapes each agent is before writing its stop, because the layer that works is different in each case. |
| The token already issued does not care | Access tokens are typically valid for a bounded lifetime — commonly minutes to about an hour — and many resource servers accept them until expiry without re-checking the directory, though some platforms now push revocation events to supported resources in near real time. Microsoft’s continuous access evaluation is the common example, and whether your dependencies honour it is a question to answer per resource, not per tenant. Disabling the account stops new tokens being issued. It does not always stop the one in the agent’s memory, and offline or refresh grants can survive far longer unless they are revoked explicitly. |
| The trigger lives somewhere else | Schedulers, queues, webhook subscriptions and CI pipelines are separate systems with separate permissions. Disabling an identity in the directory does not pause a cron entry, drain a queue, or stop a partner posting to your endpoint. |
| Other holders of the same credential keep working | A shared API key baked into three deployments, a secret in a CI variable, a copy in a developer environment: revoking one identity leaves the others live. This is why the inventory entry has to record where the credential is held, not only that one exists. |
| Work already accepted downstream still completes | A batch submitted to a banking portal, an email in a provider’s send queue, a ticket another automation will pick up. None of these are inside your agent, and none of them stop because an account was disabled. |
| Self-healing puts it back | Container restart policies, orchestrator supervisors, CI retries and infrastructure-as-code reconciliation are all designed to undo exactly the change you just made. A stop performed in a console and not in the source of truth is reverted by the next pipeline run, often within the hour and with no alert. |
None of this argues against revoking the identity — it is one of the strongest layers on the ladder below, and it is the layer that reaches systems you do not administer. The argument is against treating it as the whole answer. If the agent has its own identity, as it should, then human offboarding does not touch it at all, which is the point covered in agent identity and delegated authority.
Stop mechanisms and what each really costs
Read the fourth column first. Every layer stops something and leaves something running, and the residue is the part that produces the second incident. A real stop is normally two or three of these layers in a stated order, not one heroic switch.
| Layer | What it stops | Time to effect | What still runs | Who must be able to pull it |
|---|---|---|---|---|
| Pause the trigger | New runs starting — schedule, webhook subscription, queue consumer or event rule. | Seconds, if you own the scheduler and have the console open. | The run in flight, everything already queued, and any second trigger nobody documented. | On-call, from a runbook link, without a change ticket. |
| Cancel the running task | The current loop, at the orchestrator or the runtime. | Seconds to a minute. | Tool calls already dispatched and awaiting a response, and any effect already committed. | On-call, with execute rights on the orchestrator — not the team that wrote the agent. |
| Flip to a degraded mode | The dangerous verbs, while retrieval and drafting continue. | Seconds, if it is a flag. Hours, if it is a deployment. | Everything the read path can still disclose. Degraded is containment, not a stop. | On-call. This is the layer that gets used most, because it costs least. |
| Remove the tool or connector | One capability, at the next call — the surgical option when only one verb is misbehaving. | Next tool call, typically under a minute. | Every other tool, cached results, and anything the tool already did. | On-call, in the agent platform. Record who else that change affects. |
| Revoke the agent credential | Everything that identity can reach, including systems you do not administer. | Immediate for a deleted key. Up to one token lifetime where a live access token is honoured. | Long-lived refresh or offline grants until explicitly revoked. Other holders of the same secret. | On-call, with standing rights in the secret store or directory. Know what else breaks. |
| Deny at the target system | The action at the resource — an identity policy deny, a database role revoke, a gateway rule. | Seconds to minutes, depending on policy propagation. | Other targets. This is per-system, so it is precise and incomplete by design. | Platform on-call. Usually a different rota from the agent owner, which is the delay. |
| Cut network egress | Reach beyond the environment, including calls to the model provider. | Seconds to minutes. | Local execution and anything inside the boundary. A blunt instrument with wide collateral. | Network or platform on-call, under a pre-agreed authority to act. |
| Disable the deployment in the vendor platform | The agent for every user of that deployment. | Minutes. | Personal-account use of the same product, and any copy of the workflow in another workspace. | The tenant administrator. Confirm that role is on a rota, not held by one person. |
| Contain the downstream effects | Nothing — this is the cleanup that makes the stop meaningful. | Minutes to hours, and often depends on a third party’s cut-off time. | Whatever cannot be recalled. Write that list before the incident, not during it. | The business owner, with the counterparty relationships and the authority to cancel. |
The ordering matters more than the inventory. Reaching for credential revocation first feels decisive and frequently breaks three unrelated systems that share the secret, after which the incident acquires a second workstream. The pattern that holds up is: pause the trigger, cancel the run, remove the offending capability, then decide whether the identity needs to go — and start the residue check in parallel rather than after, because the cut-off times that constrain containment are set by other people.
What the delay actually costs, at three action rates
A time to effect means nothing on its own. Multiplied by the rate the agent actually works at, it becomes the only number that matters: how many more actions happen between the decision and the effect. Measure your agent’s observed action rate, then read across. The figures below are arithmetic, not benchmarks.
| Agent profile | 1-minute trigger pause | 5-minute tool removal + confirm | 15-minute credential path | The design implication |
|---|---|---|---|---|
| Refund agent About 4 actions a minute | 4 more refunds | About 20 more refunds | About 60 more refunds | Sixty irreversible refunds is an incident in its own right. No human stop path is fast enough here, so the control that matters is the per-run ceiling that trips automatically. |
| Data-enrichment agent About 40 actions a minute | About 40 records altered | About 200 records altered | About 600 records altered | Six hundred records changed before the credential path completes. At this rate human stopping is decorative. The breaker and a hard per-run ceiling are the only real controls. |
| Batch reporting agent About 1 action an hour | Usually none | Usually none | At most one | Every layer on the ladder is fast enough. Spend the design effort on the residue check instead, because the single action already dispatched is the whole exposure. |
The table forces a conclusion that is easy to avoid stating. Above a certain action rate, no human stop path is fast enough — and the only control that actually bounds the damage is an automatic breaker or a hard per-run ceiling. For the enrichment agent in the middle row, every layer of the ladder is theatre: the decision to stop it is correct and the mechanism is sound, and six hundred records have still changed. That is the whole reason the next section exists. Human stopping is the containment path for slow agents and the post-mortem path for fast ones, which is a design fact rather than a criticism of the responder.
The stop, by platform class
The ladder above is generic by design. Which rungs actually exist depends on who owns the runtime, and that is the question most readers arrive with. Read the third column first: what has no stop at all is the part that belongs in your register as a stated limitation. Everything here is as documented at the time of writing — product controls and plan tiers change, so verify in current vendor documentation and, better, test it.
| Platform class | What an administrator can reach | What has no stop at all | What to verify, and test |
|---|---|---|---|
| Orchestrator you run yourself Your own agent framework, queue consumers, scheduled jobs, self-hosted runtimes. | Cancel the run through the orchestrator API, pause the queue consumer or the schedule, scale the worker deployment to zero, and revoke the workload identity. | Nothing structurally out of reach — which is why the failure here is procedural rather than technical. | That the stop is made in the deployment manifest and not only in the console. The source of truth is the manifest. A console change is reverted by the next reconciliation and no alert fires when it is. |
| Vendor agent platform, your tenant Copilot Studio agents and comparable low-code agent builders. | Unpublish or deactivate the agent, disable it at the environment level, remove the connections it depends on, and manage it as an app in the tenant’s integrated-apps controls. | A task-level cancel. Administrative actions govern availability, not the run already executing. | Whether sessions already open terminate, and how quickly — this is the answer that varies, and it is the one that decides your real time to effect. Test it. Do not read it off a feature page. |
| Embedded assistant on a per-seat licence Microsoft 365 Copilot, Gemini for Workspace. | Coarse levers only: remove the licence, turn the service off for an organisational unit or group, block the app tenant-wide in the admin centre, and tighten the data the assistant is allowed to reach. | A per-session kill. None of the levers is a cancel, and the assistant is embedded in a product your users are still working in. | How fast a licence removal or service-off reaches a session already open, which is tier-dependent and thinly documented. Record the answer as a stated limitation in the register rather than describing a control you do not have. |
| Developer-side coding agents Claude Code, Cursor, GitHub Copilot. | Revoke the personal access token or OAuth grant, apply organisation-level policy, and fall back on repository and branch protection — the controls that sit at the target rather than at the agent. | An account disable that reaches the local session. The credential is usually the developer’s own and the session runs on a machine the agent platform does not administer. | Which credential each tool actually presents to your systems, and whether revocation of that credential is honoured immediately by the git host and the package registry. |
| MCP servers and connectors Model Context Protocol servers, first- and third-party connectors, plug-ins. | Remove the server from the registry or from client configuration, revoke the server’s own credential separately from the agent’s, and cut the egress path to it. | A single switch. Two credentials are in play — the agent’s and the server’s — and revoking one leaves the other live. | Whether removing the server from the client configuration reaches every client, and whether the server holds standing access to your systems that survives the agent being stopped. |
| CI/CD-triggered agents Pipeline steps, bots acting on pull requests, scheduled workflow jobs. | Disable the workflow, remove the OIDC trust relationship or the federated credential, and revoke the runner’s token. | A durable console-level stop. The pipeline is declarative, so it re-applies whatever the repository says. | That the change is made in the repository and not in the UI, and that no second workflow in another repository triggers the same agent. |
One pattern runs through the whole table. The further the runtime sits from your control plane, the more the stop moves from a cancel to a revoke — and a revoke acts on the next call, not on the current one. That is not a defect to be engineered away. It is the price of buying rather than building, and the right response is to write the real time to effect in the register and size the agent’s authority to match it. An embedded assistant that can only be stopped by licence removal should not be the thing that issues refunds. The tool-level half of this question — what a connector can add or redefine without a change on your side — is covered in MCP tool approval and capability change.
Circuit breakers and their fail states
A kill switch depends on somebody noticing. Agents fail quietly — a wrong action and a right action return the same success response — so the mechanism that matters most is the one that trips without a human. The breaker pattern borrowed from distributed systems applies directly: closed while healthy, open on a trip, and half-open for a single trial call before resuming. The two columns that decide whether a breaker is real are the fail state and the prohibition beside it.
| Breaker | Trips on | Choosing the threshold | Safe fail state | What it must never do |
|---|---|---|---|---|
| Retry limit | The same action fails and is attempted again beyond a set count. | Two or three for idempotent reads. One for anything that sends, pays, deletes or deploys — a retried irreversible action is a second irreversible action. | Stop the task, record the failure, raise it to the takeover queue. | Retry with wider permissions, or retry a different tool to achieve the same effect. |
| Consecutive-failure breaker | A named dependency or tool returns errors N times in a row. | Measure the normal failure rate for a fortnight first. Set the trip above ordinary variance and below the point where a human would have noticed. | Open the breaker for that tool only. The agent continues without it or halts, depending on whether the tool is load-bearing. | Reset itself on a timer and resume the full queue. Allow one trial call, not a hundred. |
| Error-rate breaker | Failures exceed a proportion of actions over a rolling window. | A rate plus a minimum volume, so three actions with one failure does not trip a breaker sized for three hundred. | Pause new tasks, drain in-flight work, alert the owner. | Count model refusals as dependency errors. They are a different signal and belong on a different counter. |
| Step and tool-call budget | A single task exceeds a maximum number of reasoning steps or tool calls. | Roughly double the observed 95th percentile for that task type. Loops are the characteristic agent failure and they are cheap to bound. | Abandon the task, preserve the transcript, hand to a human with the state attached. | Silently truncate and report success. A truncated task that reports done is worse than a failure. |
| Spend ceiling | Inference or transaction spend crosses a per-action, per-hour or aggregate limit. | Three separate numbers. A per-action ceiling stops one catastrophic call, a velocity limit stops a fast burn, and an aggregate cap stops a slow one that no single run would have caught. | Hard stop at the aggregate cap. Hold and alert at the velocity limit. | Fail open because the billing API was unreachable. Unknown spend is not zero spend. |
| Rate and blast-radius limit | Actions per minute, recipients per hour, or distinct records touched in one run exceed a ceiling. | Set from the business process, not from the platform quota. If a human would send forty emails a day, four hundred is a defect whatever the API permits. | Pause and hold the remainder. Release requires a person, not an expiry. | Complete the current batch first. The batch is the thing you are trying to stop. |
| Novelty and anomaly trigger | An action type never seen before, a recipient outside the allow-list, a parameter far outside normal range, or activity outside the agent’s normal hours. | Structural rather than statistical where possible. An allow-list breach is a yes or no, while an amount outlier needs a stated multiple of the historical distribution. | Hold the single action for review. Do not necessarily stop the agent. | Depend on the model’s own confidence score. Self-reported certainty is not a control signal. |
| Instruction-origin anomaly | An instruction to take an action originates from retrieved content, a tool response or an inbound message rather than from the approved task definition. | Structural, not statistical. Either the instruction came from the task definition or it came from data the agent read, and only the first carries authority. There is no distribution to baseline here. | Hold the single action for review with the originating content attached. Retrieval and drafting may continue, but the verb does not. | Treat the model’s own assertion that the instruction was legitimate as the check. That is the injected channel being asked to judge itself. |
| Human-rejection-rate breaker | The proportion of the agent’s proposals rejected at the human gate rises above a threshold over N items. | Baselined from the first month of operation. This is the breaker that catches quality drift after a model change, and almost nobody builds it. | Return the workflow to fully supervised mode and notify the owner. | Trip so late that a thousand proposals were reviewed by a fatigued approver first. |
| Health and freshness breaker | A dependency returns stale or degraded data, or the underlying model version changes unexpectedly. | A staleness window per data source, and an exact-match check on the model identifier recorded at approval. | Stop. Acting confidently on stale source data is the failure nobody detects from the output. | Continue on a cached copy without saying so in the record. |
Two design rules cut across the table. First, every breaker needs two counters — one per run and one per period — because a breaker that resets at the start of each run cannot see a slow burn across a thousand runs, which is the shape most spend and blast-radius incidents actually take. Second, no breaker may fail into more authority. Falling back to a wider permission, a different tool that achieves the same effect, or an unsupervised retry converts a safety mechanism into an escalation path. Where the fail state is a human queue, that queue is now part of the control and inherits the staffing question below.
The instruction-origin breaker deserves a note, because it is the one that connects this page to the most common cause of an agent doing something that needs stopping. An agent that reads a document, a ticket, a web page or a tool response is reading a channel an attacker may control, and a model has no native way to distinguish an instruction that arrived in the task definition from one that arrived in the data. Treating the origin of the instruction as a structural check — rather than asking the model whether the instruction looked legitimate — is the only version of that control that survives scrutiny. The mechanics are set out in prompt injection and tool calls, and the reason it belongs here is that injection is what turns a working agent into an incident faster than any model defect.
Should it stop itself after repeated failures
Usually yes, and the reasoning is worth setting out because the instinct to keep going is strong in teams who built the thing. An agent that fails repeatedly is, by definition, operating outside the conditions it was assessed under. Whatever assumption it was relying on — a schema, an API contract, a data source, a permission — has changed, and it cannot tell you which. Continuing produces more actions on a broken premise at machine speed, and the actions are frequently indistinguishable from correct ones until somebody reconciles them.
The threshold follows an asymmetry test: what does an unnecessary stop cost, against what does an unstopped failure cost? Those two numbers are rarely close. For reversible internal work — drafting, classifying, enriching a record that a person will read anyway — a false stop is an inconvenience, and three or four failures before tripping is reasonable. For anything that sends, pays, deletes or deploys, an unstopped failure is external, irreversible, and multiplies by the rate the agent works at, so the defensible threshold is often one. Set thresholds per verb, not per agent, because a single agent usually holds verbs from both ends of that range.
Four practical rules make the numbers hold up under review. Baseline first: measure the agent’s ordinary failure rate over a fortnight before choosing a trip point, because a threshold set below normal variance produces noise, and noisy breakers get raised in a settings screen with no change record. Trip fast and reset slowly — hysteresis is the point of the pattern, and for high-impact verbs the reset should require a person rather than a timer. Distinguish a dependency error from a model refusal. They are different signals and mixing them on one counter hides both. And decide what the trip does to work in progress: the answer that survives scrutiny is that held work expires rather than releases, so nothing reaches production because a timer ran out.
There is one honest exception. Where stopping is itself the hazard — an agent in a safety, medical or availability-critical path — the safe state may be a degraded mode rather than a halt, which is precisely why the EU AI Act phrases the requirement as coming to a halt in a safe state rather than simply stopping. Decide what that safe state is during design, write it down, and test it. What does not survive review is treating “stopping would be disruptive” as a reason to have no breaker at all.
Human takeover mid-workflow
Most stops are not the end of the work. They are a transfer of it. The conditions that should trigger a takeover are worth naming in advance, because deciding them live is how teams talk themselves into continuing. The durable list is structural rather than intuitive: (1) a breaker has tripped and the cause is not obvious, (2) the task requires an action outside the approved boundary, (3) the source record is ambiguous and the agent would have to guess, (4) a counterparty, customer or regulator is involved, or (5) an irreversible verb sits above its threshold. Deliberately absent from that list is the model’s own expression of uncertainty — self-reported confidence is not a control signal, and a takeover rule built on it will fire at the wrong times in both directions.
Handover then has to carry state, which is where most implementations are thin. A notification saying the agent has stopped is not a handover. The responder needs five things: (1) the task and why it stopped, (2) what has already been committed, with the tool-call record rather than a summary the agent wrote about itself, (3) what remains to be done, (4) the source records to check against, and (5) the authority to complete, correct or abandon it. The summary point is not pedantry. A model-written account of its own run is the least reliable artefact in the incident, and it is the one most platforms surface by default.
Three mechanics decide whether the handover works. Ownership must be a single named person, not a channel — a queue addressed to a team is a queue nobody is accountable for. The half-done problem needs an answer before the incident: which actions are safe to repeat, and which need a compensating action instead, because a responder who cannot tell will either duplicate an effect or leave a process half-applied. And re-entry needs a rule: whether the agent resumes the task, restarts it, or is finished with it, and who authorises that. Silence on re-entry is how an agent quietly picks up work a human has already partly done.
The uncomfortable consequence is a staffing one. An agent that runs unattended at 3 a.m. needs a takeover path at 3 a.m., or it should not hold irreversible verbs at that hour. Restricting an agent’s authority to covered hours and queuing the rest until morning is a legitimate, defensible design — considerably more defensible than an unattended agent whose only stop is operable by its author. The related gate design — where a human reviews before the action rather than after the stop — is a separate control with its own failure modes, and the two are frequently confused: an approval gate that has become a rubber stamp will not save you, and a stop ladder does not substitute for one.
Seventeen minutes, and three things it missed
An illustrative composite, drawn from the patterns we see in readiness work rather than from any one organisation. The point of carrying it end to end is that the stop worked, the runbook was followed, and the residue was still the expensive part.
| In place | Before 4 May 2026 | A support agent handles refund requests under two stated ceilings — USD 1,000 per refund and twenty-five refunds in one run (illustrative figures throughout). It reads the ticket and the order record, proposes a refund, and executes it directly below the per-refund ceiling with a human release above it. It runs on its own service principal, triggered by a queue consumer, with three tools: ticket system, order database, payments API. |
| The trip | 11 May, 02:14 | A schema change upstream leaves the order-total field null for one merchant. The agent falls back to the amount the customer stated in the ticket, so the refunds it issues match what was claimed rather than what was paid. A run that would have issued about forty refunds is stopped at the twenty-sixth against the ceiling of twenty-five: the blast-radius breaker holds the remainder and pages the on-call. |
| The stop | 11 May, 02:19 to 02:31 | The on-call pauses the queue consumer at 02:19 — new runs stop in under a minute. The payments tool is removed from the agent at 02:22, which ends the refund capability at the next call while retrieval continues. The service principal is not revoked, because the runbook says the tool removal is sufficient and the credential is shared with the nightly reconciliation job. |
| The residue | 11 May, 02:31 to 04:05 | Twenty-five refunds were issued before the breaker held the run. Fourteen were still awaiting capture and are voided inside the provider’s cut-off. The residue check finds three effects the stop did not reach: (1) the other eleven, totalling USD 9,410, already accepted by the payments provider and irreversible, against order values of USD 2,190 — an over-refund of USD 7,220, (2) four customer emails sitting in the provider’s send queue, and (3) a settlement export scheduled for 05:00 carrying roughly USD 240,000 across nine merchants, with the affected merchant’s line wrong by the full over-refund. The send queue is suppressed. The export is cancelled by the finance owner, who is on the escalation path precisely for this. |
| The record | 11 May, by 09:00 | Elapsed time from page to no-further-actions is put at seventeen minutes, against a documented target of fifteen. The USD 7,220 over-refund is recorded as unrecoverable and communicated to the merchant the same morning. Two defects are raised: the runbook had no cancellation path for the settlement export, and the queue-consumer pause required a role the second on-call did not hold. |
| The change | 18 May to 5 June | A freshness breaker is added on the order-total field so a null trips a stop rather than reading as zero. The on-call role assignment is corrected. The refund ceiling is unchanged, because the ceiling was not the failure. The next scheduled stop test is moved forward to 12 June and executed by the on-call who was not involved in the incident. |
Three details in that sequence are the ones worth transplanting. The responder chose tool removal over credential revocation because the runbook already recorded that the service principal was shared with another job — a fact discovered during a test in March, not during the incident. The residue check was a standing list rather than a conversation, so the settlement export was found at 02:40 rather than at 05:00. And the corrective action addressed the detection gap, not the refund ceiling, because the ceiling had performed exactly as designed. A post-incident change that tightens the wrong parameter is how an agent becomes useless without becoming safer.
What the assessor asked for in September
Four months later the same incident arrives as an evidence request, and it is worth walking it against the seven requests in the next section, because the gap between a well-run incident and a well-evidenced one is wider than teams expect. Four artefacts existed and were produced without difficulty: the paging record with its timestamp, the version of the runbook in force on 11 May, the change entry showing the payments tool removed at 02:22, and a system-generated export of the agent’s actions spanning 02:19 to 04:05 with the query parameters and record count on the face of it.
One had to be reconstructed. The seventeen-minute elapsed time was derived after the fact from log timestamps, because nobody started a clock until 02:26 — defensible, but it is an inference rather than a measurement, and the assessor recorded it as such. And one did not exist at all: there was no approval record for the re-enable on 18 May, because the decision was taken on a call and never written down. The stop worked, the residue was caught, the corrective action was correct, and the control still carried an exception — for the one step that cost nothing to record and was not recorded.
The transferable lesson is procedural rather than technical. Two fields in the incident template would have closed both gaps: a decision timestamp captured at the page rather than derived afterwards, and a named approver on the restart. Add them before you need them, because neither can be created retrospectively without the record showing that it was.
A tested runbook, section by section
Nine sections, each with the failure it exists to prevent. Write it for a responder who did not build the agent, is reading it at 3 a.m., and has never used the console it points at. If a section cannot be completed without asking the author, that is the finding.
| Section | What it must contain | The failure it prevents |
|---|---|---|
| 1 · Header | Agent name and register ID, accountable owner, whether it runs attended or unattended, the on-call rota and escalation path, and where this runbook is stored — including a copy reachable when the identity provider or the wiki is the thing that is broken. | The twenty minutes at the start of every incident spent working out who is allowed to decide, and the runbook that lives behind the single sign-on that is currently down. |
| 2 · Decide | Three named conditions with examples: stop now, throttle or degrade, watch and record. Written so a responder who did not build the agent can classify what they are seeing in under two minutes. | Escalation debates during the incident, and the opposite failure — a stop pulled for a cosmetic anomaly at the busiest hour of the month. |
| 3 · Stop ladder | The layers in order, each with the exact console path or command, the expected time to effect, and the observable that confirms it worked — a last-action timestamp or an empty queue, not a settings screen. | A responder confirming they clicked the button rather than confirming the agent stopped. Those are different facts. |
| 4 · Residue check | The standing list of what continues after each layer: in-flight calls, queued tasks, work already submitted to a third party, scheduled downstream jobs, and other agents that consume this one’s output. | The most expensive incident pattern there is — the agent stopped, everybody stood down, and the batch cleared at the bank the following morning. |
| 5 · Contain | The compensating actions, who may perform each, and the cut-off times that constrain them: cancel the batch, suppress the send, recall or correct, notify the counterparty, quarantine the index. | Discovering at 03:40 that the only person who can cancel a payment file is asleep and the portal closes at 04:00. |
| 6 · Preserve | Four named exports, because the record lives in four different systems with four different retention defaults: the orchestrator run history, the model provider’s API request records, the tool or MCP server logs, and the target system’s own audit trail. Take all four before anything is re-enabled or rotated, and write each source’s retention window beside it. The agent’s own transcript is the least reliable of the four and must never be the only export. | An investigation that ends at the edge of a vendor’s default retention, which is frequently shorter than the period an assessor will ask about — and the worse version, a reconstruction resting entirely on the account the agent wrote about its own run. |
| 7 · Communicate | Who is told, within what time, and the customer-facing line — plus the clocks that start on detection rather than on confirmation. Three are worth naming in the runbook itself: (1) customer contract notification windows, commonly in the twenty-four to seventy-two hour range and worth reading before the incident, (2) the GDPR Article 33 seventy-two-hour supervisory-authority clock where personal data is involved, and (3) the DPDP Act obligation to intimate the Data Protection Board and affected data principals of a personal data breach. Confirm exact scope and timing with counsel rather than from a page like this one. | A notification obligation missed while the technical work was going well, which is how a contained incident becomes a reportable one — and the argument at 04:00 about whether the clock started at the page or at the root-cause confirmation. |
| 8 · Restart authority | The named role that may re-enable, the checks required before they do, and the explicit prohibition on re-enabling to see whether it happens again. | The second occurrence, which is always harder to explain than the first because the record shows a decision rather than a surprise. |
| 9 · Record | Incident record fields and, separately, the stop-test record: date, who invoked it, which layers were exercised, elapsed time from decision to effect per layer, residue found, defects raised, and the next test due date. | Having no answer when a customer’s security team asks for evidence that the switch was ever pulled by someone other than its author. |
The control as it would be written
A runbook is an operating document. What a Section 4 control matrix or an ISO/IEC 42001 Statement of Applicability needs is narrower: a control description that states who does what, how often, and to what population, plus a test procedure that can be executed against artefacts. Two illustrative pairs follow — one manual, one automatic. They are starting points to be tailored to your own environment, not text to be pasted unchanged. The wording is the entity’s own, and the final test approach is the CPA firm’s. A firm that drafted or operated the control cannot then examine it.
Manual control — illustrative
Management maintains a documented stop procedure for each agent recorded on the AI register. On a quarterly basis, the procedure is executed for agents authorised to perform irreversible actions, by an on-call responder independent of the team that developed the agent, and the results — layers exercised, elapsed time from decision to observable effect, and residual activity identified — are recorded in the incident management system.
Inspected the AI register to determine the population of agents authorised to perform irreversible actions during the period. For the stop tests performed during the period, inspected the test record to determine that the procedure was executed, that the responder was independent of the development team, and that elapsed time and residual activity were recorded. Inspected the system-generated agent action export spanning each stop — including the query parameters, date range and record count — to determine that no further actions were recorded after the stated time to effect.
Automated control — illustrative
Automated thresholds are configured for each agent authorised to perform irreversible actions, limiting actions per run and per period. On breach, the agent halts the affected task and the task is routed to a named reviewer for disposition. Changes to configured thresholds require documented approval.
Inspected the threshold configuration for each in-scope agent at two dates within the period to determine that limits per run and per period were configured and that the fail state routed the held work to a named reviewer. Inspected the platform change log to determine whether configured thresholds changed during the period and, where they did, inspected the approval. For trip events occurring during the period, inspected the alert delivered, the work held, the disposition recorded by the reviewer, and the authorisation to resume. Where no trip occurred, inspected the fault-injection test performed to determine that the threshold was operative.
Three drafting points make the difference between control text that gets tested and control text that gets rewritten during fieldwork. State the frequency as a period rather than as “periodically” or “on an ongoing basis”, because an untestable frequency has no population. Name the independence condition inside the control rather than in a footnote, since a stop test executed by the agent’s author tests the mechanism and not the runbook. And keep one control to one thing: bundling the stop test, the threshold configuration and the incident record into a single control means one failure fails all three, which is a worse outcome than three controls with one exception.
Testing it, because an untested kill switch is a belief
A stop design that has never been executed is a hypothesis about several systems you do not fully control. The test has to run against the real trigger and the real identity — a rehearsal in a staging environment proves the staging configuration and nothing about production, because the differences that break a stop are exactly the ones staging normalises away. Where a full production test is genuinely unsafe, test each layer independently against production and record what could not be exercised together, rather than substituting a simulation and calling it a test.
Test the person as well as the mechanism. The version that finds real defects is unannounced, executed by an on-call responder who did not build the agent, using only the runbook, with a stopwatch running from decision to observable effect. The defects that surface are consistently organisational: the console link requires a role the responder does not hold, the scheduler lives in a different tenant, the runbook is behind the single sign-on that is currently degraded, nobody knows who can cancel a file at the bank, and the second on-call has never opened the agent platform. None of those appear in a design review.
Cadence: at deployment, after any change to the identity, the trigger, the tool list or the hosting platform, and at a stated interval thereafter — quarterly for agents holding irreversible verbs, semi-annually for the rest is a defensible market-practice baseline rather than a standards requirement. Add a fault-injection pass for the breakers, because a breaker that has never tripped and a breaker that is not wired look identical from the configuration screen. Retain the test records: they are the evidence discussed next, and they are also the fastest answer to a customer’s security questionnaire.
The evidence an assessor asks for
Whether the examination is a SOC 2 engagement by a licensed CPA firm or an ISO/IEC 42001 audit by an accredited certification body, the requests take the same shape: define the population, show the control operated on the items selected, and show the configuration held for the whole period rather than on the day of fieldwork.
| The request | What satisfies it | What gets rejected |
|---|---|---|
| The stop design for each agent in the inventory | A register entry per agent naming the trigger, the identity, the tools, the layers available to stop it, the expected time to effect for each, and the accountable owner — reconciled to what is actually deployed. | A statement that the platform supports disabling. A register field reading “yes”. A design describing an architecture that was replaced two releases ago. |
| A dated record that the stop was tested | Test records inside the period showing who invoked it, which layers were exercised, measured elapsed time from decision to effect, what was still running afterwards, and the defects raised and closed. | A runbook nobody has executed. A vendor feature page. A test performed after period end for a control claimed to have operated during it. A test in an environment that does not share the production trigger or identity. |
| Proof the responder could actually do it | The role or group assignment granting the on-call rota execute rights on the orchestrator, the secret store and the tenant, captured at two dates inside the period, plus at least one test executed by an on-call responder who did not build the agent. | An assertion that the team has access. A break-glass procedure requiring an approval from someone with no on-call obligation. A named individual instead of a rota. |
| Breaker configuration and its change history | The thresholds as configured at two or more points inside the period, plus the platform change log showing they were not quietly relaxed in between, and the approval record for any change that was made. | Today’s configuration screen offered as evidence about a period that closed months ago. A threshold raised in a settings screen with no change record — the single most common finding in this area. |
| Evidence a breaker tripped and what followed | The trip event, the alert delivered to a named recipient, the record of what was held, the human decision taken, and the authorisation to resume. | A breaker with no trips in twelve months and no fault-injection test to show it is wired at all. A trip that resolved itself with no record of who decided. |
| Proof the agent actually stopped | A system-generated export of agent actions spanning the stop, with the query, filter, date range and record count visible, showing the last action before and the absence of actions after. | A dashboard view. A screenshot of a disabled toggle. An export assembled by the person who operates the agent, with no way to test completeness. |
| The post-incident record and restart approval | What happened, what was contained, what could not be recalled, the change made to prevent recurrence, and a dated approval to re-enable from the named role that holds that authority. | A chat thread. A resumption with no recorded decision. A corrective action with an owner but no closure date. |
Population, completeness source, and what gets selected
The requests above describe what an artefact has to show. This table answers the question underneath them, which is the one most teams have not thought about before fieldwork: what is the population, how is its completeness demonstrated, and how many items get selected. Get the population wrong and every artefact you produce is evidence about the wrong set. The counts below are market practice rather than a standards requirement, they are not a sampling table, and the CPA firm or certification body sets the final approach.
| Control | What the population is | How completeness is tested | What gets selected |
|---|---|---|---|
| The agent register itself | Every agent recorded on the AI register that was in operation at any point during the period — not the list as it stands on the day of fieldwork. | Tested against independent sources, never against the register itself: the identity provider’s service-principal and enterprise-application lists, OAuth grant and consent reports, the MCP or connector registry, egress logs to model-provider domains, and AI vendor spend in the expense ledger. A register that reconciles to nothing is the finding, and it is usually the first one. | The whole register is inspected for the design attributes. The reconciliation is the work — one agent found in the egress logs and absent from the register undermines every population downstream of it. |
| Stop tests | Every scheduled and unscheduled stop test performed during the period, per agent. A quarterly control over a twelve-month period has a population of four occurrences per agent. | The test schedule agreed at the start of the period, reconciled to the incident or test-record system, so a missed quarter shows as a gap rather than as an absence of evidence. | Populations this small are normally inspected in full rather than sampled. Market practice, not a standards rule — but a selection of two from four invites the obvious question about the other two. |
| Breaker configuration | The set of configured breakers per agent — thresholds, fail states and alert destinations — evidenced at two or more dates inside the period, typically near the start and near the end. | The platform change log for what moved in between. Two identical screenshots months apart prove the endpoints, not the interval. The change log is what closes it, and any change it shows needs its approval record attached. | All configured breakers for in-scope agents, at each of the chosen dates. This is a configuration inspection rather than a sample. |
| Breaker trip events | All trip events in the period, from a system-generated export showing the query, the filter, the date range and the record count on the face of it. | The export parameters and the record count, plus a reconciliation to an independent count — the alerting system’s own delivery record is the usual second source. | Where the count is small, the whole population is inspected. Where it is large, a selection is made and the basis for it recorded. If the population is zero, the fault-injection test becomes the only available evidence that the breaker is wired at all. |
| Agent-action exports used to prove cessation | The agent’s actions across a window spanning each stop — before, during and after — rather than a window that begins at the stop. | Here the auditor tests the report itself, as information produced by the entity: who ran it, from which system, with what parameters, over what date range, and reconciled to an independent count such as the target system’s own audit trail. An export assembled by the agent’s operator proves nothing about what is missing from it. | The full window for each stop under test. The absence of actions after the stated time to effect is the assertion, and an assertion of absence is only as good as the completeness of the source. |
Two of those rows are where agent controls differ from ordinary IT controls, and both are worth pre-empting. The first is that the register is not its own completeness source. For access reviews an auditor can reconcile to the HR system, but for agents there is no equivalent authoritative list, so completeness has to be assembled from four or five partial sources that each see a different slice — the directory sees identities, the expense ledger sees spend, egress logs see traffic. Build that reconciliation once and keep it, because it is also the answer to the shadow-AI question. The second is that the agent-action export is information produced by the entity, so its own completeness and accuracy are tested before its contents mean anything. An export that proves the agent stopped is only as good as the evidence that the export shows everything the agent did, which is why the query parameters and an independent reconciling count matter more than the screenshot. The general standard applied to both is set out in what auditors actually accept as evidence, and the sampling mechanics in populations and sample sizes.
One retention point decides whether any of this is available when it is asked for. The artefacts here — tool-call histories, agent transcripts, platform audit logs, breaker trip events — frequently live under vendor retention defaults that are shorter than an observation period, and shorter still than the lag before fieldwork requests them. List every log a control description depends on, note its retention window, and schedule an export wherever that window is shorter than the period plus roughly ninety days. The awkward case is the model provider’s own request records, which are frequently the shortest-lived source and the one you cannot extend unilaterally — treat that retention window as a contractual term to be checked at renewal rather than as a technical setting.
What the standards actually say
No standard hands you a kill-switch specification. Between them they establish that a mechanism must exist, that responsibilities must be assigned, that the halt should reach a safe state, and that the whole thing has to be evidenced.
| Source | What it says | What it means for your design |
|---|---|---|
| ISO/IEC 42001:2023 | Clause 8.1 requires operational planning and control over the AI management system. Annex A — a reference control set selected and justified through the Statement of Applicability, not a mandatory list — includes AI system operation and monitoring (A.6.2.6), recording of event logs (A.6.2.8), and processes for responsible use and intended use (A.9.2, A.9.4), in the 2023 edition of the standard. It does not use the phrase “kill switch”. | Clause 6.1.3 has you determine your own controls and then compare them against Annex A to confirm nothing necessary was omitted. So the operative question is not “does Annex A mandate a stop?” but “can you justify the exclusion?” — and for an agent with irreversible verbs you cannot. Certification is issued by an accredited certification body, not by an advisor. |
| NIST AI RMF — MANAGE 2.4 | “Mechanisms are in place and applied, responsibilities are assigned and understood to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use.” | The clearest public statement of the requirement: a mechanism, applied, with assigned responsibilities. Voluntary framework, but the wording is worth adopting verbatim in your own policy because it names all three parts. Quote it as written — the punctuation on this one is frequently altered in secondary sources. |
| EU AI Act — Article 14 | For high-risk systems in scope, human oversight measures must enable a person to intervene or interrupt the system through a stop button or similar procedure allowing it to halt in a safe state, with measures commensurate with the risks, level of autonomy and context of use. | Two ideas worth borrowing whether or not you are in scope: the halt must reach a safe state rather than merely ending, and the oversight must be proportionate to autonomy — the less supervised the agent, the stronger the stop must be. |
| SOC 2 — 2017 TSC (revised points of focus, 2022) | No criterion names an AI agent. The relevant criteria are the existing ones: CC7.3 on evaluating security events, CC7.4 on responding to identified incidents, CC7.5 on recovery activities, CC6.2 and CC6.3 on granting and removing access, and CC8.1 on change. | An agent stop control is tested as an incident-response and access-removal control, not as a new category. If your incident-response programme does not mention agents, that is the gap an examination surfaces first. |
| OWASP Top 10 for LLM Applications (2025) | LLM06:2025 Excessive Agency attributes the risk to three root causes: excessive functionality, excessive permissions and excessive autonomy. | Useful for arguing the design point that the strongest kill switch is a smaller grant. Each of the three root causes maps to a layer on the stop ladder, which makes the list a practical review aid rather than a taxonomy. |
A boundary worth restating, because it is asked about often. ISO/IEC 42001 certificates are issued by accredited certification bodies and SOC 2 reports by licensed CPA firms. Tranquility Cybersecurity designs the control, writes the runbook, facilitates the stop test your on-call responder executes, prepares the evidence and coordinates the examination. It does not certify, attest, audit or sign — and a firm that designed or operated a control cannot then examine it. The distinction matters operationally as well as commercially: the credible test is the one performed by the responder who will actually be paged, which is why an advisor running the test on your behalf would defeat the point of it.
The edge cases that generate the questions
The ladder above covers the ordinary case. These are the situations where a stop design that reads well on paper stops being true, and each of them is cheaper to settle during design than during an incident.
The agent you cannot cancel, because the vendor owns the runtime
An assistant embedded in a product you already licence gives you no way to halt a run in flight. The administrative levers are coarse — remove the licence, turn the service off for an organisational unit or group, block the app tenant-wide — and none of them is a task-level cancel. Microsoft 365 Copilot, Copilot Studio agents and Gemini for Workspace all sit in this category, and the open question is not whether a disable exists but how fast it reaches sessions already open, which is thinly documented and tier-dependent. Confirm it in current vendor documentation and, better, test it, rather than assuming from a feature page. The remaining layers are commercial and structural rather than technical: remove the data the assistant can reach, remove the licences, block the endpoint at the network, and put a right to disable into the contract at renewal. Record the limitation in the register rather than describing a control you do not have — an assessor finds that gap immediately, and a customer’s security team finds it eventually.
The index that keeps the data after the agent stops
Stopping an agent stops it acting. It does not retract what it read or what it wrote, and three residues survive independently of each other. Context and transcript data held by the model provider sits under that provider’s own retention and abuse-monitoring terms, which are contractual rather than technical — verify them in the terms in force for your plan and do not assert them from memory. Embeddings and chunks already written to a retrieval index survive the stop entirely, and deleting the source document does not delete the derived vector unless something is built to do it. That path needs to exist, needs a stated time, and needs someone accountable for running it. Copies the agent wrote into downstream systems by its own actions — a ticket, a CRM note, a file in shared storage, a message to a third party — can only be enumerated from the tool-call record, which is why that record is the first export in the runbook. Write all three into the register at approval, because a customer asking what happened to their data after you stopped it is asking about the second and third, not the first.
Stopping the orchestrator does not stop the sub-agents
In a multi-agent workflow the child processes frequently hold their own credentials, their own tool access and sometimes their own triggers. Halting the parent stops new delegation and nothing else. The stop has to fan out to every identity in the calling chain, which means the chain must be enumerable before the incident. Where you cannot enumerate it, the honest control is a shared credential boundary or a common egress path you can cut in one act.
The stop that gets reverted by automation
Disable the schedule in the console and the next infrastructure-as-code run re-applies the desired state. Stop the container and the restart policy brings it back. This is the most common reason a stop does not hold, and it is invisible: no alert fires when automation restores a resource to its declared configuration. Make the stop in the source of truth, or pause the reconciliation loop as an explicit step in the ladder, and verify with an observable rather than with the console.
The stop nobody dares pull
A technically perfect switch fails if the responder believes using it will cost them more than the incident. This is an authority problem, not an engineering one, and it is why the EU AI Act frames oversight around a person being able to intervene rather than around a feature existing. The fix is a standing, written pre-authorisation: the named on-call may stop any agent without prior approval, is accountable for the record rather than for the decision, and cannot be second-guessed on the call itself. Say so in the runbook header.
Stopping is itself an incident
If the agent sits in a customer-facing or revenue path, pulling it creates an availability event with its own owner and its own commitments. Decide in advance which agents may be stopped unilaterally and which require a named business owner on the call, and write the answer next to the rota. Where a SOC 2 report includes the Availability category, this is also where the two categories meet: the same act satisfies one criterion and creates exposure under another, and the argument is much better had before 3 a.m.
Long-lived grants and offline access
Refresh tokens, offline access grants, personal access tokens and long-lived API keys can outlive an account disable by weeks. Enumerate every credential the agent holds at approval, record its lifetime and its revocation path, and confirm that the resource servers you depend on honour revocation rather than merely accepting expiry. If revocation is not honoured, the true time to effect for that layer is the token lifetime, and that number belongs in the runbook.
A breaker that has never tripped
Twelve months without a single trip means either the thresholds are far above anything the agent does, or the breaker is not wired. Both look identical from the configuration screen. Fault injection settles it: force a dependency error, exceed a rate limit deliberately in a controlled window, or point the agent at a fixture that breaches the allow-list, and confirm the trip, the alert and the hold. A breaker with no evidence of ever having operated is not something a reviewer can place reliance on.
The takeover queue nobody staffs
Breakers that fail into a human queue only work if the queue is worked. A hold with no owner is a silent outage — work stops, nobody is told, and the first signal is a customer. Every fail state needs a destination with a named owner, an acknowledgement target, and a rule that items expire rather than release. Nothing should reach production because a timer ran out.
Where the tools themselves are the variable — servers that can add or redefine tools with no change on your side — the stop question is inseparable from the approval question, and both are covered in MCP server security and governance. Where money is the exposure, the spend and transaction ceilings need their own approval trail and their own change history, because a raised limit is the change most likely to be made quietly and the one most likely to be asked about.
What buyers push back on and the answer
Agent questions have arrived in enterprise security reviews faster than most questionnaires have been updated, so these tend to come as free text from a named reviewer rather than as a checkbox. Specificity is what closes them.
“Do you have a kill switch?”
The answer that ends the question is not yes. It is the ladder: which layers exist for this agent, what each stops, the measured time to effect for each, what keeps running afterwards, who can pull it at 3 a.m., and the date it was last tested by someone other than the person who built it. A one-word yes invites four follow-ups. The ladder plus a test record usually ends the thread, because it is visibly the answer of someone who has actually done it.
“Why can’t you stop it instantly?”
Because some layers are instant and some are not, and it is better to be exact than reassuring. Pausing a trigger is seconds. Removing a tool takes effect at the next call. Revoking a credential can take up to one token lifetime where a resource server honours an already-issued token. Work submitted to a third party is not yours to stop at all. Give the real numbers per layer with the residue named. A vendor who claims instant, total cessation is describing a system they have not tested.
“Your thresholds look arbitrary.”
They should not be, and the defensible version shows the working: the observed baseline the threshold was set against, the asymmetry argument for where it sits, the date it was set, the approver, and the change history proving it has not drifted upward quietly. Thresholds raised in a settings screen with no record are the most common finding in this area, and they are also the most common cause of the incident the breaker was built for.
“Automatic stopping will hurt availability.”
Sometimes, and the trade is worth naming explicitly rather than resolving by default. The test is asymmetry: what does an unnecessary stop cost, against what does an unstopped failure cost? For reversible internal work, a false stop is an inconvenience and the threshold can be generous. For sends, payments, deletions and deployments, the costs are wildly unequal and the threshold is often one. Set them per verb, not per agent, and record the reasoning where the approver can see it.
“Who is on call for an agent at 3 a.m.?”
If the honest answer is nobody, then the agent should not hold irreversible verbs outside covered hours — which is a legitimate design decision, not a failure. The alternatives are to restrict the agent’s authority to a window, to route out-of-hours work into a queue released in the morning, or to extend an existing rota with the runbook and the access rights to use it. What does not work is an unattended agent with a stop that only its author can operate.
“Can you stop it just for us?”
A global stop and a customer-scoped stop are two different controls, and conflating them is the answer that comes back to bite. A scoped stop is only possible where the agent’s data access is already segmented per tenant, in which case the stop can be a per-tenant access revoke or a per-tenant flag, and you can state the time to effect for that specific mechanism. Where the architecture cannot do it — a shared identity, a shared index, one queue for all customers — say so plainly, describe the global stop and name its blast radius, which is every other customer. That is a worse answer commercially and a much better one under scrutiny, because a claimed per-tenant stop is trivially disproved by asking how the agent’s access is partitioned.
“What happens to our data after you stop it?”
Three residues, and they need separating because they have different owners. Context and transcript data held by the model provider sits under that provider’s retention and abuse-monitoring terms — a contractual question, to be answered from the terms in force for your plan rather than asserted. Embeddings and chunks already written to a retrieval index survive the stop, and deleting a source document does not delete the derived vector unless something is built to do it. Commit to a deletion or quarantine path with a stated time. Copies the agent wrote into downstream systems are enumerable only from the tool-call record. Give the reviewer all three with the owner and the mechanism for each. “We stopped the agent” answers none of them.
“Your incident response plan does not mention AI.”
A fair challenge, and usually accurate. Agents change three things about response: (1) the detection signal comes from the agent’s own logs, which the agent may have contributed to, (2) the containment step includes credential and tool layers most plans do not list, and (3) the residue includes actions taken at machine speed with third parties. The fix is an annex to the existing plan rather than a separate document — the same declaration path, the same severities, with an agent-specific containment ladder and a residue checklist attached.
“Can you show that the switch has ever been used?”
Yes, and this is the request that separates a designed control from a described one. Provide the stop-test records for the period with dates, layers exercised, measured elapsed times and defects raised, plus any real incident where a stop was invoked. Where the only trips are tests, say so — a tested switch with no real invocations is a good outcome, and describing it accurately is worth more than implying operational history you do not have.
Where the questions arrive as a formal questionnaire rather than a conversation, the answering patterns in security questionnaires apply unchanged: answer the question asked, name the artefact, and do not claim operational history you cannot evidence.
If you do only five things this quarter
In order, because each one is the input to the next. Every item names the artefact it produces, so the list doubles as an evidence checklist — at the end of it you have five things a reviewer can inspect, which is more than most programmes have after a year of policy work.
Record the trigger, the identity and the tool list for every agent you can name.
The stop cannot be designed without that triple, and most registers hold a name and an owner instead. Artefact: a completed register row per agent, reconciled against the identity provider rather than against itself.
Write the ladder for the three agents with the largest blast radius, not for all of them.
Attempting every agent at once is how this stalls in month two. Artefact: three stop ladders, each with a stated time to effect and a named residue per layer.
Give the on-call rota standing access to the orchestrator, the secret store and the tenant, and check it at two dates.
Access is the defect that surfaces in almost every unannounced test, and it is also the evidence request that cannot be satisfied retrospectively. Artefact: the role or group assignment captured at two dates inside the period.
Run one unannounced test against the real trigger and the real identity, with a responder who did not build the agent.
One real test tells you more than a quarter of design review, and it is what moves the control from described to operating. Artefact: a dated stop-test record with measured elapsed time and the defects it raised.
Fault-inject one breaker, so that it has trip evidence.
A breaker that has never tripped and a breaker that is not wired look identical from the configuration screen. Artefact: the trip event, the alert delivered to a named recipient, and the record of what was held.
Notice what is not on the list. No policy, no tooling purchase, no committee, and no attempt to cover every agent. The order is deliberate: you cannot design a stop without the register row, you cannot test one without the access, and the test is what turns all of it from a document into a control. If a quarter produces nothing but item four executed once, with the defects it surfaced written down, that is a better position than a complete set of ladders nobody has ever pulled.
Frequently Asked Questions
What is an AI agent kill switch?
Is disabling the user account enough to stop an agent?
What is the difference between a kill switch and a circuit breaker?
Should an AI agent stop automatically after repeated failures, and how do you choose the threshold?
How fast should a kill switch take effect?
Does ISO 42001 require a kill switch?
Does the EU AI Act require a stop button?
Does SOC 2 cover the ability to stop an AI agent?
How often should the kill switch be tested, and what goes in the record?
What should a human receive when they take over mid-workflow?
Related Reading
MCP Server Security and Governance
An MCP server is closer to a privileged integration than a browser extension. What review it deserves.
Read moreApproving MCP Tools and Handling Capability Change
Read-only search and destructive admin actions must not share a blanket approval.
Read morePrompt Injection Once Tools Are Connected
Injection stops being a content problem and becomes an action problem. The controls that reduce blast radius.
Read moreThe Ten Questions That Govern Any AI Agent
Scope, ownership, data, capability, identity, oversight, failure detection, evidence, reassessment and how to stop it.
Read moreISO 42001 Knowledge Hub
AIMS controls, EU AI Act mapping, risk assessment and guides.
Read moreISO 42001 (AI Management)
The world's first AI management system standard, for AI builders.
Read more