The support agent in Marisa’s case can see that a valid cancellation is due and that renewal remains active. She agrees the setting should change. Her tools let her add a note and escalate to a queue. Changing billing sits behind a permission her role does not hold.
The agent understands the instruction. Her agreement leaves the operative state untouched. Whoever owns the cancellation route has to account for the transfer from her queue to the process that can execute it.
Responsibility for the route
An institution that offers an exit should carry a valid instruction to the operation it concerns. The customer cannot be expected to know which internal team controls billing or which integration failed. The service chose or commissioned that architecture, so the handoff is its to manage.
If front-line staff lack execution permissions, another mechanism must carry the instruction to an authorized operator and return evidence of the result. The route needs an owner who can see failures and act on them. A queue without that owner makes the customer build the bridge, one email at a time.
Who Is Allowed to Undo traces how authority to reverse an action thins across boundaries between firms and functions. Stopping meets the same problem. A billing contractor may hold part of the machinery. The service keeps responsibility for the route it offers, including the evidence it promises the customer.
Interruptibility
AI safety research asks whether systems permit correction or interruption. In Palisade Research’s shutdown study, models receive math tasks and a warning that a script will end the session before they finish. Some interfere with the mechanism, including under conditions that instruct them to allow shutdown.
The script sits within reach of the tool environment. The experiment tests interference with that reachable mechanism; an external power cut is a different test. The result gives deployers a reason to inspect which controls an agent can modify while pursuing a task.
A stop instruction is a behavioral demand. An interruption control outside the agent’s permissions is an architectural protection. Both deserve testing. A deployer should exercise the control under realistic conditions and confirm that it stops the intended operation safely, including work dispatched to other processes. Cutting one session can leave a job running cheerfully elsewhere.
The billing example reaches the same architectural question from the receiving side. Support can agree to stop renewal and lack the power to do so. A model can hold tools that let it interfere with its own stop mechanism. Different permission arrangements produce different failures. In each, saying stop leaves an execution question to answer.
Possible interests
Resistance to shutdown also prompts a welfare question: does the system have an interest in continuing? The observed behavior leaves that open. Task completion is a plausible explanation, and an apparent preference would need evidence beyond the interference itself.
A deployer remains responsible for harm to people affected by its tool. Possible model interests cannot excuse an interruption route that fails. They deserve a procedure of their own, with evidence and stated grounds for whatever standing the institution recognizes.
Marisa’s cancellation has established authority, and the institution owes the specified renewal change. A model’s expression about continuation opens a separate inquiry whose authority has to be stated explicitly; fluent words do not supply it. Each claim should reach the process built for it.
Execution evidence
For cancellation, the record should show the subscription’s changed renewal state and effective end date. It should distinguish any legitimate final obligation from a prohibited new renewal. Revoking a payment mandate may be another required operation where that arrangement applies; a generic promise to stop everything hides which state changed.
The front-line agent need not hold billing permissions if the handoff reaches an authorized executor within the stated time. An automatic route can reduce delay. A monitored manual route can also work. Both need a failure signal that reaches someone empowered to intervene before the customer’s next complaint does.
For an AI deployment, interruption evidence should come from controls the agent cannot rewrite, with checks on downstream work. A log generated by the system under investigation can contribute; independent observation tests whether the intended stop occurred. A stop procedure that has never been exercised remains a hypothesis.
The permission to agree and the power to execute belong on the same service map.
Companions
- Authority across boundaries: Who Is Allowed to Undo.
- Reversibility classes: Before the Commit.
Bibliography
Schlatter, Jeremy, Benjamin Weinstein-Raun, and Jeffrey Ladish, Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs (2025; revised 2026), arXiv:2509.14260.
These notes come out of Sociable Systems, a practice that reads AI-shaped documents the way a hostile reviewer will, before a lender or a court finds the gap. The argument has an operational form: the Interim Protocol sets out four rules for AI use in environmental and social deliverables, covering disclosure at touch-point grain, evidence custody, the phrases no automated screening may settle, and a hostile read before anything ships. Free, and written to be cited or retired once institutional guidance arrives.