Here is a night, and please treat it as made up. A finance team has an AI agent that pays supplier invoices. At 2:07 a.m. it starts paying the same invoice twice. The analyst on call, whom I will call Ana, sees it happen on the dashboard. There is a pause button, clearly labelled, a few centimetres from her cursor. She does not press it. The CFO approved the agent and is asleep, the vendor’s support line opens at nine, and nobody has ever told Ana whether pausing is her decision or a career event.
Eleven minutes later the agent has paid four more invoices, and Ana is composing a message to someone who will not read it until morning. The button worked the whole time. What was missing was permission.
Notice that even a perfect pause would have left a mess. Pause, stop and undo are three different things, and companies tend to plan for one of them. Pause freezes new actions. Stop cuts the agent’s access so it cannot restart on its own. Undo reverses what already happened: the payments recalled, the records restored, the apology sent. Undo has an owner outside the agent, such as the bank or the customer, and it takes longer than any button. Ana’s pause would have saved the next four invoices and done nothing for the first two.
The European Union noticed exactly this problem and, in the AI Act, picked a telling word for it. Article 26 says a company using a high-risk AI system must assign human oversight to people who have “the necessary competence, training and authority, as well as the necessary support”. Competence and training are what companies usually buy, in the form of a workshop. Authority is the one that costs something, because it means somebody senior has decided in advance that a junior person may stop the machine without asking.
The Act wants the button too. Article 14 requires that high-risk systems are provided so that the people overseeing them can intervene or interrupt the system through a “stop” button or a similar procedure that lets it come to a halt in a safe state. Article 26 then adds duties for the company using the system, which the Act calls the deployer. It must keep the logs the system generates automatically, to the extent they are under its control, for at least six months. And if it has reason to think that using the system as instructed may present a risk, it must inform the provider or distributor and the market surveillance authority, and suspend use.
So far the law sounds like a fix for Ana. Now read the small print on scope, because this is where the compliance slide decks usually get quiet. Those duties apply to high-risk systems, which are listed in Annex III: areas such as biometrics, education, employment and the credit scoring of individuals. They apply from 2 December 2027, after the AI Omnibus moved the date from 2 August 2026. An agent that pays supplier invoices appears on none of those lines, as I read the list, and I would want counsel to confirm that before anyone relies on it. An agent that screens job applications does appear on it. And Article 25 has a sting: a company that changes the intended purpose of an AI system so that it becomes high-risk is treated as its provider. A chatty assistant bought for customer emails and then pointed at CVs has quietly changed its owner’s legal job.
For the invoice agent, then, the Act may have nothing to say about who presses stop. Liability law does not step in either, at least not for the company. The new Product Liability Directive, (EU) 2024/2853, treats software, including AI systems, as a product, which is a real change from the 1985 rules. It applies to products placed on the market after 8 December 2026; the date originally read 9 December, and a corrigendum published in May moved it by a day. But it compensates natural persons, for death, personal injury, damage to property and the loss of data, and it excludes property used exclusively for professional purposes and data used for professional purposes. A business that pays an invoice twice does not appear among the listed types of damage, as I read Article 6. The money is a matter for whatever your contract says.
One more thing in that directive is worth taking away, though. According to the EUR-Lex summary, a manufacturer is liable for defects that appear after updates or upgrades under its control, and through continuous learning where the AI system stays under its control. The logic follows control. Whoever can change the system, or stop it, is the party the law tends to look at first. For most agents, then, the stop button is a private arrangement. Regulators will write rules for the high-risk corner; everything else is governed by your own habits and your own signatures. Which brings the question back to your own building: who, among your people, can stop your agent, and who among your vendor’s people can change it?
I should say what I am not claiming. I have no survey of what AI contracts typically say, and I will not pretend to describe the market. What I can say is where the questions live: who on our side may pause the agent without opening a ticket, how fast the vendor answers an emergency, which logs we can export and for how long, how much notice we get before the model behind the agent changes, and what the liability cap is, set against the worst night the agent could have. Those belong in writing, before signing and again at every renewal.
The answer to Ana’s hesitation does not belong to the contract, and it costs nothing but a decision. In my experience the tool tends to arrive before the decision rights do, and I offer that as a view, not as a statistic.
Why do companies skip it? Because delegating authority means delegating risk. A senior person who writes “anyone on duty may stop this” has accepted in advance that some stops will be false alarms, and that a false alarm is cheaper than a real one. Leaders who accept that trade have an easier 2 a.m. Leaders who leave it open get Ana, composing messages to people who are asleep. It is a judgment about people, and no vendor can sell it to you. Even where the AI Act does not apply, its six-month log rule is a decent floor to borrow: the first question after any bad night is what exactly the agent did, and an answer that begins “we will ask the vendor” is the beginning of the second bad night.
So the most useful sentence in this whole affair is not in the AI Act or in any contract. It is the one a senior person writes into the runbook next to the agent: the person on duty may pause this agent at any time, without approval, and will not be asked to justify a pause made in good faith. A stop button that needs a signature is a decoration. The sentence takes a minute to write and some leaders a very long time to sign, because signing it is the moment the risk stops being theoretical and starts carrying their name.
Somewhere tonight an agent will do something nobody intended, and someone like Ana will be looking at the button. Whether her hand moves was settled weeks earlier, in daylight, by somebody who was never on call.
If this was useful, subscribe at vanguards.media/subscribe and send it to the person who signs your vendor contracts.
David
Sources: Regulation (EU) 2024/1689 (AI Act), Articles 14, 25 and 26 and Annex III, as amended by Regulation (EU) 2026/1744 (Digital Omnibus on AI); Directive (EU) 2024/2853 on liability for defective products and its corrigendum of 7 May 2026; EUR-Lex summary on defective-products liability. Legal references checked on EUR-Lex and the Commission’s AI Act Service Desk on 18 Oct 2026; where a point is my reading, I say so.