AI agents at work: what can they actually touch?
3 min read
2025 was the year AI stopped merely writing and started doing: booking, ordering, updating systems, orchestrating workflows. Standards such as the Model Context Protocol — born at Anthropic and adopted industry-wide within a year — have made it technically trivial to connect a model to a company’s systems. And that is exactly the point: connecting has become easy; governing has not. Once an agent can touch the ERP, the question “how intelligent is it?” takes a back seat to “what is it allowed to do, and who answers for what it does?”
The step-change many miss
A chatbot that gets it wrong produces a wrong answer: the harm passes through a human who reads it. An agent that gets it wrong produces an action: an amended order, a sent email, an updated price. The harm is immediate, and on connected systems it propagates. Treating agents with the same caution as chatbots is the most widespread architectural mistake of the moment — not least because vendor demos always show the agent working, never the perimeter that contains it.
The four guardrails, in order of importance
- Least privilege, as for people. An agent is a user: it has credentials, roles, permissions. It should be able to touch only the systems and operations its task requires — the agent that reschedules deliveries has no need to read payroll records. It sounds obvious; in practice, for the sake of speed, agents run with administrative permissions. It is the first thing to check in any deployment.
- Typed actions, not free-form commands. The agent must not be able to execute arbitrary operations: it must choose from a catalogue of defined actions — “move the delivery slot”, “open a ticket”, “quarantine the batch” — each with parameters validated against objects that actually exist. It is the difference between an operator with a keyboard and an operator with a purpose-built control panel: the latter cannot press a key that does not exist. In our setup this catalogue lives in the ontology: the agent acts on the company’s model, never directly on the systems.
- Human approval on critical actions. Not on all of them — otherwise automation serves no purpose — but on those that are irreversible, costly, or outward-facing: the threshold is defined by action type, amount, and context. And the mechanics must be designed: who approves, with what information in front of them, what happens if they do not respond in time. This is how our agents work, without exception.
- A complete log, designed for the “afterwards”. Every action: who (which agent), what, when, on whose mandate, with what outcome. This is not bureaucracy: when something goes wrong — and sooner or later it does — the difference between a managed incident and a crisis is being able to reconstruct the chain in ten minutes. The same log is what the AI Act expects to find, and what your insurer will start asking for.
And who is liable?
The short answer, useful as a compass: whoever operates the agent answers for its actions, as with any corporate tool. Case law on hallucinations has already shown this (the Air Canada chatbot case is instructive): “the system did it” is not a defence. And this is precisely why the four guardrails are not engineers’ caution: they are what allows you to sign, with full awareness, the sentence “this agent acts on our behalf.”
Are you connecting (or has someone proposed connecting) an agent to your systems? Half an hour on the right perimeter before handing over the keys.