An agent needs a boundary

When somebody calls a system autonomous, the first thing I want to know is which decisions it’s allowed to make. Can it choose the next step? Change the goal? Spend money? Contact another person? Keep running when nobody’s watching?

With GrasShopper this is easy, because the agents only do research. They search, go through reviews, compare prices and write a guide. The buying stays with you.

But searching catalogs, comparing specs, choosing a seller, paying and arranging delivery are different acts with different risks, and they involve different people. A system that’s great at the first two hasn’t earned authority over the rest.

Anthropic’s guide Building effective agents from December 2024 separates workflows, where the path is predefined, from agents, which direct their own process and tool use. It’s a useful engineering distinction and that’s all it is. It says nothing about whether the thing has a right to act on somebody’s behalf.

What we’re really doing here is delegation, which people have always done: one person sets the purpose and another carries out the steps. Software changes the speed and the scale, and it can make the delegation harder to see. The permission sits in some integration setting while the interface talks to you like one helpful character who handles everything.

I’m not saying a human should approve every step. Too many confirmations make the system useless, and people start clicking “approve” without reading. The boundaries should follow real consequences. A research assistant can explore freely within a budget and need a separate decision before it commits to anything outside. A maintenance tool can propose changes and run reversible checks while deployment stays with a named person.

You could object with Sutton’s The Bitter Lesson, which argues against building our own ideas about how to solve a problem into AI systems. A boundary could look like the same mistake, but prescribing a solution and authorizing an action are different things. An assistant can find a replacement part in a way I’d never think of and still be unable to spend more than the agreed budget. (Sutton’s essay has no theory of permissions, this is me applying it to system design.)

Responsibility splits the same way. The user brings the intention, the developer provides capabilities and defaults, the organization decides where the system runs. “The agent did it” explains none of those decisions. NIST’s AI RMF is a good reminder that governance has to come together with the technical measurement, and my own rule is that whoever could understand, authorize, prevent or repair an action should be answerable for it.

The stop button deserves more attention than it gets. It’s weak if nobody knows the system is acting, if stopping leaves commitments half-done, or if the person who carries the consequences can’t reach it. (In the post about maintenance as creative work I described an assistant that nobody had the authority to switch off.)

I like agents because they can carry an intention through complicated work for me. I want that help to leave me with more room to act, and clear boundaries are how I’d get there.