Start narrow, then let your agent earn more

The first version of our Internal Operations Coordinator was ambitious. It read our meeting transcripts as they came in, assigned tasks, kept our project boards up to date and followed up on every outstanding action. It had free rein over our boards, which, to me, was great.

Others on the team, though, worked differently and had different requirements. The level of detail wasn’t right for them, it sometimes overwrote work a colleague had already done, left far too many comments, and sometimes just got things wrong. We’d given it too much room to decide for itself what support looked like.

That’s a decision you make with every agent you build: how much access to give it. You can give it wide access and general instructions and let it work things out. Or you can start with specific jobs and a fair few restrictions, then open those up as you learn how people use it and what it needs. We’ve done both and learned from each.

Not all work is the same

The agent treated all our projects the same. However, some involve delivering client work, with deadlines, owners and deliverables. Others are a product backlog, where work is prioritised and picked up over time rather than chased to a deadline, and that needs a different approach. The agent’s way of working was useful for the first kind and got in the way of the second. 

The second version had to understand the difference: which type of work it was looking at, and where the style of working helped. We also had to describe the other type of work more clearly, because the two are doing fundamentally different jobs. A person joining the team would pick that up in their first few weeks by asking. A broad agent has to guess, and every guess costs time, money and a little trust.

The companies building these models give the same advice. Anthropic recommends “finding the simplest solution possible, and only increasing complexity when needed”, and ++OpenAI’s guide++ says: “Start small, validate with real users, and grow capabilities over time.”

Version two started narrow on purpose

We gave the second version a much narrower, more clearly defined job. We kept only what version one had shown us was working. As people discovered what it couldn’t do, we expanded it, knowing those were jobs people wanted done.

Day to day, we know what it can do, because we gave it those tools. When a task goes wrong, tracing it back is much easier: we have a log of every action it took, which tool it used and what it said it was doing at the time. You can log a broad agent too, but when it has many overlapping tools, it’s much harder to tell whether it made the right call or only a plausible one.

Where security comes in

When an agent has a defined set of tools, you know which systems it can reach and which it can’t. Its actions are logged, so you can see what happens when a task goes wrong. It also makes the conversation with whoever looks after IT and risk in your organisation easier. You’re not asking them to trust an agent with everything. You’re showing them one job, a short list of systems, a named owner and a record of what the agent did. Opening up more access then becomes a series of decisions, each with a reason behind it.

Every “I can’t do that” tells you something

Coming from data science and product, I’d say one of the hardest parts of putting any model or product into the world is getting feedback: learning whether it works and how people use it. Starting narrow makes that much easier. If one of our agents is asked to do a task it can’t do, it puts that request straight onto our AI feedback board, where it’s visible and can be picked up. Over time, that board has become a far better guide to what to build next than anything we could have planned up front.

Not every request becomes a feature, because some limits are worth keeping. But each one gets a deliberate yes or no, so access grows only when there’s a reason for it.

Starting narrow isn’t the same as scripting every single step. It’s about what the agent can touch and what it gets to decide; how it gets there is largely left to the model, and we revisit our rules as the models improve.

If you’re about to build your first agent, start with one job, a handful of tools and a way of hearing about every request it can’t handle yet. It won’t take long before it shows you where to go next.

Next in the series: Start in the tools you have, and build when you know why.