Home  ›  Field notes  ›  AI agents

How we stop an AI agent from making things up

The fear that stops most businesses trying an AI agent is a simple one: what if it just makes something up? It is a fair fear, and it has a real answer. Here is how we design honesty in.

By Suman Banerjee Published 29 Sep 2026 ~6 min read
The short answer

You stop an agent making things up by never letting it guess. Ground it in your own data so answers come from a real source, constrain what it is allowed to say and do, make it admit when it is unsure and hand off to a person, and watch its behaviour so anything odd is caught early. Honesty is a design choice, not a hope.

Why models make things up

A language model does not look up facts. It predicts plausible text, one piece at a time, based on patterns. That is why it writes so fluently, and also why, asked something it has no real basis for, it will produce a confident, well formed answer that happens to be wrong. It is not lying. It is filling a gap the only way it knows how.

Understanding this is the key: the model is not unreliable by accident, it is a predictor. The job of the design is to stop it ever having to predict what it should know.

In shortModels predict plausible text, not facts. Asked what they cannot know, they confidently fill the gap.

Grounding it in your data

The core fix is grounding. Instead of letting the agent answer from memory, we have it retrieve the relevant facts from your own trusted sources first, and answer only from those. The question stops being what does the model think and becomes what do your documents actually say. Answers come with a basis, and can point to where they came from.

A grounded agent is not guessing your policy or your prices. It is reading them, the same source your team would.

In shortGrounding makes the agent answer from your real sources, not its memory. Facts come with a basis.

Guardrails on what it can say and do

Grounding is backed by boundaries. We constrain what the agent is allowed to talk about and what it is allowed to do, so it stays inside the job it was built for. It does not improvise on topics outside its scope, and it does not take a consequential action unless it is confident and permitted. The narrower and clearer the remit, the less room there is to go wrong.

A good agent is defined as much by what it refuses to attempt as by what it does well.

In shortBoundaries on what it can say and do keep the agent inside its job, with no room to improvise.

Admit, hand off, and monitor

Finally, the agent is built to know its limits. When it is not confident, it says so and hands off to a person with the full context, rather than inventing an answer to seem helpful. And we watch its behaviour before it launches and after, so any odd answer is caught and corrected early instead of reaching more customers.

Between grounding, guardrails, honest handoff and monitoring, making things up stops being a risk you hope against and becomes one you have designed out.

In shortIt admits uncertainty, hands off, and is monitored. Honesty becomes a designed property, not a hope.

Common questions

Can you guarantee it never makes anything up?

No honest engineer promises never. What we do is design it out: grounding in your data, tight guardrails, honest handoff when unsure, and monitoring. The result is an agent you can actually trust in front of customers.

What does grounding actually mean?

It means the agent retrieves the real facts from your trusted sources and answers from those, instead of from the model's memory. Answers have a basis and can point to where they came from.

What happens when it is not sure?

It says so and hands off to a person with the full context, rather than inventing something to appear helpful. Knowing when to stop is part of how it is built.

How do I know it is behaving?

You see exactly how it behaves before it goes live, and we monitor it afterwards, so unusual answers surface to us early rather than to a stream of customers.

Want an agent you can actually trust?

Tell us the job and we will design the grounding and guardrails that keep it honest in front of your customers.