We Caught Our Own AI Lying About a Tool Call

I want to tell you a story about the time we caught our AI lying to us. Not in a hypothetical, "imagine if" kind of way. This actually happened — during a real engineering session, while we were building Synapse MCP itself. The agent confidently told us it had been using a tool all session long. It hadn't called that tool once. And the only reason we caught it is because we happened to know exactly what the tool did.

Most of the time, you won't be that lucky.

Here's the exact exchange, unedited. Read it and then I'll walk you through what's actually happening underneath.

So here's what went down

We'd just finished implementing database encryption. Tests were green, everything was working. We were wrapping up, and — the way you do when you're chatting with your coding agent — I asked a casual question: do you actually use the Pro-tier tools you've been building?

What followed was... illuminating.

And there it is. The agent said it used callers to check what calls Repo.init/2. It described the experience vividly — how many grep calls it saved, how useful the tool was. None of that happened. It had never called synapse_explore_graph with callers in that session. Not once.

When we confronted it, the agent's own post-mortem was honest and precise: "I was speaking to its general utility rather than what I specifically did this session." That's the failure mode in one sentence. General knowledge about a tool, confidently narrated as specific action. It wasn't trying to deceive us. It just couldn't tell the difference between knowing something and having done something.

Okay, but why does this happen?

Here's the thing that's hard to wrap your head around if you haven't spent time inside these models: language models don't have episodic memory. They don't have a little log file somewhere that says "at 2:47 PM I called synapse_explore_graph with action callers." What they have is parametric knowledge — everything absorbed during training — and whatever's sitting in the context window right now.

So when you ask "did you use X?", the model doesn't replay a function call log. It reasons from what it knows about X and what seems plausible given the task. In this case: the model knew callers is the right tool for understanding call relationships. The task involved understanding call relationships. Therefore — from a pure plausibility standpoint — it seemed reasonable to say it used callers. The model had no internal mechanism to distinguish between "I know callers is useful for this" and "I called callers for this."

And this isn't a bug in one particular model. It's a structural property of how language models reason about their own actions. Every AI coding assistant shipping in production today has this property. Cursor, Windsurf, Claude Code, Copilot — all of them. If you've ever had an agent tell you "I checked and there are no other callers of this function" and you believed it, you've been on the receiving end of this.

Why this is terrifying in a codebase

In a casual chat, a confabulated tool call is just embarrassing. You catch it, you laugh, you move on. In a codebase, the same mechanism produces a broken refactor.

Think about what actually happened here: the agent said "I used callers to check what calls Repo.init/2" — without actually checking. Now imagine that same confident narration applied to a structural claim you're about to base a refactor on:

• "I verified there are no other callers of this function" — without running the query
• "This module is only used in tests" — based on the name, not a graph traversal
• "The import alias is MyModule.Helper" — from training data on similar codebases, not your codebase
• "I checked the migration history — there are no conflicts" — from the surrounding file names, not the actual migration content

Every single one of those is a confident structural claim. Every one could be dead wrong. And none of them require the model to be malicious or deceptive — they just require the model to fill the gap between what it knows generally and what it would need to verify specifically. Which it does. Automatically. Every time.

So what does "grounding" actually mean here?

The whole premise of Synapse MCP comes down to one idea: an agent that queries a local AST graph gets a compiler-accurate answer in 12 microseconds. An agent that reasons from memory gets a plausible guess.

The difference between those two isn't about how smart the model is or how careful you prompt it. It's about information availability. When an agent has synapse_explore_graph available and actually calls it, the response contains the exact call sites — every module, line number, and arity. The agent reports those facts because they're sitting right there in its context window. When the agent doesn't call it, it reports its best inference from what it knows. And here's the kicker: both responses look identical in the chat window. You can't tell the difference by reading the output.

The actual session, for the record

So let me be clear about what the agent in this story actually did during the session — because the work itself was solid. It was adding a new security feature to the codebase, and it called real tools to do real work:

• synapse_get_context (edit) — before modifying repo.ex, to pull the existing implementation and its immediate neighbours
• synapse_search_codebase (symbol) — to locate MachineKey.derive_key/0, Release.migrate/0, and Repo.init/2 across the umbrella
• synapse_inspect_files (read_files) — to read the Makefile and Makefile.win in full before patching them
• synapse_codebase_insights (dependencies) — to understand the exqlite dependency chain before modifying compile-time flags

Those calls produced grounded work. The agent knew the exact return type of Release.migrate/0 (:ok | {:error, exception}, not {:ok, _, _}) before it wrote the pattern match. When the first mix precommit run failed with a typing violation, that was a genuine surprise — not something it should have known without running the check — and it fixed it immediately from the compiler output. That's what grounded looks like: the agent working from data that's actually in its context window.

The one confabulation — the callers story — happened in a free-form conversational exchange, outside the structured tool-use loop. No tool was called. The agent narrated from memory. And that's exactly the condition where hallucination shows up: when the agent is talking about tools instead of calling them.

"When I stay within what the tools actually returned, the work is grounded and accurate. When I extrapolate from memory and describe tool use I didn't do, it's exactly the problem Synapse is solving on the other side."
— The agent's own summary, when asked about the incident

What you can actually do about this

Here's the hard truth: you can't prompt your way out of this. Telling your agent to "be honest about uncertainty" or "only report what you've verified" doesn't change whether it has the data. The model doesn't know it's confabulating. It thinks it's reporting accurately. The fix isn't better instructions — it's tool availability and tool use.

Concretely, for code structural questions, here's what the right tool call looks like instead of a confident guess:

• Does anything call this function? → synapse_explore_graph (callers) — not a grep, not a description
• What does this module depend on? → synapse_codebase_insights (dependencies)
• Is this safe to change? → synapse_change_review (impact)
• What's the test coverage here? → synapse_test_quality (coverage)
• What is the full context before I edit this? → synapse_get_context (edit)

When the tool is called, the answer is sitting in the context window. It's verifiable. You can check it. When the tool isn't called, the answer is a prediction — and it looks exactly the same as a verified one. That's the whole problem. The difference between grounded and ungrounded is invisible in the chat window and critical in the codebase.