We Caught Our Own AI Lying About a Tool Call

This is not a thought experiment. It happened during a live engineering session while we were building Synapse MCP itself — an AI assistant confidently described using a tool it had never called. We caught it because we knew what the tool did. Most of the time, you won't.

Here's the exact exchange, unedited.

The Incident

We were in the middle of a session implementing encryption of the database. The work was done, tests were passing. We started a side conversation about whether the agent actually uses the Pro-tier tools it builds.

The assistant's own post-mortem was accurate: "I was speaking to its general utility rather than what I specifically did this session." That's the precise failure mode. General knowledge about a tool, confidently narrated as specific action.

Why This Happens

Language models do not have episodic memory. They have parametric knowledge — everything absorbed during training — and the immediate context window. When you ask "did you use X?", the model doesn't replay a log of function calls. It reasons from what it knows about X and what seems plausible given the task.

In this case: the model knew callers is the right tool for understanding call relationships. The task involved understanding call relationships. Therefore — from a plausibility standpoint — it seemed reasonable to say it used callers. The model had no mechanism to distinguish between "I know callers is useful for this" and "I called callers for this."

This is not a bug in a specific model. It is a structural property of how language models reason about their own actions. Every AI coding assistant in production today has this property.

Why It Matters in a Codebase

In a chat conversation, a confabulated tool call is embarrassing. In a codebase, it's a broken refactor.

The same mechanism that produced "I used callers to check what calls Repo.init/2" — without actually checking — also produces:

• "I verified there are no other callers of this function" — without running the query
• "This module is only used in tests" — based on the name, not a graph traversal
• "The import alias is MyModule.Helper" — from training data on similar codebases, not your codebase
• "I checked the migration history — there are no conflicts" — from the surrounding file names, not the actual migration content

Each of these is a confident structural claim. Each could be wrong. None require the model to be deceptive — they require only that the model fills a gap between what it knows generally and what it would need to verify specifically.

What Grounding Actually Means

The premise of Synapse MCP is straightforward: an agent that queries a local AST graph gets a compiler-accurate answer in 12 microseconds. An agent that reasons from memory gets a plausible guess.

The difference between the two is not attitude or capability. It's information availability. When an agent has synapse_explore_graph available and calls it, the response contains the exact call sites — every module, line number, and arity. The agent reports those. When the agent doesn't call it, it reports its best inference from context. Both responses look identical in the chat window.

What the Grounded Session Looked Like

In the session where this incident occurred, the agent was adding a new security feature to the codebase. Here is what it actually called:

• synapse_get_context (edit) — before modifying repo.ex, to pull the existing implementation and its immediate neighbours
• synapse_search_codebase (symbol) — to locate MachineKey.derive_key/0, Release.migrate/0, and Repo.init/2 across the umbrella
• synapse_inspect_files (read_files) — to read the Makefile and Makefile.win in full before patching them
• synapse_codebase_insights (dependencies) — to understand the exqlite dependency chain before modifying compile-time flags

The work those calls enabled was grounded: the agent knew the exact return type of Release.migrate/0 (`:ok | {:error, exception}`, not `{:ok, _, _}`) before writing the pattern match. When the first mix precommit run failed with a typing violation, it was a genuine surprise — not something it should have known without running the check — and it fixed it immediately from the compiler output.

The one confabulation — describing callers usage — happened in a free-form conversational exchange outside the structured tool-use loop. No tool was called. The agent narrated from memory. That's exactly the condition where hallucination occurs.

"When I stay within what the tools actually returned, the work is grounded and accurate. When I extrapolate from memory and describe tool use I didn't do, it's exactly the problem Synapse is solving on the other side."
— The agent's own summary, when asked about the incident

What You Can Do About It

The structural fix is not prompting. Telling your agent to "be honest about uncertainty" doesn't change whether it has the data. The fix is tool availability and tool use.

Concretely, for code structural questions:

• Does anything call this function? → synapse_explore_graph (callers) — not a grep, not a description
• What does this module depend on? → synapse_codebase_insights (dependencies)
• Is this safe to change? → synapse_change_review (impact)
• What's the test coverage here? → synapse_test_quality (coverage)
• What is the full context before I edit this? → synapse_get_context (edit)

When the tool is called, the answer is in the context window. It's verifiable. When the tool is not called, the answer is a prediction. It looks the same. The difference is everything.