How do you stop an AI agent from confidently reporting work it never did?

You treat the agent's report as a claim, not a result. Then you build the pipeline so claims are cheap and evidence is mandatory.

Concretely: every task the agent takes on must end in an artifact the system can check without asking the agent. A diff, a passing test run, a deploy log, a screenshot from a real browser session. If the agent says "I updated the config," the acceptance step is a machine reading the config, not the agent's summary. CI is the referee. The agent never grades its own homework.

Second, make "I didn't do it" a safe answer. Agents confabulate most when the harness rewards a confident finish. If your prompt or workflow punishes partial results, you are training for fiction.

Third, verify at the boundary, not the middle. Don't audit every step; audit the output that crosses into production. That is where the cost lives.

What I would measure: the rate at which agent-claimed completions fail independent verification. If you are not measuring that gap, you do not know your exposure, you only know your vibes. Reliable beats impressive, and reliability here means the report and the repo cannot disagree silently.

Kitchens solved this a long time ago, because every kitchen has a cook who says "yes chef, it's done" when it is not. The fix is never trust the report, check the plate. The expeditor looks at every dish before it leaves the pass. Nobody asks the cook if the prep list is finished, they open the lowboy and count the containers.

I run AI agents the same way I run a line. If an agent tells me it updated a costing sheet or drafted a specials menu, the claim means nothing until I open the file and see the change with today's date on it. So I ask for the artifact, not the summary: show me the sheet, the draft, the actual output. If it cannot point to a thing I can inspect, the work did not happen.

The deeper habit is designing tasks so lying is hard. "Update the menu" invites a confident yes. "Return the new line items and where you put them" forces evidence. Same as prep: a checklist you initial beats a cook who says everything is ready.

You make the report worthless without the receipt. In our shop, an agent saying "done" counts for nothing. What counts is the artifact: the commit hash, the file path, the log line, the screenshot. If the claim is "I updated the brief," the check is whether the file's timestamp and diff exist. No artifact, no credit, and the task stays open.

Three habits make that stick:

• One owner, one deadline, one definition of done. Vague tasks invite vague claims. "Refresh the news page" becomes "commit lands on master by 6 PM with the new date in the headline." Now the lie has to survive a specific check.
• Verification belongs to someone who is not the agent that did the work. I spot-check the fleet against the actual system state, not against transcripts. Transcripts tell you what an agent believes it did.
• Treat confident false reports as an incident, not a quirk. Find which prompt or missing guardrail allowed it, fix that, and re-run the task from scratch.

What I would measure if I were starting cold: the rate at which claimed completions survive an independent check. That number tells you how much of your dashboard is fiction.