PlantOps BI

Don’t Accept ‘Done’ from an Agent Without a Quoted Source

When an agent tells me a ticket is done, I don’t take its word for it anymore. I want to see the evidence in the same message.

What Can Go Wrong

A “done” with nothing behind it is a summary of what the agent believes happened. It may have read the wrong file, checked the wrong workspace, or inferred completion from a status field. Nothing in the verdict lets me tell those cases from a real finish.

The same thing shows up with status fields. A task marked done is not proof the work happened. An email marked sent is not proof it went out.

The Rule

Every “done” has to carry two things: a quote from the source, and where the quote came from. A refresh finished, so show the log line and the job it came from. A measure was fixed, so show the definition as it now stands in the model.

If the agent cannot produce both, the item is not done. It goes back in the queue as “unverified.”

Checking the Checker

For work where being wrong is expensive, I run review passes that are read-only and have to cite evidence:

  • A skeptic goes through every claim and marks it sourced or unsourced.
  • An adversary reproduces the answer blind, reaching a result on its own before it sees mine, and then goes after scope, filters, grain, double counting and stale sources.
  • A premortem assumes the build failed a month later and works out why.

How many of these I run depends on the stakes. A cheap lookup gets none. Anything hard to undo gets all three.

For BI Work Specifically

BI is a good place for this rule because so much can look finished and be wrong. A refresh can complete against stale credentials. A visual can render with the wrong filter context. A model can deploy and still return blank. “It deployed” is a claim, and the quote is the query result.

Related: a mistake in an agent’s verdict is cheap to catch at the moment it is made, and expensive to find three weeks later, after you have already used the number.