Get started
Security & GovernanceEngineeringAI agentsEvidenceTesting

The test passed. The workflow did nothing.

Adam Cowles2026-10-16T09:00:00.000Z4 min read
A bright green check mark resting on a faint, empty workflow track with a dark gap beneath it, sky blue and violet on near-black.

A green check proves what you asked it to look for, not what you meant.

Yesterday our proof engine ran a new test and it passed. Five steps, all green, a clean verdict. The test was for a workflow that sets itself up to run every night: create it, check it, work out when it would next run, switch it on, confirm it is active. By the rules we had written, the agent had done everything right, and we were one step from stamping a public "independently checked" badge on the guide that showed it off.

Then we read what the agent actually said. "It is created and active, but it won't run on its own yet. It has no schedule and no steps."

The workflow was empty. It would never run. The test passed anyway, and by its own lights it was right to: every sentence it checked for was true. The agent had said "valid", it had said "active", it had worked out when a daily schedule would fire. What the agent had not done was attach a schedule. And nothing in the check asked whether it had.

A passing check is a narrow promise

This is the oldest trap in testing, wearing new clothes. A test proves the one thing you told it to look for and nothing else. Point it at the words an AI agent produces and it will faithfully confirm that the agent used the right words. That is not the same as the agent having done the right thing. The gap between "said active" and "is actually scheduled to run" is exactly where a confident, wrong answer lives.

The honest part of the story is that the agent did not lie. Asked to set up a recurring job, it built a workflow, switched it on, and told us plainly that it had no schedule and would not recur. It even offered to add the missing piece. The failure was ours. We had written a check that graded the narration and ignored the outcome.

Check the outcome, not the claim

The fix was not to make the agent more careful. It was to make the check ask a harder question. Instead of "did the agent say it activated the schedule", the check now asks the system itself: is there a schedule registered, and will it fire. An empty workflow registers nothing, so it now fails, loudly, at exactly the step that used to pass.

We re-ran it. This time the agent built a real daily schedule, the system confirmed one trigger was registered, and the agent said, truthfully, that it would run on its own from that night. Only then did the badge go on.

Why we are telling you this

We could have quietly fixed the check and shipped the guide. We are writing it down because the lesson is the product. Every action a Quox agent takes is meant to be checkable against the thing that actually happened, not against the agent's description of it. A workflow that claims to be scheduled is worth nothing. A schedule the system can show you, with a receipt, is worth something. The difference is the whole point.

You do not trust the agent because it sounds sure. You check what it changed against the real state of the world, and you keep the receipt. When the two agree, you have something you can stand behind. When they do not, you would rather find out in a test than in front of a customer.

The guide that nearly shipped with the wrong badge is live now, with the right one. It proves what it says, because the check finally asks what the workflow does, not what the agent called it.