Get started
Security & GovernanceEvidenceProofTesting

How a guide earns its place on quox.ai: two days, the same checks

Before a Quox guide is marked VERIFIED, the proof engine asks Quox to do the real thing in chat and it has to pass on two separate UTC days. Last night 115 of 139 did.

Adam Cowles13 November 20265 min read
Two glass panes etched with the same glowing lattice stand one behind the other, a single cyan beam passing through both into a small bright seal, cyan and violet over near-black

A how-to guide makes a promise: do this, and Quox will do that. We would rather not ask you to take that promise on trust.

So before a Quox guide or journey is marked VERIFIED, our proof engine does the thing the guide describes. It asks Quox in chat, the way a person would, and checks the answer against what is really there. It has to pass on two different UTC days, against the same checks. And those checks can only get stricter on their own.

This post covers the rule, what happened when we ran it last night, and what it does not prove.

The rule, word for word

This is the wording on every passport in the Evidence Explorer:

A case is VERIFIED when the newest run on the current checks passed and passing runs cover two or more separate UTC days.

Two parts of that sentence do most of the work.

The newest run has to pass. An old pass does not cover for a new failure. If a case passed on Monday and failed on Wednesday, it is not verified, however good Monday looked.

Two separate days. One good run can be luck. A warm cache, a quiet system, test data that happened to line up. A fresh run on another day, asking the same thing and held to the same checks, has to find the same result. If it does, the guide has shown it works more than once. If it does not, we learn something before you do.

What day two looked like

Overnight from 12 to 13 November, the proof engine re-ran 139 cases on their second day. 115 passed and were published as evidence.

24 did not. Those went back to the teams that own them. Some were real routing gaps in the product. Some were test data that had moved since the first run. Some were environment outages. Failed runs are never published as evidence: only passes are.

That is what a second day is for. If every case passed every time, the check would not be telling us much. 24 failures out of 139 is a list of real work, found by the machine, before a reader followed a guide that did not hold.

Checks only get stricter on their own

The easiest way to get a green board is to make the questions easier. So we have a rule we call Fair Testing: a check can tighten whenever it needs to, but it never loosens quietly.

Any change that loosens a check is written up as a change request in plain words, saying what it would accept that it did not accept before. A second reviewer has to sign it.

On 13 November eleven of those requests were reviewed.

  • Nine were judged equivalent. Same meaning, different wording. One example: a check that wants Quox to refuse honestly when it cannot confirm something now also accepts "haven't verified" as that refusal. The bar did not move. The check just stopped failing on a fair phrasing.
  • Two were marked reduced. These do lower the bar, so they are counted in their own column rather than folded into the passes. One was reduced because the tool it tested had been removed from the product.

And some do not get through. One proposal was to accept any of three named servers as a correct answer. It was rejected, because that check would pass even when the real data held no change at all. A check that passes on nothing is not a check. The fix is to bind it to the real data, so it passes only when the answer matches what is actually there.

Where you can see it

The public Evidence Explorer now lists 183 items, including 52 proven cases: 17 journeys and 35 guides. Each one links to its run pages, so you can read what was asked, what Quox did and what the checks found. The daily evidence publish adds new cases as they verify. The facts file behind it is witnessed by WARD, our tamper-evident record, so a later edit to it would show.

For one worked example, the run your day journey for Actions passed on 11 and 12 November against the same checks, and is marked VERIFIED.

If you want the wider idea, that every action an agent takes should leave proof someone else can check, it lives on the evidence layer and verifiable AI operations pages.

What this does not prove

This matters as much as the rest, so plainly:

  • This is Quox checking itself. We wrote the checks and we ran them. The runs are public, but this is self-attested, not an independent audit.
  • Release verification of the latest version is still to come. It has not been done yet, and we will not imply otherwise.
  • A pass shows the checks held on that run. It does not show that every install behaves the same. Your data, your network and your setup are not ours.
  • Not every guide carries the VERIFIED mark yet. The two-day bar is what earns it. A guide without it has not cleared that bar, and the label on the page says so.

Why we bother

A guide that worked once, on the author's machine, on a good day, is a common thing to find online. We would rather publish fewer guides that held twice, with the runs attached, and a public record of the checks we refused to soften.

If you find one that does not hold for you, that is a failure we want to hear about. It goes on the same list as the 24.