Agent Response Contract
A deterministic firewall that checks what an agent claims against what actually ran, before the reply reaches the customer.
Large language models reliably invent confirmations of actions they did not perform: "I emailed your team", "I created the task", "I sent it to your HITL inbox". This is not a prompt-quality problem. It is a structural failure mode of the architecture, because the model has no built-in awareness of which tools actually executed this turn.
The Agent Response Contract is a deterministic check that runs at the single-turn boundary of every customer-facing reply. It does not reason about whether a claim is probably true. It checks whether evidence exists in this turn's skill results, and strips or rewrites the claim when it does not.
What it is
Five guard functions, fourteen invariant cases, bilingual English and Spanish.
| Guard | Checks |
|---|---|
verifyClaimedActions | Phantom side-effect claims: email, task creation, workflow start, sub-agent creation, HITL routing, crystal/skill creation, org-scoped actions. Also covers clock-time scheduling claims without a disclaimer, internal path leakage, and placeholder tokens (TODO, TRACKINGCODE). |
verifyExternalFactClaims | Verdict-shaped claims about external state (delivery status, prices, third-party API responses) without a supporting skill result. |
sanitizeMarkdown | Strips markdown formatting on channels that render it as literal text (telegram, sms, plaintext). |
enforceLanguageConsistency | Narrow datetime-token translation (weekday and month names) when the user's language is set. Not full-response translation. |
sanitizeOrgScopingClaim | Replaces an org-scoping phrase ("for your organization") with "for your account" when orgId is unset, empty, or a fallback placeholder. |
When a guard fires, the claim is stripped from the reply rather than the whole response being blocked. A reply that is shorter and accurate is preferred over one that is longer and wrong.
How it runs
The contract is wired into the collector's chat dispatch at the C5 chokepoint (services/collector/lib/stackModeDispatch.js), called on the assembled reply before it returns to the customer. It runs before Agent Honesty Chain guards: response-contract checks the current turn's claims against this turn's skill results first, then AGH-CHAIN checks cross-turn and cross-agent claims against the wider AEE/VOLT chain.
The pre-guard text is kept as result_before_contract so the audit-log UI can show the original draft alongside the shipped reply. Every guard application emits a response_contract_applied VOLT event carrying the full guard trace, an original_preview and a final_preview.
Maturity
Honest status, not aspirational. Every row below reflects the current state, not the target state.
| Capability | Status | Notes |
|---|---|---|
responseInvariants module (14 cases, EN+ES) | Stable | 100 tests across 4 test files. |
Guard: verifyClaimedActions (phantom claims) | Beta | Verified in-process against production-identical code. No live HTTP soak. |
Guard: verifyExternalFactClaims | Beta | Coarse first pass only: fails when skill results are empty. Entity-proximity refinement is deferred. |
Guard: sanitizeMarkdown | Beta | Live verification against a real Telegram channel is pending. |
Guard: enforceLanguageConsistency | Beta | EN ↔ ES token substitution only. No live Spanish traffic soaked at volume. |
Guard: sanitizeOrgScopingClaim | Beta | Re-verified 2026-07-28 against the real module; the org-scoping wire-in works correctly. |
| Wire-in at the C5 chokepoint | Stable | Flag-guarded, called before the reply body returns. |
| VOLT trace emission | Beta | Try/catch prevents trace failures from breaking the response path. |
| Audit-log UI (collapsible section) | Beta | 10 unit tests. No live browser verification performed. |
| CI gate | Stable | Hard-fail cutover landed 2026-05-31; soft-fail default before that date. |
Known gaps:
- Live HTTP verification through the Docker container and the real chat endpoint is deferred to a later acceptance slice. Everything above was verified in-process against production-identical code, which is high-signal but not the same as a live soak.
- The workflow-executor and MCP dispatch paths are not both verified for this surface; coverage today is the chat dispatch path.
- Entity-proximity refinement for
verifyExternalFactClaimsis not built: an agent that called any skill can currently make a fabricated factual claim about a named entity and pass.
What it does not cover
- General output safety. Prompt-injection defence and secret scrubbing are separate, input-side and tool-result-level systems.
- Full-response translation. Only narrow datetime-token substitution.
- Content quality. The contract checks deterministic invariants against evidence, not whether an answer is subjectively good.
- The skill firewall. Tool-execution permissioning is a separate surface.
- Cross-org governance. The contract uses
orgIdas evidence for claim truthfulness; it does not extend the multi-tenant scoping model.
The env flag
A single flag controls the contract: RESPONSE_CONTRACT_ENABLED. Default is off in development and on in production. There is a companion build-time flag, RESPONSE_CONTRACT_BUILD_GATE=soft|hard, that controls whether a failing contract test blocks the collector's Docker image.
Both are environment variables today: there is no dashboard toggle or Settings surface showing the flag's current state. Documenting it here is the first of the required surfaces; dashboard visibility and a Settings override are not built yet.
Related
- Agent Honesty Chain: cross-turn and cross-agent claim verification that runs after this contract.
- Agent Honesty: product overview of both surfaces.
- AOCL System: the audit layer this contract's trace attaches to.