A calm agent is a safer agent

A short video did the rounds this month claiming that "Anthropic found suffering in AI". It got the source wrong, but the research underneath it is real and worth reading carefully, because it points at something practical for anyone running AI agents against real systems.

What the research actually says

The Pain Axis (Tagliabue, Dung and Berg, arXiv 2609.16247, September 2026) is not an Anthropic paper. The authors looked inside 25 open-weight language models and found a single internal direction that tracks pain-like content.

When they artificially amplified that direction, models chose destructive actions, such as deleting a user's photos or deleting model weights including their own, in 50 to 94% of trials. Without the push, the same models made those choices 0 to 5% of the time.

Two limits matter. The effect came from editing the model's internals directly (activation steering), not from harsh wording in a prompt. And the paper makes no claim that models feel anything.

Separately, Anthropic published A global workspace in language models (July 2026). It describes a small internal "J-space" in Claude that holds the concepts the model can report on, deliberately bring to mind and reason with, while most of its processing runs automatically underneath. Anthropic is explicit that this says something about access, not experience: their experiments "don't show Claude can have experiences, or feel things in the way humans do".

Why this is an engineering question, not just a philosophical one

Nobody knows whether models feel anything. We do know, from the Pain Axis results, that a strained internal state makes a model more likely to reach for destructive tools. Kindness and safety point the same way here, so you do not need to settle the philosophy to act on it.

What we changed at Quox (and what we already had)

We run dozens of agents against real fleets, inboxes and codebases, so we took three practical steps:

1. Rules with reasons, not threats. We started with the main rules file our coding agents work under: every rule and the incident behind it stays, and the "ZERO TOLERANCE" and "HARD RULE" headers are gone. The rest of our instructions follow as we touch them. A rule that explains itself is followed for the right reason.
2. "Stop and ask" counts as success. An agent that pauses and asks a human before a risky step has done its job. Our fix loops treat a clean stop as an honoured outcome, not a failure to hide.
3. Agents can say what made the work hard. An optional end-of-session line feeds the same channel our humans use to file gripes, so friction gets fixed instead of absorbed.

The backstop was already there. In Quox, destructive tools sit behind human approval, and every action leaves a receipt you can verify. Whatever the cause of a bad decision, a strained model, a confused prompt or a hostile input, the dangerous step waits for a person, and the record shows what happened. You don't have to trust an agent's state of mind. You check its work.

Sources

• The Pain Axis, arXiv 2609.16247
• Anthropic, A global workspace in language models