Qluster

Your Quox fleet, under one roof.

Qluster is a standalone fleet controller for your own QuoxCORE installations. If QuoxCORE is one governed AI infrastructure command center, Qluster is Proxmox for Quox — the console you run when you have more than one box and need to see, check, and (carefully) task all of them from a single pane of glass.

Qluster is not a QuoxCORE plugin and does not run inside a Quox install. It's a separate overlay process with its own server, its own database, and its own login. Qluster going down never means a Quox box is down — the boxes keep running exactly as they were.

What It Is

One operator runs many QuoxCORE installations for different reasons: a research box, a security/SOC box, a dev/staging box, a compliance box, a box per client. Each box is fully governed on its own — its own policy, its own memory, its own DEFCON posture, its own human-in-the-loop gate.

What's missing without Qluster is the view across them: which boxes are healthy right now, which ones have drifted from baseline, whether last night's backups actually landed, and a safe way to ask one box's agent to do something without breaking its sovereignty.

Qluster answers those questions from a single controller process:

• A datacenter view of the whole fleet — boxes as rows, resource gauges, status at a glance.
• Cross-instance checks — version drift, config fingerprint drift, remote WARD-chain verification, backup freshness.
• Secure enrollment — a box only joins the fleet if it cryptographically proves who it is, and it can revoke Qluster's access at any time.
• Encrypted backup pull — centralised, hash-verified, encrypted-at-rest backups pulled from every enrolled box.
• Governed delegation — task a box's agent from the console, always with your explicit approval, and the box still applies its own policy and its own HITL gate to the request.
• Colonel Qluster — a read-only meta-agent that can answer questions across the whole fleet.
• QLOGS — a tamper-evident evidence log of every controller action and every cross-box message.

Core Concepts

Who It's For

Getting Started

Run the controller

Qluster ships as a Node/Express server with a React UI served statically from the same process, backed by a local SQLite database.

git clone https://github.com/quoxai/quox-qluster.git
cd quox-qluster
npm install
cp .env.example .env       # set QLUSTER_ADMIN_PASSWORD at minimum

npm run build               # builds ui/dist
npm run dev                 # starts the server on QLUSTER_PORT (default 4200)

Or with Docker Compose:

cp .env.example .env
docker compose up -d --build

Visit the controller's address (default http://localhost:4200) — you'll land on the login page, then an empty datacenter view.

First-run setup

The controller supports multiple operators, each with a role (admin, operator, or viewer). The first operator's password, the master key, and the session secret can be supplied via environment variables, or set through a first-run setup screen the first time the controller boots with no configuration.

The operator password is hashed and persisted to the controller's own database, so it survives a restart without re-supplying secrets in the environment. Further operators are added with qluster operators add <name> --role <role>.

Connect a box

Once logged in, adding a box to the fleet is a stepped wizard, not four separate screens. It walks through a capability-tier picker — tick only the tiers you actually need, since each tier beyond Monitor needs a corresponding action on the box side:

The wizard adds the box, polls it immediately so you see a real status the moment it's created, then walks through wiring whatever tiers were ticked (read token, enrollment) before a final readiness check — an honest "here's what's still needed" summary scoped to exactly what you asked for, not a generic checklist.

The same readiness checklist is available later from the box's detail view, with one-click fixes for anything still missing.

Fleet Monitoring

The datacenter view

The main Qluster screen is a node tree and fleet grid — boxes as rows, each with live CPU, memory, and disk gauges, a stats strip for the fleet as a whole, and per-box status labels. It's built for an operator watching an estate, not one machine.

Status Wall

A reduced, NOC-style view: every box is a tile, colour-coded — healthy, watch, or critical — with motion only on state change so a quiet fleet stays quiet. Built for a screen on the wall, not a screen you're staring at.

Health states

Each box is polled on a jittered interval and resolves to one of three states:

• online — responding normally.
• degraded — the box answered, but with an error (HTTP 5xx). The box is up, a service on it is not.
• unreachable — no response at all (connection refused, timeout, DNS failure).

This distinction matters: a box returning 502 because one internal service crashed is very different from a box that's actually down, and Qluster reports them differently rather than collapsing both into a scary "unreachable."

Muted boxes

A box with alerts disabled (alerts_enabled=false) is shown muted — badge dimmed, excluded from the fleet-health score — without being hidden from the view entirely. Uptime is hidden while a box is offline rather than showing a stale number, and a last-seen timestamp that's gone stale is flagged amber so an operator can tell "quiet" from "not being checked."

Cross-Instance Checks & Drift

Beyond up/down, Qluster runs scheduled checks across the fleet to catch drift before it becomes an incident:

• version.drift — flags boxes running a version that's out of step with the rest of the fleet (unknown versions are excluded from the vote rather than treated as a false positive).
• config.fingerprint — detects configuration drift between boxes that should be aligned.
• Remote WARD verify — the controller recomputes a box's WARD hash chain remotely and compares it against the box's own manifest, so you know a box's tamper-evident log is actually intact, not just present.
• backup.freshness — flags a box whose most recent backup is older than policy allows.

Secure Enrollment

Connecting a box for anything beyond basic monitoring requires enrollment — a deliberate trust handshake, not an API key pasted once and forgotten:

1. The box admin issues a single-use enrollment token (15-minute TTL, stored hashed) with an explicit scope checklist — node.read by default, backup.trigger / backup.download / agent.delegate opt-in.
2. The controller exchanges that token for a long-lived, scoped bearer token (hashed at rest) and pins Ed25519 keys in both directions.
3. Every subsequent operational request from the controller to the box is a signed envelope — timestamped, checked against a ±120-second window, and carrying a replay nonce so a captured request can't be resent later.
4. The box admin can list and revoke any controller's access at any time; revocation takes effect within about 60 seconds (the box's validation-cache TTL).

Credentials — bearer tokens and pinned keys — live encrypted in the controller's own database (AES-256-GCM under the controller's master key), not in any shared vault, so a compromise of one surface doesn't leak the other's secrets.

Encrypted Backup Pull

For boxes enrolled with backup scopes, Qluster centralises backup retrieval instead of every box running its own export script:

1. Trigger — the controller asks the box to start a backup for an allowed org.
2. Wait — the controller polls until the backup completes, tolerating transient errors along the way.
3. Download — the archive streams from the box to the controller over the signed, scoped channel.
4. Verify — the downloaded archive is SHA-256 hash-checked against the box's own manifest. A hash mismatch fails the pull; the controller never trusts an unverified archive.
5. Encrypt at rest — the verified archive is encrypted (AES-256-GCM, per-box key) before it's written to Qluster's own storage.

The controller never extracts archives — an untrusted archive is treated as untrusted input, and restoring from a pulled backup stays a deliberate manual act, not something Qluster does silently. Retention keeps a bounded number of recent backups per box (default: 5).

Governed Delegation

Qluster can ask a box's own agent to do something — but the design keeps every box's sovereignty intact:

1. Always human-in-the-loop. A delegation request never auto-sends. It sits as a pending approval in the Qluster console until the operator explicitly approves it. This is not configurable off.
2. Signed send. On approval, the controller sends a signed, scoped envelope to the target box under a distinct qluster:<controller_id> identity — never impersonating a local user or forging credentials.
3. The box runs it under its own policy. The receiving box's own policy engine and its own HITL gate apply in full. A remote request that its policy would deny gets denied, exactly as if a local user had tried it. Qluster does not — and cannot — waive a box's own governance.
4. Honest outcomes. A delegated task resolves to one of four states, and Qluster is careful not to collapse them into a single misleading verdict:
   • completed — the task ran and returned a result.
   • denied_by_box: the box's own policy genuinely refused the request. This is real sovereignty at work.
   • box_exec_failed — the request passed the box's identity and policy checks, but failed for a runtime or credential reason on the box (for example, a missing or stale LLM key). This is not a policy denial, and Qluster labels it distinctly rather than telling the operator "the box denied you" when the real story is "the box's runtime needs attention."
   • pending_hitl — the box queued the request in its own human-in-the-loop inbox rather than resolving it immediately.
5. Box-side control. A box can require operator approval for every delegated task via a require-approval toggle, layering the box's own human review on top of the controller's.

Soaked 2026-08-06: 161 governed delegations driven end to end (create pending, approve or deny, send, terminal) against the demo Qluster estate, of which 140 completed, 20 were denied_by_operator at the approval gate, and 1 failed with a box-unreachable reason after the target box's process was killed mid-flight on purpose. Approve-step latency across 71 timed round trips ranged from 8ms to 491ms, median 29ms. The run also drove failure and edge paths: a delegation to a box not enrolled for delegation was refused before any row was created; approving or denying an already-terminal delegation was correctly rejected with a 409, leaving the original outcome untouched; ten delegations approved concurrently all resolved correctly with distinct ids and no interleaving; the killed box resumed normal delegation traffic once restarted. The controller's WARD evidence chain grew from 93 to 699 entries across the run and still verifies PASS.

Known gap: the demo box used for that soak always answers a delegation with a synchronous completed reply, so the run could not produce a genuine box-side denial (denied_by_box), a box-side runtime failure (box_exec_failed), or a deferred pending_hitl outcome; those remain unproven at volume. The full path through a real box's own execution still needs that box's collector rebuilt to carry the agent.delegate scope and its /delegate route; until then the scope gate correctly refuses delegation rather than pretending to run it. nw-prod-01, the enrolled production box, is not currently scoped for delegation. pending_hitl outcomes are also not yet reconciled automatically: if a box queues a delegation in its own inbox, the controller shows pending_hitl but does not yet poll for the eventual verdict.

Colonel Qluster

Colonel Qluster is the fleet's meta-agent — a read-only chat agent that lives in the controller and can answer questions across the whole estate: which boxes are behind on WARD verification, which one is under load, which client box needs attention.

It reads fleet state through a small set of dedicated tools (box status, fleet-wide stats, running a check, listing backups) and does not itself take action on any box — delegation is a separate, explicitly-approved flow.

Colonel Qluster is OAuth-first: the default provider path uses a Claude subscription login, with no API tokens required to get started. A provider switcher supports the Anthropic API and OpenAI as alternatives, with keys stored encrypted if you choose to configure them.

QLOGS

QLOGS is Qluster's evidence surface — a queryable, tamper-evident record of what the fleet controller did and what it exchanged with every box, built on the same four protocols as QuoxCORE:

• AEE envelopes — every controller action and every cross-box message is wrapped in a spec-conformant envelope (unique ID, timestamp, sender, recipient, intent, correlation ID), so any result can be traced back through the requests that produced it.
• AOCL layer verdicts — each envelope carries the relevant policy-layer trace. For delegation specifically, the honesty split is enforced at this layer too: a real policy denial shows as an L3 deny, a box-side runtime failure shows as an L7 failure — never mislabeled as the other.
• VOLT bundles — a per-correlation evidence bundle, metadata-only (hashes and payload shapes, never the actual payload contents), so evidence can be reviewed without exposing what was said.
• WARD hash-chain — a controller-local, append-only, content-free chain. QLOGS ships a live verifier with a three-way verdict — VERIFIED, UNVERIFIED (couldn't be checked), or FAILED (actually tampered) — so "couldn't check right now" is never shown as if it meant "tampered."

On top of the log itself, QLOGS provides a filterable query view (by intent, action, correlation, box, actor, type, layer, priority, and time), a cross-box correlation timeline for requests that touch more than one box, and streamed NDJSON/CSV export.

Security Posture & Limits

Qluster is honest about where it stands today rather than overselling maturity it hasn't earned yet:

• LAN/VPN-only posture. The controller is designed for trusted operators on a private network. It has its own multi-operator, role-based login (admin, operator, viewer roles; password hashing, HMAC-signed sessions, rate limiting; live-verified 2026-08-06: a viewer-role session was refused a privileged call server-side) but has not yet gone through the hardening required for a public, internet-facing deployment.
• No per-org access scoping. There is no multi-tenant or per-org access control on the box registry or on QLOGS: any authenticated session, regardless of role, sees every enrolled box's data. Role-based access now gates which actions an operator can perform, but not which boxes' or organisations' data a session can see; a hard gate on that is required before any multi-tenant use.
• WARD tail-verify has a cap. Remote WARD verification checks up to 5,000 entries from genesis on very large chains; checkpoint-anchored tail-verification for chains larger than that is future work, and the cap is reported honestly in the check result rather than silently truncated.
• Replay-nonce store is in-process. It protects against replay within a single controller instance but is not yet a shared, durable store — relevant only if you ever run more than one controller instance against the same boxes.
• Creating a delegation is rate-limited. Capped at 30 requests per 60 seconds per IP, confirmed live 2026-08-06 when a soak script hit the cap and backed off. Delegation remains trusted-operator-only regardless.
• pending_hitl isn't reconciled automatically. If a box queues a delegated task in its own HITL inbox, Qluster records the pending state but does not yet poll for the eventual verdict — check the outcome on the box itself.
• Backups are only as robust as the box that produces them. Qluster pulls a box's backup archive as-is; a fragile or partially-covered backup subsystem on the box is a box-side concern, not something Qluster's pull flow can fix.

How It Connects

Qluster is an overlay, not a dependency. Every QuoxCORE box it manages continues to run exactly as it would standalone — its own auth, its own policy, its own memory. Qluster reaches out to each box over HTTPS using the scopes granted at enrollment:

                        Qluster Controller
                    (fleet DB, its own login)
                    /        |         \
                   /         |          \
        QuoxCORE box A  QuoxCORE box B  QuoxCORE box C
        (research)      (SOC)           (client — Acme)
        - own policy    - own policy    - own policy
        - own memory    - own memory    - own memory
        - own HITL      - own HITL      - own HITL

A box works fully without Qluster. Enrolling in a fleet adds a monitored, governed remote-control surface on top of a box — it never replaces or weakens anything the box already enforces on its own.

Technical Stack

Next Steps

• QuoxCORE — the AI command center that runs on each box in the fleet
• AEE Protocol · AOCL Protocol · VOLT Protocol · WARD Protocol — the four protocols QLOGS is built on
• Compare All Products — see how Qluster fits alongside the rest of the Quox lineup