System Architecture

Quox is the self-hosted control plane for AI agents. This guide explains how all the pieces fit together - from high-level concepts to technical implementation details.

The 30,000 foot view

Quox is the self-hosted control plane for AI agents. It gives agents real access to your infrastructure while you control what they can do, approve what matters, and keep proof of every governed action - you speak naturally, and it handles the technical details.

What problem does Quox solve?

Managing 50+ servers, containers, and services is complex. Traditional approaches require:

Memorizing dozens of CLI commands
SSH-ing into multiple machines
Manually tracking what is running where
Maintaining runbooks and documentation

Quox replaces this with a single conversational interface. You say "check disk space on all docker hosts" and it happens - no memorization, no manual SSH sessions.

The core idea

┌─────────────────────────────────────────────────────────────────────┐
│                         YOU (Natural Language)                       │
│                    "restart nginx on nw-web-01"                       │
└─────────────────────────────┬───────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      Quox DASHBOARD (React)                       │
│                    Command Centre + Voice Interface                  │
└─────────────────────────────┬───────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     ORCHESTRATION LAYER (QuoxFlow)                     │
│         Smart Router + Claude AI + Memory + Agent Hierarchy          │
└─────────────────────────────┬───────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      INFRASTRUCTURE LAYER                            │
│              Bastion Host + QuoxAgent Agents + Your Servers                │
└─────────────────────────────────────────────────────────────────────┘

System components

Quox is composed of four main layers, each with a specific responsibility.

Layer 1: Frontend (what you see)

The Dashboard is a React web application with a voice-enabled interface. It provides:

Component	Purpose
Command Centre	Primary interface with chat, monitoring, and management views
Voice Interface	Voice-enabled assistant with animated eye visualization
Memory UI	View and manage what Quox remembers about you
Agent Views	Monitor specialized AI agents and their activities
Workflow Builder	Create and deploy automation workflows

Why React? Fast, component-based development with excellent tooling. The modular design means new features can be added without touching existing code.

Layer 2: Orchestration (the brain)

This is where the magic happens. The orchestration layer includes:

Component	What It Does	Why It Matters
QuoxFlow	Workflow automation engine with governance	Typed execution, policy gates, audit trails
Claude AI	Understands your intent, plans actions	Natural language understanding
Smart Router	Pattern-matches common requests	Faster responses, reduced API costs
Memory System	Remembers context, preferences, history	Continuity across conversations
Agent Hierarchy	Specialized agents for different domains	Right expertise for each task

Why QuoxFlow? Purpose-built for AI agent orchestration with typed inputs/outputs, policy gates, and full audit trails. Self-hosted - your data stays on your infrastructure.

Layer 3: Execution (the hands)

Commands need to reach your servers. This layer handles secure execution:

Component	Purpose
Bastion Host	Single secure gateway to all infrastructure
QuoxAgent Agents	Lightweight agents running on each managed host
SSH Connections	Encrypted command execution

Why a Bastion? Security. Your servers are not exposed directly. All access flows through a single, hardened entry point with full audit logging.

Layer 4: Infrastructure (your servers)

The actual hosts, VMs, containers, and services you are managing:

Category	Examples
Hypervisors	Proxmox nodes running VMs and containers
Docker Hosts	Containers running applications
Monitoring	Grafana, Prometheus, Uptime Kuma
Storage	NAS, TrueNAS systems
Network	pfSense, Pi-hole DNS

How data flows

Understanding how your request travels through the system helps you troubleshoot and extend Quox.

Request flow: text commands

1. You type: "check disk space on nw-web-01"
                    │
                    ▼
2. Dashboard sends to Collector API (/api/chat)
                    │
                    ▼
3. Smart Router checks: Is this a known pattern?
         │                        │
         │ YES                    │ NO
         ▼                        ▼
4a. Direct execution      4b. Send to Claude with context
    (faster, no AI cost)      - System prompt + self-knowledge
         │                    - Memory context (WSM + semantic)
         │                    - Agent-specific capabilities
         │                        │
         └────────────┬───────────┘
                      │
                      ▼
5. Execute command via QuoxAgent agent or SSH
                      │
                      ▼
6. Response flows back through same path
                      │
                      ▼
7. Dashboard displays formatted result

Request flow: voice commands

1. You speak into microphone
                    │
                    ▼
2. Audio sent to /webhook/transcribe
                    │
                    ▼
3. Whisper AI converts speech to text
                    │
                    ▼
4. Processed as text command (see above)
                    │
                    ▼
5. Response sent to /webhook/speak
                    │
                    ▼
6. ElevenLabs converts text to speech
                    │
                    ▼
7. Audio plays through speakers

Memory flow

Every Conversation
        │
        ├──▶ Episodic Memory (full logs, by date)
        │
        ├──▶ Semantic Memory (extracted facts, preferences)
        │
        ├──▶ Entity Memory (hosts, services, relationships)
        │
        └──▶ Working Set Memory (active context, open loops)

On Next Request
        │
        ├──◀ WSM provides current focus, recent entities
        │
        ├──◀ Semantic provides relevant learned facts
        │
        └──◀ Hybrid search ranks and retrieves context

Technology choices

Every technology choice has a reason. Here is why Quox uses what it uses.

Frontend stack

Technology	Why
React 19	Component architecture, strong ecosystem, fast rendering
Vite	Lightning-fast dev server and builds (10x faster than Webpack)
React Router	Clean URL-based navigation
CSS (custom)	Full control over visual aesthetic, no framework bloat

Backend stack

Technology	Why
QuoxFlow	TypeScript workflow engine with typed nodes, governance, and audit trails
Docker	Consistent deployment, easy updates, isolation
Claude API	Strong reasoning for infrastructure commands
Qdrant	Vector database for semantic memory search

Execution stack

Technology	Why
QuoxAgent (Go)	Lightweight binary, compiles to single executable, low overhead
SSH	Industry standard for secure remote execution
Bastion pattern	Security best practice for infrastructure access

Why these choices?

Self-hosted first - Your data stays on your infrastructure
Extensible - Each component can be replaced or extended
Observable - Full logging and audit trails throughout
Performant - Smart Router bypasses AI for common patterns (cheaper, faster)

Agent architecture

Quox uses a hierarchical agent system where specialized agents handle different domains.

Agent hierarchy

Level 0: ORCHESTRATOR
         Quox (full access, coordinates all agents)
              │
              ├────────────────┬────────────────┐
              ▼                ▼                ▼
Level 1: MANAGERS
         WARDEN            DAEDALUS        (others)
         (Security)    (Infrastructure)
              │                │
              ▼                ▼
Level 2: SPECIALISTS
         CIPHER           VAULT           ORACLE
         (Network)       (Storage)       (Analytics)
              │
              ▼
Level 3: WORKERS (task-specific agents)

Level 4: OBSERVERS (read-only monitoring)

Agent domains

Agent	Domain	Key Capabilities
Quox	Orchestration	Full access, delegation, coordination
WARDEN	Security	Threat detection, access control, auditing
DAEDALUS	Infrastructure	Host management, deployments, scaling
CIPHER	Network	Network ops, connectivity, troubleshooting
VAULT	Storage	Backup operations, storage management
ORACLE	Analytics	Pattern detection, anomaly detection
ARCHIVIST	Memory	Knowledge management, memory extraction

Why hierarchy?

Separation of concerns - Each agent is expert in its domain
Security boundaries - Agents only access what they need
Escalation paths - Complex issues route to appropriate experts
Parallel processing - Multiple agents can work simultaneously

Integration points

Quox is designed to integrate with your existing tools.

Inbound (things talking to Quox)

Integration	Mechanism	Use Case
Web Browser	React app on port 80	Primary user interface
API Clients	API webhooks	Programmatic access
Voice Input	Whisper transcription	Hands-free operation
Slack/Telegram	QuoxMCP connectors	Chat-based commands

Outbound (Quox talking to things)

Integration	Mechanism	Use Case
Linux Hosts	SSH via bastion	Command execution
Docker Hosts	QuoxAgent agent API	Container management
Proxmox	API (via QuoxFlow)	VM and LXC control
Monitoring	Prometheus/Grafana	Metrics and dashboards
Notifications	ElevenLabs, Slack	Alerts and responses

Extending Quox

New integrations follow this pattern:

Add QuoxFlow workflow for the new service
Register capability in the agent that owns the domain
Add Smart Router pattern for common operations (optional)
Update memory to track new entity types

Security model

Security is not an afterthought - it is built into every layer.

Defense in depth

Layer	Protection
Network	Bastion host, no direct server exposure
Authentication	SSH keys (no passwords in transit)
Authorization	Agent permissions, memory domain boundaries
Audit	AEE protocol logs every action with full envelope trail
Execution	Safety levels (GREEN/BLUE/AMBER/RED) with approval gates

Safety levels

Level	Description	Approval Required?
GREEN	Safe, read-only operations	No
BLUE	Low-risk, logged operations	No
AMBER	Caution, potentially impactful	Yes
RED	Critical, system-affecting	Explicit authorization

What Quox never does

Store passwords in memory (credentials use SSH keys)
Execute commands without audit trail
Bypass safety gates for destructive operations
Expose internal services directly to internet

Key terms glossary

Term	Plain English Explanation
AEE Protocol	A messaging format that ensures every action is logged and traceable
Bastion Host	A secure gateway server - the single front door to your infrastructure
Context Injection	Giving Claude relevant information before it answers your question
QuoxAgent	Quox Helper Agent - lightweight software running on each managed server
QuoxFlow	Workflow automation engine (TypeScript, audit-logged)
Smart Router	Pattern matching that handles common requests without calling AI
SSH	Secure Shell - encrypted protocol for running commands on remote servers
WSM	Working Set Memory - the active context Quox maintains about your current work
Webhook	A URL endpoint that triggers an action when called

For technical readers

The sections above give the conceptual overview. Below is the detailed technical reference from the architecture documentation.

QUOX Architecture

This document describes the system architecture, tech stack, and project structure for QUOX.

Overview

Quox is the self-hosted control plane for AI agents. QuoxCORE, the platform this architecture describes, sits between agents and the systems they act on: enforcing policies and approvals, issuing scoped credentials, and recording verifiable evidence of every governed action.

Tech Stack

Frontend: React 19 + Vite + React Router
Styling: CSS (hal.css for voice interface)
Backend: n8n workflow automation (Docker)
AI: Claude API (Anthropic)
Voice: Whisper (OpenAI) for STT, ElevenLabs for TTS
Infrastructure: 50+ Linux hosts via SSH bastion

High-Level Architecture

Browser (React) → n8n Webhooks → Claude/APIs → SSH Bastion → Hosts

Voice Flow

Microphone → /webhook/transcribe → Whisper → text
text → /webhook/chat → Claude → response
response → /webhook/speak → ElevenLabs → audio

Project Structure

/home/control/quox-dashboard/
├── src/
│   ├── App.jsx              # Router setup
│   ├── main.jsx             # Entry point
│   ├── pages/
│   │   ├── HALHome.jsx      # Legacy voice interface
│   │   ├── CommandCenter.jsx # Main command center (primary UI)
│   │   └── views/
│   │       ├── WorkflowLibraryView.jsx  # Workflow template browser
│   │       ├── WorkflowBuilderView.jsx  # AI workflow builder
│   │       ├── InboxView.jsx            # HITL inbox
│   │       └── PluginsView.jsx          # Plugin management
│   ├── components/
│   │   ├── hal/             # Eye visualization components
│   │   ├── monitoring/      # Proxmox, Metrics, Uptime panels
│   │   ├── memory/          # Memory management UI
│   │   ├── activity/        # AEE activity views
│   │   ├── settings/        # Settings panels
│   │   ├── agents/          # Agent stream views
│   │   ├── mha/             # MHA components
│   │   ├── flowchart/       # Mermaid flowchart viewer
│   │   └── workflow/        # Workflow components
│   │       ├── WireMap/     # Visual n8n workflow viewer
│   │       │   ├── WireMap.jsx      # React Flow diagram
│   │       │   ├── WireMap.css      # Styling
│   │       │   ├── wireMapParser.js # n8n JSON parser
│   │       │   └── index.js         # Exports
│   │       ├── WorkflowChat.jsx     # AI chat builder
│   │       ├── WorkflowExplainer.jsx # Workflow analysis
│   │       └── CollectionImporter.jsx # ZIP/folder import
│   ├── services/
│   │   ├── memoryManager.js  # Memory system (episodic, semantic, entities)
│   │   ├── vectorMemory.js   # Qdrant vector store integration
│   │   ├── proxmoxClient.js  # Proxmox API client
│   │   ├── quoxContext.js # QUOX self-knowledge (hardwired)
│   │   ├── mhaClient.js      # MHA API client
│   │   ├── workflowLibrary.js # Workflow library search/filter
│   │   ├── workflowLoader.js  # On-demand workflow JSON loader
│   │   ├── capabilityIndex.js # Unified capability discovery
│   │   └── n8nClient.js       # n8n REST API client
│   ├── config/
│   │   ├── hosts.js          # Centralized host registry
│   │   ├── assistants.js     # Agent personalities/UI
│   │   ├── agentRegistry.js  # Agent capabilities/hierarchy
│   │   └── workflow-library-index.json # 1,340 indexed workflows
│   ├── lib/
│   │   └── aee/              # AEE protocol library
│   ├── test/
│   │   └── memoryManager.test.js # 103 tests
│   └── styles/
│       └── command-center.css # CommandCenter styling
├── public/
│   └── workflows/            # Symlink to workflow JSON files
├── scripts/
│   └── index-workflows.cjs   # Workflow indexer script
├── docs/                     # Documentation
└── dist/                     # Built files served by nginx

External Directories

/home/control/quox-memory/  # Persistent memory storage
├── profile/                 # User profile data
├── episodic/               # Conversation logs by date
├── semantic/               # Learned facts and preferences
└── projects/               # Project-specific memory

/opt/n8n/
├── docker-compose.yml       # n8n + dashboard services
├── .env                     # API keys (OPENAI, ELEVENLABS)
└── data/                    # n8n persistent data

/home/control/mha/           # MHA Helper Agent project
├── cmd/mha/                 # Daemon entry point
├── internal/               # Core implementation
├── configs/                # Configuration files
└── deploy/                 # Deployment scripts

Infrastructure Access

Bastion Host: nw-edge-gw-01 (10.20.0.2)
SSH User: control
Pattern: SSH through bastion to target hosts

Host Categories

Category	Hosts
Proxmox	proxmox01, proxmox02, proxmox03
Docker	nw-worker-01, nw-worker-02, nw-worker-03, nw-worker-04
Monitoring	grafana01, prometheus01
Storage	nas01, truenas01
Network	pfsense01, pihole01, pihole02

Context Injection

Claude receives system prompt with:

QUOX system identity and purpose
Full host inventory including IPs, roles, and relationships
Agent hierarchy (QUOX → SENTINEL/CIPHER/NOVA/GEMINI → specialists)
Assistant-specific self-knowledge (name, role, capabilities, rules, personality)
MHA documentation with installation commands
Current fleet status (connected agents, hosts without MHA)

Usage

javascript

import { formatContextForSystemPrompt } from '../services/quoxContext'

// Get full context with assistant-specific self-knowledge
const context = await formatContextForSystemPrompt(currentAssistant)

// Check if query is QUOX-related
import { isQuoxQuery } from '../services/quoxContext'
if (isQuoxQuery(text)) { /* special handling */ }

Session Continuity

Each conversation has a session_id for maintaining context across queries.

State Machine (Eye Visualizer)

IDLE (red) → LISTENING (blue) → THINKING (orange) → SPEAKING (green) → IDLE

Voice Modes

Mode	ElevenLabs Voice	Personality
Assistant	Rachel (21m00Tcm4TlvDq8ikWAM)	Warm, helpful
Cyberpunk	Arnold (VR6AewLTigWG4xSOukaG)	Deep, commanding
Precise	Adam (pNInz6obpgDQGcFmaJgB)	Measured, precise

Known Limitations

Voice requires API keys (not pre-configured)
SSH may fail if bastion connection drops
Browser may block autoplay audio
No authentication on dashboard (yet)
Direct Proxmox API calls blocked by CORS (use n8n proxy)

Workflow System

WireMap Visual Viewer

The WireMap component renders n8n workflows as interactive diagrams:

Technology: React Flow (@xyflow/react v12.10.0)
Layout: Top-to-bottom DAG with automatic depth calculation
Features:
- Category-colored nodes (trigger, AI, logic, io, storage, notify, transform)
- Hover highlighting of connected paths
- Click to select with detail panel
- Smooth animated edges

Workflow Library

1,340 indexed workflow templates from AI Agent Vault collection:

Categories: ai-llm, communication, productivity, data-processing, integration, automation
Search: Full-text, category, integration, tag filtering
Schema: AEE-compatible manifests for unified discovery

Key Services

Service	Purpose
`workflowLibrary.js`	Library search, filter, stats
`workflowLoader.js`	On-demand JSON loading via public symlink
`capabilityIndex.js`	Unified capability discovery across entities
`n8nClient.js`	n8n REST API for create/execute/status

Future RAG Collections

n8n_workflows - 7000+ templates
infrastructure_docs - Host configs
conversations - Session history
code_patterns - Reusable snippets