Get started

System Maintenance

Backup, update, and restore - one unified system.

System Maintenance is the unified backup, update, and restore plugin for QuoxCORE. It replaces the standalone backup-restore module and adds full update lifecycle management — version tracking, update detection from GitHub Releases, one-click updates with automatic pre-update backups, real-time progress streaming, and rollback with data safety guarantees.

Overview

QuoxCORE deployments need three things to stay healthy: regular backups, timely updates, and a fast path back if something goes wrong. System Maintenance handles all three from a single dashboard view.

What It Does

CapabilityDescription
Version TrackingEvery service reports its version via health endpoints. The dashboard shows current version at a glance.
Update DetectionPolls GitHub Releases every 6 hours. Compares remote version against local using semver. Notifies via inbox when updates are available.
One-Click UpdatesExecutes the full pipeline: preflight checks, automatic backup, git pull, per-service build, rolling restart, and post-update verification.
Real-Time ProgressSSE (Server-Sent Events) stream shows each pipeline step as it happens — no page refreshing, no guessing.
Automatic BackupsPre-update backups are created automatically and tagged with the update operation ID. Existing scheduled and manual backups continue unchanged.
RollbackIf something goes wrong, rollback reverts to the pre-update git tag and restores the tagged backup. One click.
Audit TrailEvery update and backup operation is logged with timestamps, duration, performer, and outcome.

Architecture

                  GitHub Releases
                  (version check)
                        |
                        v
                  Auth Service (3101)
                  - RBAC enforcement
                  - Update history & settings
                  - Backup management
                        |
                        v
                  Orchestrator (3201)
                  - Docker socket access
                  - git pull / checkout
                  - docker compose build
                  - Rolling restart
                  - SSE progress stream
                        |
                        v
                  All Services
                  - Health endpoints with version
                  - Rebuilt and restarted in order

The Auth service handles the request layer — RBAC, audit, history, and settings. The Orchestrator handles the execution layer — Docker operations, git, and builds. This split matches the existing QuoxCORE architecture.

Update Detection

How It Works

  1. Read local VERSION file (semver, e.g. 4.2.0)
  2. Fetch the latest GitHub Release from the QuoxCORE repository
  3. Compare the release tag against the local version using semver comparison
  4. If a newer version exists, store the result in the update_checks table and create an inbox notification

Checking for Updates

Updates are checked automatically every 6 hours (configurable). You can also trigger a manual check from the Update Panel by clicking Check Now.

There is a 5-minute cooldown between manual checks to prevent rate limiting.

Notification Escalation

ConditionAction
Update detectedInbox notification (low priority), blue dot on sidebar widget
Update available >24hDismissible banner in header
Update available >7dInbox notification escalated to normal priority
Security updateInbox notification at high priority
Update failedInbox notification at urgent priority with rollback CTA

Update Execution

The Update Pipeline

When you click Update to vX.Y.Z, QuoxCORE runs a 6-step pipeline:

Step 1: Preflight

  • Verify all critical services are healthy
  • Check available disk space
  • Confirm git working tree is clean
  • Validate the target version exists

Step 2: Pre-Update Backup

  • Automatically creates a full backup tagged with the update operation ID
  • Backup name: Pre-update v4.1.0 → v4.2.0
  • Waits for backup completion before proceeding
  • If backup fails, the update is aborted

Step 3: Git Pull

  • git fetch origin
  • git checkout <target-tag> or git pull origin main
  • Verifies no merge conflicts

Step 4: Build

  • Runs docker compose build for each service in dependency order
  • Services are built sequentially: auth, memory, files, tasks, collector, orchestrator, dashboard
  • Build failures abort the pipeline

Step 5: Rolling Restart

  • Each service is restarted individually with docker compose up -d <service>
  • Health check with 120-second timeout after each restart
  • If any service fails health check, the pipeline is aborted

Step 6: Post-Update Verification

  • Full health check across all services
  • Verify VERSION matches the target version
  • Mark update operation as completed
  • Create inbox notification: "QuoxCORE updated to vX.Y.Z"

Real-Time Progress

The update pipeline streams progress via Server-Sent Events (SSE). The dashboard connects to this stream and shows a step-by-step progress display:

Step            Status
─────────────────────────
Preflight       ✓ Completed
Backup          ✓ Completed
Git Pull        ● Running — Pulling latest changes...
Build           ○ Pending
Restart         ○ Pending
Verify          ○ Pending

Each step shows its current status (pending, running, completed, failed) with a real-time message describing what is happening.

Rate Limiting

  • One update per hour (prevents rapid re-attempts on transient failures)
  • Admin role required for all update operations
  • Cancel available while update is in progress

Rollback

If an update causes problems, rollback:

  1. Checks out the previous git tag
  2. Restores the pre-update backup (tagged with the update operation ID)
  3. Rebuilds all services
  4. Restarts with health checks
  5. Verifies everything is back to the previous state

Rollback follows the same SSE progress pattern as updates, so you can watch each step in real time.

Backups

The backup system within System Maintenance is the same battle-tested backup-restore system, now integrated into the unified view:

  • Full Backups — Complete system state including auth database, memory, and configuration
  • Selective Backups — Choose specific components (auth_db, memory_system, etc.)
  • Scheduled Backups — Cron-like scheduling with configurable retention
  • Verification — Integrity checks on backup archives
  • One-Click Restore — Restore from any backup in the history

Backup Components

KeyComponentDescription
auth_dbAuth DatabaseUsers, orgs, roles, sessions, API keys
memory_systemMemory SystemFacts, preferences, entities, observations
filesFile StorageUploaded documents and folder structure
tasksTasks DatabaseTask records and history
configConfigurationService configuration and environment

Settings

Update behaviour is configurable per organisation:

SettingDefaultDescription
auto_check_enabledOnAutomatically check for updates every N hours
check_interval_hours6Hours between automatic checks
auto_backup_before_updateOnAlways create a backup before updating
release_channelstableWhich release channel to follow (stable, beta, nightly)
maintenance_window_enabledOffRestrict updates to a specific window
maintenance_dayDay of week for maintenance window
maintenance_hourHour (UTC) for maintenance window
notification_lead_hours24Hours of advance notice before scheduled updates

Dashboard Views

System Maintenance lives at /system in the QuoxCORE dashboard. It has three tabs:

Updates Tab

  • Current version and available version display
  • Check Now button for manual update checks
  • Changelog (rendered from GitHub Release notes markdown)
  • Pre-update safety checklist (auto-backup, health check)
  • Update button with real-time progress
  • Rollback option for completed updates

Backups Tab

  • The existing backup panel, unchanged
  • Create manual backups
  • View backup history
  • Restore from any backup
  • Schedule configuration

Audit Log Tab

  • Combined history of all update and backup operations
  • Filterable by type, status, and date
  • Shows duration, performer, and outcome for each operation

The System Maintenance sidebar widget shows a compact status indicator:

  • Green dot — Backups healthy, no updates pending
  • Blue dot — Update available (with "Update available" label)
  • Red pulsing dot — Critical: backup health issue or update failure
  • Grey dot — Status unavailable

Clicking the widget navigates to /system.

Dashboard Card

The System Maintenance card on the main dashboard shows:

  • Current version
  • Available version (if update exists)
  • Last backup time
  • Backup health indicator
  • Quick link to the System Maintenance view

Database Schema

Three tables in the auth service database:

update_checks

Stores the results of version checks against GitHub Releases.

ColumnTypeDescription
idTEXT PKUnique check ID
org_idTEXTOrganisation scope
current_versionTEXTLocal version at time of check
available_versionTEXTRemote version found
update_availableINTEGER1 if remote > local
release_notes_mdTEXTMarkdown release notes
published_atTEXTRelease publish date
release_urlTEXTGitHub Release URL
checked_atTEXTTimestamp of check

update_operations

Records every update and rollback execution.

ColumnTypeDescription
idTEXT PKOperation ID
org_idTEXTOrganisation scope
from_versionTEXTStarting version
to_versionTEXTTarget version
statusTEXTpending, running, completed, failed, cancelled, rolling_back, rolled_back
pre_update_backup_idTEXTTagged backup ID
started_atTEXTStart timestamp
completed_atTEXTEnd timestamp
duration_msINTEGERTotal duration
current_stepTEXTActive pipeline step
steps_jsonTEXTJSON array of step results
error_messageTEXTError details (if failed)
error_stepTEXTWhich step failed
performed_byTEXTUser who triggered the operation

update_settings

Per-organisation update preferences.

ColumnTypeDescription
org_idTEXT PKOrganisation scope
auto_check_enabledINTEGERAuto-check toggle
check_interval_hoursINTEGERCheck frequency
auto_backup_before_updateINTEGERPre-update backup toggle
release_channelTEXTstable, beta, or nightly
maintenance_window_enabledINTEGERWindow restriction toggle
notification_lead_hoursINTEGERAdvance notice period

API Endpoints

Auth Service (Request Layer)

MethodPathPurpose
GET/api/orgs/:orgId/update-statusCurrent + available version, last check
POST/api/orgs/:orgId/update-checkForce version check against GitHub
GET/api/orgs/:orgId/update-changelogRelease notes markdown
POST/api/orgs/:orgId/update-startBegin update pipeline
GET/api/orgs/:orgId/update-progressSSE stream (proxied from orchestrator)
POST/api/orgs/:orgId/update-cancelCancel in-progress update
POST/api/orgs/:orgId/update-rollbackRollback to pre-update state
GET/api/orgs/:orgId/update-historyPast update operations
GET/api/orgs/:orgId/update-settingsRead update settings
PATCH/api/orgs/:orgId/update-settingsModify update settings

Orchestrator (Execution Layer)

MethodPathPurpose
GET/system/versionRead VERSION file
POST/update/executeRun update pipeline
GET/update/progressSSE progress stream
POST/update/cancelAbort running update

All auth service endpoints require authentication and admin role. The orchestrator is internal-only (not exposed to the internet).

Version Infrastructure

Every QuoxCORE service includes version information:

  • VERSION file at repo root — single source of truth (e.g. 4.2.0)
  • Docker build argsQUOXCORE_VERSION injected at build time via docker-compose.yml
  • Environment variableprocess.env.QUOXCORE_VERSION available in every service
  • Health endpoints — Every service returns version in its health check response
  • Frontendimport.meta.env.VITE_QUOXCORE_VERSION available at build time via Vite

Plugin Registration

System Maintenance is registered as a free plugin:

  • Plugin name: system-maintenance
  • Display name: System Maintenance
  • Tier: Free
  • License: Free
  • Features: Update detection, update execution, rollback, full backup, selective backup, verification, scheduling, audit logging, version tracking

The legacy backup-restore plugin remains registered for backwards compatibility and redirects to the System Maintenance view.