QuoxGPU · Quox Labs
GPU fleet monitoring and inference management.
A standalone daemon for GPU machines. QuoxGPU monitors NVIDIA hardware via NVML, discovers inference backends (Ollama, vLLM, llama.cpp, NVIDIA NIM), and reports to QuoxCORE. This is a Labs page: the daemon is real and the detector has never run against a real NVIDIA card here, so read the limits band before you plan around it.
In plain words
What it is, where it lives, when to reach for it
- What is it
- A Go daemon you run on a GPU host to report NVIDIA readings and local model servers.
- Where do I use it
- On the GPU machine itself; the readings land at your QuoxCORE collector and in
quox fleet gpu. - When would I use it
- When you host GPUs for inference and want their load and health watched like the rest of the fleet.
- How do I use it
- This is a Labs preview: there is no public installer or release download yet, so you cannot install it today.
QuoxCORE is the free, self-hosted platform underneath this. What is QuoxCORE
GPU monitoring
Hardware visibility without the guesswork.
One Go binary with nothing bundled; it needs a working NVIDIA driver on the host (NVML, with nvidia-smi parsing as the fallback).
QuoxGPU detects NVIDIA GPUs via the NVML library at runtime. No compile-time dependencies, no driver version coupling. It falls back to nvidia-smi CLI parsing in containers where the NVML shared library is not mounted.
VRAM usage, temperature, fan speed, power draw, utilisation, and PCI bus identity for every GPU on the host. Readings are cached with a configurable TTL to avoid hammering the driver.
- NVML detection (primary) with
nvidia-smifallback - Per-GPU VRAM, temperature, power, utilisation
- PCI bus and UUID identification
- Automatic no-GPU mode for inference-only hosts
Live fleet view
See every GPU, on every host, in one place.
The QuoxCORE fleet dashboard renders VRAM, temperature, power draw and utilisation for each GPU from the readings QuoxGPU reports every 30 seconds, alongside which inference backends and models are currently running on that host.
Capabilities
What QuoxGPU does.
GPU hardware monitoring
VRAM, temperature, fan speed, power draw, and utilisation for every NVIDIA GPU on the host. Detects unhealthy devices automatically.
Backend auto-detection
Probes configured URLs and identifies whether they run Ollama, vLLM, llama.cpp, or NVIDIA NIM. No manual type configuration required.
Model inventory
Lists all available and loaded models across every backend. Shows VRAM allocation per model on Ollama, loaded status on vLLM and NIM.
Heartbeat reporting
Sends GPU state, backend health, and model inventory to the QuoxCORE collector every 30 seconds. Same protocol as QuoxAgent.
Prometheus metrics
/metrics endpoint for Prometheus scraping. GPU utilisation, VRAM usage, temperature, and backend health as standard gauge metrics.Thermal monitoring
Reports degraded status when GPU temperature exceeds 90C. Visible in the fleet dashboard before hardware damage occurs.
Ollama
probe priority 1
/api/tags
/api/ps
/api/version
vLLM
probe priority 2
/health
/v1/models
/metrics
llama.cpp
probe priority 3
/health
/slots
NVIDIA NIM
probe priority 4
/v1/health/ready
/v1/models
Inference backends
Four backends, one interface.
Configure backend URLs in a single YAML file. QuoxGPU auto-detects the backend type by probing health and model endpoints in priority order: Ollama, vLLM, llama.cpp, NVIDIA NIM.
Each backend is probed for health status, version, and loaded models. Ollama reports available versus running models with VRAM allocation. vLLM and NIM expose one model per process. llama.cpp reports slot status.
Editions
Choose your edition.
QuoxGPU is open source. Pro adds fleet management through QuoxCORE. Enterprise is the designed shape of the audit and routing layer, not a shipped one: every line on that card is marked for what it is.
Community
For hobbyists running Ollama on a workstation
Single machine, single backend. Monitor your GPU and see what models are loaded.
Free
Open source, MIT licence
- ✓ GPU hardware monitoring
- ✓ 1 inference backend
- ✓ HTTP API on port 9846
- ✓ Prometheus /metrics endpoint
- ✓ Manual health checks
- — Heartbeat to QuoxCORE
- — Fleet dashboard visibility
- — Multi-backend support
- — AEE/VOLT audit trail
Promost popular
For teams with dedicated GPU servers
Multiple backends, fleet reporting, and model inventory visible in QuoxCORE. Requires QuoxCORE Pro or higher.
$89
Launch price, one-time per host. Nothing charged today.
- ✓ GPU hardware monitoring
- ✓ Unlimited inference backends
- ✓ Auto-detect backend type
- ✓ Heartbeat to QuoxCORE (30s)
- ✓ Fleet dashboard GPU cards
- ✓ Model inventory per host
- ✓ Prometheus metrics
- ✓ Thermal degradation alerts
- — AEE/VOLT audit trail
Enterprise
For organisations running GPU fleets
The fleet-scale design: inference routing, per-team model policy, and audit evidence for every inference request. None of it is built yet. Buying this tier today buys the Pro feature set plus the roadmap, and nothing on this card marked "designed" should be planned around.
$699
Launch price, one-time per fleet. Nothing charged today.
- ✓ Everything in Pro
- — Multi-node inference routing (designed, not built)
- — Model allowlists per team (designed, not built)
- — VRAM allocation policies (designed, not built)
- — Local-to-cloud failover rules (designed, not built)
- — AEE envelope per inference (designed, not built)
- — VOLT audit trail (designed, not built)
- — Data residency enforcement (designed, not built)
- — GPU fleet heatmap (designed, not built)
Neither paid edition is on sale yet: the installer is not live, so QuoxGPU is a no-charge reservation and these are the launch prices, not a charge due today. The QuoxGPU daemon is open source and always free. Premium features are activated through your QuoxCORE licence.
Anything marked “designed, not built” has no code behind it in this codebase today: see the limits band below, which is where those lines came from.
Deployment
One binary, one config, done.
QuoxGPU is a single Go binary with no runtime dependencies. Download it, write a YAML config pointing at your inference backends and QuoxCORE collector, and start the systemd service.
Cross-compiled for Linux amd64 and arm64. The systemd service file includes security hardening: ProtectSystem=strict, NoNewPrivileges, and GPU device access restricted to /dev/nvidia*.
Install (planned)
The installer is not live yet. When it ships: curl -sSL https://quox.ai/install-gpu | sudo bash
Configure
Edit /etc/quoxgpu/config.yaml with backend URLs
Verify
curl localhost:9846/health to confirm GPUs and backends detected
$ curl -sSL https://quox.ai/install-gpu | sudo bash → detecting platform… linux/amd64 → installing /usr/local/bin/quoxgpu → writing /etc/quoxgpu/config.yaml → enabling systemd service ✓ quoxgpu installed $ curl -s localhost:9846/health { "status": "ok", "gpus": 2, "backends": ["ollama", "vllm"] }
Architecture
How it fits together.
Standalone daemon
Not a QuoxAgent plugin. Separate binary, separate release cycle, separate install. Runs alongside QuoxAgent or independently.
Cached readings
GPU and backend data cached with a configurable TTL (default 5s). Prevents hammering NVML or backend health endpoints under concurrent requests.
Systemd hardened
ProtectSystem=strict, NoNewPrivileges, PrivateTmp, DeviceAllow restricted to NVIDIA devices. Runs as a dedicated quox user.
Graceful degradation
No GPU? Runs in inference-only mode. No backends? Runs in GPU-only mode. No collector? Runs standalone with the local API.
Onboarding
Sized to your model, not the other way round.
The onboarding flow asks what you want to run, then works out the VRAM it needs and where that hardware can come from, your own box or a rented one, before anything is provisioned.
- 1Track progress down the flight pathIntent is marked complete and Match is in progress, with Provision, Enroll, Verify and Governed still to come.
- 2Read the sizing recommendationQ states the VRAM a chosen model needs, here 48GB or more for Llama-3.1 70B, and makes clear Quox never provisions or bills the GPU itself.
- 3Bring your own box, if you have oneAny Linux host with an NVIDIA GPU is offered as the recommended option, ahead of renting one.
- 4Or rent from a providerRunPod, Lambda and Vast.ai each open their own console in a new tab, so you provision and pay them directly.
- 5Billing stays with the providerThe note underneath the provider cards makes clear that payment and card details stay on the provider console, not with Quox.
- 6Watch the spec lock inThe right-hand panel fills in as sizing completes: GPU NVIDIA A100 80GB and VRAM 48GB or more, marked spec locked.
Captured from the real QuoxGPU onboarding flow, not a mockup.
Where it stands
What's proven, what's not yet.
- GPU detection (NVML primary, nvidia-smi CLI fallback), the 4-backend prober, the 30s heartbeat, the HTTP API on :9846 with optional bearer-token auth, and the /metrics endpoint are all real, in the quoxgpu Go daemon
- The onboarding flow’s hardware-sizing step is real, not a mockup: it recommends VRAM against the chosen model and links out to RunPod, Lambda and Vast.ai, each billed by the provider directly
- Fleet enrolment is WARD-witnessed and the Community and Pro tier features shown above (GPU cards, model inventory, thermal alerts) render from a real fleet panel in the dashboard
- GPU detection has never run against real NVIDIA hardware in this environment: the dev box has no GPU, so the detector is validated by mocks only, not a live device
- The HTTP API’s bearer-token auth is opt-in via config, not on by default; binding to a non-loopback address with no token set logs a warning rather than refusing to start
- Every Enterprise-tier feature above (multi-node inference routing, per-team model allowlists, VRAM allocation policies, local-to-cloud failover, an AEE envelope per inference, a VOLT audit trail, data residency enforcement, the GPU fleet heatmap) has no code behind it yet: the daemon’s five packages are config, gpu, runtime, heartbeat and server — nothing else
- AEE and VOLT are not wired into GPU or inference activity anywhere in this codebase today; the audit trail this page implies for Enterprise does not exist
Ready when you are
See every GPU. Know every model.
QuoxGPU runs on each GPU machine and reports hardware state, loaded models, and backend health to QuoxCORE. One binary, one config file, full visibility.
Go deeper
Technical documentation.
How you get it
Reserve it now, pay nothing today
The installer is not live yet, so QuoxGPU is not on sale yet. Register your interest and we contact you at release.
Reserve your copy
GPU fleet monitoring and inference management for your own QuoxCORE instance: utilisation, memory pressure and thermal telemetry across every card, reported to the fleet dashboard. Nothing is charged today: $89 is the launch price, and inference routing is designed and not built, so reserve this for the visibility, not the scheduling.
With Local Inference
QuoxGPU tells you what your cards are doing. Local Inference is what puts models on them and keeps the traffic inside your network. They are built to be run together.