Schedulers

Background work that has to survive a closed browser tab lives here: reflection mining and The Feed, the collector's two always-server-side scheduled passes.

Quox's rule for anything that needs to run on a timer is fixed: server-side, never a browser timer. A setInterval in React only runs while a tab is open; it stops the moment the tab closes, the laptop sleeps, or the user is on a different page. Every scheduled or background job in the platform has to live in a long-running server process instead, so it keeps running whether or not anyone has the dashboard open.

The collector runs two such schedulers today: reflection mining and The Feed. QuoxBrain, a separate service, runs a third: the embedding backfill scheduler. A fourth, related scheduler (the tasks service's team schedule trigger loop) is covered at the end for completeness, and it is where a real env-var naming trap lives.

Reflection mining

Reflection mining replaced the last browser setInterval in the codebase (reflectionLoop.js) with a server-side cycle inside the collector. On each tick it:

1. Lists idle conversations (AEE envelopes with no recent activity) since a per-org watermark.
2. Extracts memories, decisions, and entities from each one and writes them to QuoxMemory.
3. Detects patterns across recent envelopes: repeated failures, error-hour clustering, host-level error rates, activity spikes, and heartbeat silence.
4. Generates a proactive alert for anything that crosses a threshold, deduplicated so the same underlying condition does not re-fire on every tick.

Reflection mining has no on/off flag: it starts unconditionally when the collector boots. The only configurable knob is its interval:

A durable per-org watermark (not an in-memory set) means a restart never reprocesses conversations already mined, and a hard cap (maxOrgsPerCycle, default 25) round-robins across ticks on a multi-tenant deployment with more live orgs than that, rather than fanning out to all of them on every tick.

The Feed

The Feed is QuoxBrain's autopilot ingestion scheduler: it pulls from a registry of sources (AEE conversations, transcripts, folders, Obsidian vaults, fleet logs, memory, git history, RepoBrain, and several database adapters) and pushes documents into Brain2, so the second brain grows without someone manually connecting every source every time. Its scheduler shape deliberately mirrors QuoxDream's: a warmup delay, then a recurring interval, both .unref()'d so they never block a clean process exit.

Unlike reflection mining, The Feed ships dark:

Per-(org, source_type) enable state lives in a feed_sources registry table, not an env var: an autopilot source with no row is enabled by default (the "connect, don't configure" design), and every ingest attempt, success or failure, writes its outcome back to that row. Every source ingester runs inside its own try/catch, so one failing source or org never blocks any other on the same tick.

Embedding backfill

QuoxBrain's hybrid retrieval (keyword plus dense vector search) depends on chunks having embeddings. Until this scheduler existed, nothing ever called backfillEmbeddings() in production at all, so coverage sat wherever a one-off manual run had left it: 345 of 76,356 chunks embedded. The embedding backfill scheduler (services/quoxbrain/lib/embeddingBackfillScheduler.js) closes that gap, mirroring the same warmup-then-interval shape as QuoxDream and The Feed, both .unref()'d so they never block a clean process exit. Each tick discovers the orgs with missing embeddings, capped per tick, and runs the backfill for each one: best-effort, so one org's failure never blocks another's turn on the same tick.

Ships dark, same as The Feed:

A related fix landed alongside the scheduler: the embed client (embeddingClient.js) now bounds every request with QUOXBRAIN_EMBED_TIMEOUT_MS (default 45 seconds), using AbortSignal.timeout(). Before this, a single hung request against the embedding provider could hold a batch open indefinitely, silently halting the whole backfill rather than failing that batch and moving on. A tick that discovers missing embeddings but embeds nothing that pass now logs a warning rather than being indistinguishable from "already caught up".

Coverage itself is now a visible number rather than something only inferrable from server logs or manual SQL: GET /brain/status returns an embeddings object (total, embedded, missing, percent), and the Brain2 settings cockpit renders it, with a plain-English warning under 50 per cent that semantic search is falling back to keyword matching for most queries.

A named env-var divergence: SCHEDULE_POLL_MS vs SCHEDULER_POLL_MS

Not a collector scheduler, but worth stating plainly because it is a real, live divergence: the tasks service's team-schedule trigger loop (services/tasks/lib/teamScheduler.js) reads process.env.SCHEDULE_POLL_MS for its poll interval (default 30 seconds if unset). The deployment configuration, both docker-compose.yml and .env.example, sets and documents SCHEDULER_POLL_MS (with an "r") instead. The two names never match, so setting SCHEDULER_POLL_MS in the environment has no effect on the team scheduler's poll interval: it always falls back to the 30-second default, silently.

This is exactly the kind of drift the Agent Response Contract family exists to catch in agent replies; here it is named directly rather than left implicit. If the poll interval genuinely needs tuning, the variable that reaches the code is SCHEDULE_POLL_MS, not SCHEDULER_POLL_MS.

Maturity

Visibility gap: the scheduler flags themselves (QUOXFEED_ENABLED/QUOXFEED_ORGS, REFLECTION_MINING_INTERVAL_MS, and the embedding backfill flags above) have no dashboard toggle or Settings surface today; they are environment variables only. The one exception is the outcome of the embedding backfill scheduler: coverage is visible in the Brain2 settings cockpit via GET /brain/status, even though the scheduler's own on/off switch is not. The closest thing to a scheduler-level UI otherwise is the feed_sources registry's per-source enable state, which is not the same thing.

Related

• QuoxDream: the consolidation pass The Feed's and the embedding backfill scheduler's shape was mirrored from, and one of the schedulers reading from the same Brain2 store.
• Memory System: the destination for reflection mining's extracted memories and alerts.
• Agent Response Contract: a different kind of drift-catching guard, checking claims rather than config. </content>