Smart Router

Overview

The Smart Router is Quox's first line of defense for handling user queries. Instead of sending every message to Claude (which costs tokens and takes 2-5 seconds), the Smart Router identifies common patterns and responds instantly with pre-computed answers.

Key benefits

• Token Savings: Bypass the LLM for predictable queries (saves 100-500 tokens per query)
• Latency Reduction: Instant responses vs. 2-5 second LLM calls
• Offline Capability: Core commands work even when LLM is unavailable
• Consistent Responses: Same query = same response (deterministic)
• User Escalation: Every response includes "Ask Claude" for complex follow-ups

Why Smart Router?

Most AI chatbots send every message to the LLM, even simple commands like "list hosts" or "fleet status". This is wasteful and slow. The Smart Router recognizes these patterns and handles them directly.

The conservative routing principle

When Smart Router routes

• Highly predictable commands: "mha status", "list hosts", "show agents"
• Static information: "what is quox", "what can you do"
• Simple lookups: "tell me about nw-web-01"
• High-frequency operations: Queries that occur 100+ times per day

When Smart Router passes to Claude

• Complex questions requiring reasoning
• Ambiguous or context-dependent queries
• Anything with nuance or multiple interpretations
• Queries where confidence is below threshold

Architecture

The Smart Router uses a two-layer cascading classification system. Each layer produces a confidence score, and results are merged to find the best match.

User Query
    │
    ▼
┌─────────────────────────────────────────────┐
│  Layer 1: Regex Classification               │
│  - Pattern matching against 18 intents       │
│  - Fast: <1ms                                │
│  - High precision for exact commands         │
└─────────────────────────────────────────────┘
    │
    ├── HIGH confidence (>0.9) → Execute Handler
    │
    ▼
┌─────────────────────────────────────────────┐
│  Layer 2: Semantic Classification            │
│  - Embedding-based similarity matching       │
│  - 130+ example utterances                   │
│  - Handles paraphrases and variations        │
└─────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────┐
│  Merge Results                               │
│  - Pick highest confidence across layers     │
│  - Check for ambiguity                       │
│  - Apply risk-based thresholds               │
└─────────────────────────────────────────────┘
    │
    ├── HIGH confidence → Execute Handler
    ├── MEDIUM + Ambiguous → Clarifying Question
    └── LOW confidence → Pass to Claude

Layer 1: Regex classification

The first layer uses regex pattern matching for speed. Each pattern is analyzed for:

• Anchoring: Patterns that start with ^ or end with $ are more specific
• Word Boundaries: \b markers increase specificity
• Match Coverage: How much of the query was matched
• Conversational Penalties: "Can you...", "Please..." reduce confidence
• Common Word Detection: Matching "the", "a", "my" as hostnames = low confidence

Layer 2: Semantic classification

When regex confidence is uncertain, the semantic layer provides paraphrase understanding:

• 130+ Example Utterances: 5-15 examples per intent
• Embedding-Based Matching: Uses OpenAI text-embedding-3-small or fallback
• Cosine Similarity: Finds most similar examples to the query
• Aggregation: Scores are aggregated by intent, not just best match

Confidence scoring

Every classification produces a confidence score between 0 and 1. Higher scores mean more certainty about the intent.

Risk levels

Different intents have different risk levels, which determine the confidence threshold required for routing:

Confidence thresholds

The router uses multiple thresholds to make routing decisions:

• High Confidence (0.9+): Route immediately, skip ambiguity check
• Intent Threshold: Varies by risk level (0.5 - 0.85)
• Minimum Confidence (0.4): Below this, always pass to Claude
• Ambiguity Threshold (0.3): Max allowed gap between top intents

Key features

Clarifying questions

When the router is uncertain between multiple intents, it asks clarifying questions instead of guessing:

User: "status"

Router: "I want to make sure I understand. Are you trying to:

1. Check QuoxAgent fleet status
2. Get router statistics
3. Something else (let Claude help)

Click an option or rephrase your query for better results."

User escalation

Every routed response includes an "Ask Claude" button. This allows users to escalate to the full LLM when the router's response isn't sufficient.

Escalations are tracked for analytics - high escalation rates on a pattern indicate the handler needs improvement.

Telemetry & analytics

The router tracks detailed metrics for optimization:

• Bypass Rate: Percentage of queries handled without LLM
• Reformulation Rate: Users quickly rephrasing (indicates bad response)
• Escalation Rate: Users clicking "Ask Claude"
• Per-Pattern Stats: Usage, escalations, reformulations per pattern

Access stats with: router stats

Supported intents

The Smart Router currently supports 18 intents across different categories:

Infrastructure commands

Information queries

Memory & preferences

Utility

How it works

Here's the complete flow when a user sends a message:

1. Query Received: User sends "how's the fleet looking"
2. Layer 1 - Regex: Patterns are tested. No exact match found. Confidence: 0.0
3. Layer 2 - Semantic: Query is embedded and compared to 130+ examples. Top match: "fleet_status" with 0.72 confidence.
4. Merge Results: Semantic result wins (higher confidence).
5. Threshold Check: 0.72 > 0.65 (fleet_status threshold). Trusted.
6. Ambiguity Check: No close alternatives. Proceed.
7. Execute Handler: fleet_status handler runs, returns status.
8. Add Escalation: Response includes "Ask Claude" button.
9. Log Telemetry: Match recorded for analytics.

Configuration

The router configuration is in src/services/routerConfig.js:

// Router operating modes
ROUTER_MODES = {
  STANDARD: 'standard',     // Regex + Semantic, conservative routing
  ENHANCED: 'enhanced',     // Future: Add LLM classifier layer
  CONSERVATIVE: 'conservative'  // Higher thresholds, skip when uncertain
}

// Default configuration
DEFAULT_ROUTER_CONFIG = {
  mode: 'standard',

  thresholds: {
    confidence: 0.65,        // Default threshold
    ambiguity: 0.3,          // Max gap between top intents
    highConfidence: 0.9,     // Skip ambiguity check
    minimumConfidence: 0.4   // Below this, always pass to Claude
  },

  // Per-intent overrides
  intentThresholds: {
    greeting: 0.5,           // Low risk
    install_mha: 0.85,       // High risk - requires high confidence
    // ...
  },

  clarification: {
    enabled: true,
    minConfidence: 0.4,
    maxConfidence: 0.7,
    maxOptions: 3
  }
}

API reference

routeQuery(query, context)

Main entry point for routing a user query.

import { routeQuery } from './services/quoxRouter.js'

const result = await routeQuery('quoxagent status', { currentAssistant: 'quox' })

// Result:
{
  handled: true,
  response: "Fleet Status Report...",
  type: 'status',
  pattern: 'fleet_status',
  confidence: 0.95,
  layer: 'regex',
  canEscalate: true,
  escalateQuery: 'mha status'
}

canRouteQuery(query)

Quick synchronous check if a query might be routable.

import { canRouteQuery } from './services/quoxRouter.js'

const check = canRouteQuery('list hosts')
// { canRoute: true, pattern: 'list_hosts', description: 'List all known hosts' }

getPatterns()

Get all registered patterns for debugging.

import { getPatterns } from './services/quoxRouter.js'

const patterns = getPatterns()
// [{ name: 'install_mha', pattern: '/.../', description: '...' }, ...]

Telemetry

Export the logEscalation function for tracking user escalations:

import { logEscalation } from './services/quoxRouter.js'

// When user clicks "Ask Claude" button
logEscalation(originalQuery, patternName)