Metrics

OpenViking provides a machine-oriented metrics system for exposing runtime health, request quality, model usage, resource processing throughput, and probe health states.

Unlike the human-facing /api/v1/observer/* endpoints and the analytics-oriented /api/v1/stats/* endpoints, Metrics are designed for:

• high-frequency scraping by Prometheus, Grafana Agent, and similar systems
• low-cardinality, aggregatable metric models
• monitoring, alerting, capacity observation, and regression diagnosis

Overview

Why Metrics

Metrics are well suited to answer questions like:

• Has HTTP traffic increased abnormally over the last few minutes?
• Are resource ingestion, retrieval, or model calls getting slower?
• Is there queue backlog?
• Are key dependencies such as storage, model providers, VikingDB, encryption, and async systems currently healthy?
• Is a specific tenant showing abnormal traffic or error rates?

Compared with logs and observer snapshots, metrics are better for:

• continuous scraping
• time-series aggregation
• dashboard visualization
• alert rules

How Metrics Differ from Observer and Stats

The boundary is:

• /metrics only carries low-cardinality, low-cost metrics
• /api/v1/stats/* continues to carry analytics-oriented statistics without being constrained by the Prometheus scraping model

Metrics Architecture

The current metrics stack in OpenViking has four layers:

Business logic / HTTP requests / background tasks
          │
          ▼
      DataSource
   (event emission / state reads)
          │
          ▼
      Collector
 (semantic routing + labels)
          │
          ▼
    MetricRegistry
   (in-process metric store)
          │
          ▼
      Exporter
 (Prometheus text rendering)
          │
          ▼
       /metrics

DataSource

DataSources provide inputs to the metrics system in two main forms:

• Event-based: business code emits events at key points, such as retrieval completion, successful model calls, or resource ingestion stage completion
• Read-based: current state is read before /metrics export, such as queue state, lock state, or probe state

Collector

Collectors turn inputs into metric semantics:

• choose which metric to write
• choose which labels to attach
• define how failure is exposed, such as valid=1/0

MetricRegistry

The MetricRegistry is the in-process metric store that keeps the current metric values and serves them to the exporter.

Exporter

The first exporter implementation is the Prometheus exporter, which renders registry contents into Prometheus exposition text.

Usage

Accessing /metrics

In the current implementation, /metrics is not wired to get_request_context or other auth dependencies, so from the code-path perspective it currently behaves as a public scrape endpoint.

curl http://localhost:1933/metrics

If your deployment protects /metrics at the gateway, reverse proxy, or service discovery layer, attach auth according to the deployment environment.

Prometheus Scrape Example

scrape_configs:
  - job_name: openviking
    metrics_path: /metrics
    static_configs:
      - targets: ["localhost:1933"]

Understanding Common Labels

Notes:

• account_id is only enabled on controlled allowlisted metric families to prevent high-cardinality growth
• valid=0 means the current state/probe sample is a fallback or stale value, not that the label itself is malformed
• stage semantics depend on the metric family:
  • openviking_resource_stage_*: resource ingestion pipeline stages (for example parse/persist/process)
  • openviking_operation_tokens_total: token attribution stages (for example embed_query/rerank/vlm)

Key Metric Families

The metric summaries below are based on representative metrics currently exposed by the collectors in openviking/metrics/collectors/.

Requests and Operations

Typical usage:

• inspect whether /api/v1/search/find or /api/v1/resources is slowing down
• inspect whether a specific operation has elevated error rates

Retrieval and Resource Processing

Typical stage values include:

• request
• parse
• summarize
• persist
• finalize
• process

Vector, Memory, and Semantic Metrics

Model Calls and Tokens

Notes:

• openviking_model_* gives a unified cross-model view for embedding and VLM usage
• openviking_vlm_* and openviking_embedding_* are better suited for workload-specific dashboards

Queues, Locks, and Runtime State

These help answer:

• Is there queue backlog?
• Is there lock contention or stale locking?

Tasks and Task Tracker

Cache

Session

Probes and Health State

Meaning of valid:

• valid="1": the sample was produced by a successful refresh
• valid="0": the sample is a fallback or stale value and should be treated with caution

Encryption (Operational Metrics)

Component and Observer Aggregate Metrics

Typical component values include:

• queue
• models
• lock
• retrieval
• vikingdb

VikingDB and Model Usage Statistics

Possible model_type values include:

• vlm
• embedding
• rerank

Configuration Example

Enabling Metrics

In ov.conf, the metrics subsystem can be explicitly enabled through server.observability.metrics:

{
  "server": {
    "observability": {
      "metrics": {
        "enabled": true,
        "account_dimension": {
          "enabled": true,
          "max_active_accounts": 100,
          "metric_allowlist": [
            "openviking_http_requests_total",
            "openviking_http_request_duration_seconds",
            "openviking_http_inflight_requests",
            "openviking_operation_requests_total",
            "openviking_operation_duration_seconds",
            "openviking_vlm_calls_total",
          "openviking_vlm_call_duration_seconds",
          "openviking_rerank_*"
          ]
        }
      }
    }
  }
}

Recommended mental model:

• server.observability.metrics.enabled: master switch for the metrics subsystem
• server.observability.metrics.account_dimension: controls whether account_id labels are enabled and where they are allowed

Exporters

By default, OpenViking exports metrics via Prometheus exposition format at /metrics. You can also enable additional exporters under server.observability.metrics.exporters.

Key fields:

• server.observability.metrics.exporters.prometheus.enabled: enable the Prometheus exporter (serves /metrics)
• server.observability.metrics.exporters.otel.enabled: enable OTLP export from the same in-process registry
• server.observability.metrics.exporters.otel.protocol: "grpc" or "http"
• server.observability.metrics.exporters.otel.tls.insecure: OTLP/gRPC only; true means plaintext (no TLS)
• server.observability.metrics.exporters.otel.endpoint: OTLP endpoint (for gRPC, use host:4317; for HTTP, use a full URL)
• server.observability.metrics.exporters.otel.service_name: OTLP service.name resource attribute (default "openviking-server")
• server.observability.metrics.exporters.otel.export_interval_ms: OTLP push interval in milliseconds (default 10000)
• server.observability.metrics.exporters.otel.headers: optional custom OTLP headers; sent as gRPC metadata for gRPC and HTTP headers for HTTP
• When using gRPC, header keys in headers should be lowercase, for example x-byteapm-appkey; HTTP does not have this restriction

Example:

{
  "server": {
    "observability": {
      "metrics": {
        "enabled": true,
        "exporters": {
          "prometheus": {
            "enabled": true
          },
          "otel": {
            "enabled": true,
            "protocol": "grpc",
            "tls": {
              "insecure": true
            },
            "endpoint": "otel-collector:4317",
            "service_name": "openviking-server",
            "export_interval_ms": 10000,
            "headers": {}
          }
        }
      }
    }
  }
}

Recommended account_id Usage

• enabled by default, but only allowlisted metric families will receive tenant ids (empty allowlist still yields __unknown__)
• do not turn user_id, session_id, or resource_uri into labels
• only enable tenant dimensions on a small set of critical dashboard and alert metrics
• metric_allowlist supports a limited wildcard syntax: only trailing * prefix matches (e.g. openviking_rerank_*, openviking_embedding_*)
• a standalone * is not supported, nor full glob/regex patterns

Related Documentation

• Architecture Overview - overall OpenViking architecture
• Multi-Tenant - account/user/agent isolation model
• Data Encryption - storage-layer encryption and isolation
• Metrics API - /metrics endpoint usage
• Metrics Design - metrics system design details