Back to Activepieces

BullMQ Prometheus Metrics

docs/handbook/engineering/playbooks/queue-metrics.mdx

0.86.31.9 KB
Original Source

The app exposes per-queue BullMQ job counts in Prometheus text format so a scraper can track queue depth by state (waiting, active, completed, failed, delayed, paused).

Endpoint (self-hosted only)

GET /v1/worker-machines/queue-metrics/prometheus/:queueName
  • Available on self-hosted Community and Enterprise. Not registered on Cloud.
  • :queueName is optional and defaults to workerJobs (the shared queue). Pass a worker-group queue name (e.g. platform-<groupId>-jobs) to scrape a dedicated queue.
  • Auth: Authorization: Bearer <platform API key> (an sk-... key created under platform admin). Platform admin user tokens work too.
  • Returns text/plain. Responds 404 if the queue has not been initialized on this app instance.

Example

bash
curl -H "Authorization: Bearer $PLATFORM_API_KEY" \
  https://your-instance.example.com/api/v1/worker-machines/queue-metrics/prometheus/workerJobs
# HELP bullmq_job_count Number of jobs in the queue by state
# TYPE bullmq_job_count gauge
bullmq_job_count{queue="workerJobs", state="waiting"} 0
bullmq_job_count{queue="workerJobs", state="active"} 3
bullmq_job_count{queue="workerJobs", state="completed"} 0
bullmq_job_count{queue="workerJobs", state="failed"} 1
bullmq_job_count{queue="workerJobs", state="delayed"} 0
<Note> Bearer auth works with standard scraper config: Prometheus `authorization.credentials`, Azure managed Prometheus `bearer_token_file`, Grafana Alloy `bearer_token`. No header-injecting proxy is needed. </Note>

Metrics come from BullMQ's built-in Queue.exportPrometheusMetrics(). Each queue only reports the states it holds on the app instance being scraped; in a multi-worker-group deployment, scrape each group's queue by name (platform-<groupId>-jobs / project-<groupId>-jobs). A JSON variant with waiting/active counts only lives at GET /v1/worker-machines/queue-metrics (all editions, used for KEDA autoscaling).