telemetry/README.md
A privacy-respecting telemetry system for SeaweedFS that collects cluster-level usage statistics and provides visualization through Prometheus and Grafana.
SeaweedFS Cluster → Telemetry Client → Telemetry Server → Prometheus → Grafana
(protobuf) (metrics) (queries)
The telemetry system uses Protocol Buffers exclusively for efficient binary data transmission:
message TelemetryData {
string cluster_id = 1; // In-memory generated UUID
string version = 2; // SeaweedFS version
string os = 3; // Operating system
// Field 4 reserved (was features)
// Field 5 reserved (was deployment)
int32 volume_server_count = 6; // Number of volume servers
uint64 total_disk_bytes = 7; // Total disk usage
int32 total_volume_count = 8; // Total volume count
int32 filer_count = 9; // Number of filer servers
int32 broker_count = 10; // Number of broker servers
int64 timestamp = 11; // Collection timestamp
}
-telemetry=false on the master stops all reporting| Field | Description | Example |
|---|---|---|
cluster_id | In-memory UUID (changes on restart) | a1b2c3d4-... |
version | SeaweedFS version | 3.45 |
os | Operating system and architecture | linux/amd64 |
volume_server_count | Number of volume servers | 5 |
total_disk_bytes | Total disk usage across cluster | 1073741824 |
total_volume_count | Total number of volumes | 120 |
filer_count | Number of filer servers | 2 |
broker_count | Number of broker servers | 1 |
timestamp | When data was collected | 1640995200 |
A master starts reporting once its cluster stores at least 10 GiB, and the server drops reports under that floor. Every fresh weed server, CI job and throwaway container mints its own cluster id, and at tens of thousands a day they were nearly all of the counted clusters and almost none of the bytes.
# Clone and start the complete monitoring stack
git clone https://github.com/seaweedfs/seaweedfs.git
cd seaweedfs
docker compose -f telemetry/docker-compose.yml up -d
# Or run the server directly
cd telemetry/server
go run . -port=8080 -dashboard=true
# Reporting to telemetry.seaweedfs.com is on by default
weed master
# Send to your own telemetry server instead
weed master -telemetry.url=http://localhost:8080/api/collect
# Turn reporting off
weed master -telemetry=false
weed server -master.telemetry=false
# Disable telemetry (enabled by default)
-telemetry=false
# Set custom telemetry server URL (optional, defaults to telemetry.seaweedfs.com)
-telemetry.url=http://your-telemetry-server:8080/api/collect
In weed server and weed mini the flags are prefixed: -master.telemetry=false and -master.telemetry.url=....
# Server configuration
-port=8080 # Server port
-dashboard=true # Enable built-in dashboard
-cleanup=24h # Cleanup interval
-max-age=2160h # Maximum data retention (90 days)
-state-file=data/telemetry-state.json # Persist state across restarts (empty to disable)
-state-save=1h # How often to save changed state
# Example
./telemetry-server -port=8080 -dashboard=true -cleanup=24h -max-age=2160h
The telemetry server exposes these Prometheus metrics:
seaweedfs_telemetry_total_clusters: Total unique clusters (30 days)seaweedfs_telemetry_active_clusters: Active clusters (7 days)seaweedfs_telemetry_confirmed_clusters: Active clusters seen on 7+ distinct days — one-shot reports don't count, and the version/OS distributions in /api/stats are computed over theseseaweedfs_telemetry_volume_servers{cluster_id}: Volume servers per clusterseaweedfs_telemetry_disk_bytes{cluster_id}: Disk usage per clusterseaweedfs_telemetry_volume_count{cluster_id}: Volume count per clusterseaweedfs_telemetry_filer_count{cluster_id}: Filer servers per clusterseaweedfs_telemetry_broker_count{cluster_id}: Broker servers per clusterseaweedfs_telemetry_cluster_info{cluster_id, version, os}: Cluster metadataValue gauges are keyed by cluster_id only, so a cluster keeps one continuous
series across upgrades. To slice values by version or OS, join with
cluster_info, e.g.
seaweedfs_telemetry_disk_bytes * on(cluster_id) group_left(version, os) seaweedfs_telemetry_cluster_info.
seaweedfs_telemetry_reports_received_total: Total telemetry reports receivedseaweedfs_telemetry_reports_skipped_total: Reports dropped because the cluster stores less than 10 GiB# Submit telemetry data (protobuf only)
POST /api/collect
Content-Type: application/x-protobuf
[TelemetryRequest protobuf data]
# Get aggregated statistics
GET /api/stats
# Get recent cluster instances
GET /api/instances?limit=100
# Get metrics over time
GET /api/metrics?days=30
# Get one cluster's daily usage history (disk bytes, volumes, volume servers)
GET /api/history?cluster_id=<uuid>&days=90
# Get per-cluster disk usage and volume servers over time, largest first,
# the rest summed as "other"
GET /api/cluster-sizes?days=30&limit=20
# Get how many clusters ran each version over time, oldest version first,
# the rest summed as "other"
GET /api/versions?days=30&limit=8
# Prometheus metrics
GET /metrics
# docker-compose.yml
version: '3.8'
services:
telemetry-server:
build:
context: ../
dockerfile: telemetry/server/Dockerfile
ports:
- "8080:8080"
command: ["-port=8080", "-dashboard=true", "-cleanup=24h"]
prometheus:
image: prom/prometheus:latest
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
grafana:
image: grafana/grafana:latest
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
- ./grafana-provisioning:/etc/grafana/provisioning
- ./grafana-dashboard.json:/var/lib/grafana/dashboards/seaweedfs.json
# Deploy the stack
docker compose -f telemetry/docker-compose.yml up -d
# Scale telemetry server if needed
docker compose -f telemetry/docker-compose.yml up -d --scale telemetry-server=3
# Build and run telemetry server (build from repo root to include all sources)
docker build -t seaweedfs-telemetry -f telemetry/server/Dockerfile .
docker run -p 8080:8080 seaweedfs-telemetry -port=8080 -dashboard=true
# Generate protobuf code
cd telemetry
protoc --go_out=. --go_opt=paths=source_relative proto/telemetry.proto
# The generated code is already included in the repository
# Build telemetry server
cd telemetry/server
go build -o telemetry-server .
# Build SeaweedFS with telemetry support
cd ../..
go build -o weed ./weed
# Test telemetry server
cd telemetry/server
go test ./...
# Test protobuf communication (requires protobuf tools)
# See telemetry client code for examples
The included Grafana dashboard provides:
# Total active clusters
seaweedfs_telemetry_active_clusters
# Disk usage per cluster
seaweedfs_telemetry_disk_bytes{cluster_id="<uuid>"}
# Disk usage by version (join with cluster_info for version/os)
sum by (version) (seaweedfs_telemetry_disk_bytes * on(cluster_id) group_left(version) seaweedfs_telemetry_cluster_info)
# Volume servers by operating system
sum by (os) (seaweedfs_telemetry_volume_servers * on(cluster_id) group_left(os) seaweedfs_telemetry_cluster_info)
# Broker servers across all clusters
sum(seaweedfs_telemetry_broker_count)
# Growth rate (weekly)
increase(seaweedfs_telemetry_total_clusters[7d])
SeaweedFS not sending data:
# Check telemetry configuration
weed master -h | grep telemetry
# Verify connectivity
curl -v http://your-telemetry-server:8080/api/collect
Server not receiving data:
# Check server logs
docker-compose logs telemetry-server
# Verify metrics endpoint
curl http://localhost:8080/metrics
Prometheus not scraping:
# Check Prometheus targets
curl http://localhost:9090/api/v1/targets
# Verify configuration
docker-compose logs prometheus
# Enable verbose logging in SeaweedFS
weed master -v=2 -telemetry=true
# Check telemetry server metrics
curl http://localhost:8080/metrics | grep seaweedfs_telemetry
# Test data flow
curl http://localhost:8080/api/stats
This telemetry system is part of SeaweedFS and follows the same Apache 2.0 license.