Skip to content

Monitoring.md

Latest commit

 

History

History
54 lines (45 loc) · 2.01 KB

Monitoring.md

File metadata and controls

54 lines (45 loc) · 2.01 KB
Raw

Monitoring

Colonies has built-in support for Prometheus instrumentation through a standalone monitoring server, started with colonies monitor start. The monitoring server connects to a Colonies server, periodically collects statistics, and exposes them as Prometheus metrics on the port specified by the COLONIES_MONITOR_PORT environment variable. Note that colonies server start does not start a monitoring server by itself.

export COLONIES_SERVER_HOST="localhost"
export COLONIES_SERVER_HTTP_PORT="50080"
export COLONIES_MONITOR_PORT="21120"
export COLONIES_MONITOR_INTERVAL="1"
export COLONIES_SERVER_ID="<server-id>"
export COLONIES_SERVER_PRVKEY="<server-private-key>"
export COLONIES_TLS="false"

colonies monitor start

The server id and private key can be generated with colonies security generate, or use the values from the .env file created by ./scripts/generate-dev-env.sh. Setting COLONIES_TLS="false" is what makes the monitor connect to the Colonies server without TLS.

Prometheus configuration

Use the following prometheus.yml to configure Prometheus.

global:
  scrape_interval: 15s
  external_labels:
    monitor: 'codelab-monitor'

scrape_configs:
  - job_name: 'colonies'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:21120']

Start a Prometheus and a Grafana server

docker run -p 9090:9090 -v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
docker run -p 3000:3000 grafana/grafana-oss:latest-ubuntu

Log in to Grafana (http://localhost:3000) and add the Prometheus server as a new datasource.

Metrics

The following metrics are exported:

  • colonies_server_colonies
  • colonies_server_executors
  • colonies_server_processes_waiting
  • colonies_server_processes_running
  • colonies_server_processes_successful
  • colonies_server_processes_failed
  • colonies_server_workflows_waiting
  • colonies_server_workflows_running
  • colonies_server_workflows_successful
  • colonies_server_workflows_failed

Grafana