Monitoring with Prometheus

Prometheus is an open-source monitoring server developed under under the Cloud Native Computing Foundation.

Ozone supports Prometheus out of the box. The servers start a prometheus compatible metrics endpoint where all the available hadoop metrics are published in prometheus exporter format.

Prerequisites

  1. Install the and start an Ozone cluster.
  2. Download the prometheus binary.

Monitoring with prometheus

  • To enable the Prometheus metrics endpoint you need to add a new configuration to the ozone-site.xml file.
  1. <property>
  2. <name>hdds.prometheus.endpoint.enabled</name>
  3. <value>true</value>
  4. </property>

Note: for Docker compose based pseudo cluster put the
OZONE-SITE.XML_hdds.prometheus.endpoint.enabled=true line to the docker-config file.

  1. global:
  2. scrape_interval: 15s
  3. scrape_configs:
  4. - job_name: ozone
  5. metrics_path: /prom
  6. static_configs:
  7. - targets:
  8. - "scm:9876"
  9. - "ozoneManager:9874"
  • Start with prometheus from the directory where you have the prometheus.yaml file:
  1. prometheus
  • Check the active targets in the prometheus web-ui:

http://localhost:9090/targets

Prometheus target page example

  • Check any metrics on the prometheus web ui.
    For example:

http://localhost:9090/graph?g0.range\_input=1h&g0.expr=om\_metrics\_num\_key\_allocate&g0.tab=1

Prometheus metrics page example

Note

The ozone distribution contains a ready-to-use, dockerized environment to try out ozone and Prometheus. It can be found under compose/ozone directory.

  1. cd compose/ozone
  2. export COMPOSE_FILE=docker-compose.yaml:monitoring.yaml
  3. docker-compose up -d

Next >>