Prometheus Integration

Overview

Prometheus in Hydrolix

The Hydrolix stack includes Prometheus, an open-source metrics database. Hydrolix continuously updates its Prometheus instance with metrics information.

You can query, view, and actively monitor this information using a stack's Grafana instance, or you can access it with your own monitoring platform. See Prometheus Integration for more information about setting up Prometheus with an external server.

View metrics with Prometheus

Access Prometheus directly in a web browser, or use an external Prometheus server.

Use the Prometheus web interface

Use the Prometheus UI

Prometheus has its own web-based UI.

This view is a basic metric view, suitable for quickly entering queries and seeing simple, graphed results. This feature is available in Hydrolix without any additional setup.

Navigate to https://<YourHostname>/prometheus to view dashboards.

For more information about Prometheus metric types, refer to the Prometheus documentation.

Use an external Prometheus server

You can use an external Prometheus server to display metrics. To do this, configure the internal server to forward metrics to your external server, and enable remote writing on the external server.

This example uses the Prometheus Remote Write Server functionality to link the two servers.

In these examples, Hydrolix runs its Prometheus process, then forwards the metrics to your external Prometheus server for querying. It uses memory and CPU resources to aggregate metrics.

To set up an external Prometheus server:

1. Verify your Prometheus server hostname and port. The default port is 9090.

2. Open the hydrolixcluster.yaml file and include this line in the spec section. Edit to include your Prometheus hostname and port.

       spec:
       ...
       prometheus_remote_write_url: http://<prometheus server hostname>:9090/api/v1/write
   

3. Configure your Prometheus server to run with this flag.

   Shell

      ./prometheus --web.enable-remote-write-receiver
   

To set up an external Prometheus server using basic auth:

1. If the external Prometheus server uses basic auth, set the username in the hydrolixcluster.yaml file and set the password in a curated secret.

2. Edit the hydrolixcluster.yaml file and add this line to the spec section: prometheus_remote_write_username: <username>.

3. Apply the changes to the Hydrolix cluster.

      kubectl -f hydrolixcluster.yaml apply
   

4. Create a Prometheus secret YAML file named prom-secret.yaml.

       ---
       apiVersion: v1
       kind: Secret
       metadata:
         name: curated
         namespace: $HDX_KUBERNETES_NAMESPACE
       stringData:
         PROMETHEUS_REMOTE_WRITE_PASSWORD: <password>
       type: Opaque
   

5. Use eval to interpolate the $HDX_KUBERNETES_NAMESPACE variable and apply the secret to your Kubernetes cluster with kubectl.

     eval "echo \"$(cat prom-secret.yaml)\"" > secrets.yaml
     kubectl apply -f prom-secret.yaml
   

📘 Note

Using an external Prometheus server is not the same as Prometheus Remote Read/Write.

The Prometheus Remote Read and Write feature uses Hydrolix to serve as the database for a Prometheus installation. This offers longer retention and cost savings for large volumes of data.

For a complete list of the metrics used by Hydrolix, including Prometheus, RabbitMQ, and others, see All Metrics.