Grafana Ops Dashboards

Preparing the data source

1. Visit your Grafana instance.

2. Select ⚙️ Configuration from the left menu bar, and then select Data Sources.

3. Click the Add data source button.

4. Select Prometheus from the list of compatible data sources.

5. Enter the name hdx-monitoring-prometheus in the field.

6. Enter https://YOUR-HYDROLIX-HOSTNAME.hydrolix.live/prometheus as the data source's URL.

Finally, click Save & Test. The message "Data source is working" should appear immediately, completing this setup.

If you do not see that success message, confirm that your provided URL is correct, and that your stack has at least one prometheus service running. If you still have trouble getting Grafana to connect to Prometheus, please contact Hydrolix support.

Importing Monitoring Dashboard

Hydrolix has deployed built-in monitoring dashboard for our different services in Grafana community dashboard.

You can easily add those dashboards in your Grafana deployment:

1. Visit your stack's Grafana instance.

2. Select + Create from the left menu bar, and then select Import.

3. Specify Hydrolix Dashboard ID and click on Load

4. Hydrolix has several dashboards ID deployed:

   1. 16154 For HTTP Streaming Ingest Monitoring
   2. 14444 For Kafka Ingest Monitoring
   3. 17844 For Kinesis Ingest Monitoring
   4. 14442 For Table / Partitions Monitoring
   5. 17165 For an Overview of your Hydrolix logs
   6. 17166 For Query Performance Monitoring
   7. 17916 For Merge Monitoring

When you import the dashboard it'll requires to select the datasources you should use:

• hdx-monitoring-prometheus for prometheus
• hdx-monitoring-query for clickhouse (for Active Queries and table montoring).

Based on your deployment you may want specific dashboards or all of them.

Querying Grafana

After setting up Prometheus as a data source, you can query, graph, and monitor your Hydrolix metrics through all the tools and techniques Grafana makes available.

For a simple example, select 🧭 Explore from Grafana's left menu bar, and then enter a basic metric-query such as process_open_fds into the text entry field. This results in a multi-line graph showing the open file descriptors in use by several of your stack's components.

To turn this static graph into a dynamic monitor, select 5s from the pull-down menu next to the Run query button. The graph then refreshes itself every five seconds.

For more information on using Grafana and Prometheus together, you may consult Prometheus's documentation on that topic.