Grafana Automatic Installation

Goal

Enable a working Grafana instance in your Hydrolix cluster using the automatically configured Grafana service.

Overview of steps

Modify the Hydrolix configuration
Obtain your Grafana administrative password
View Grafana interface

Before you begin

Make sure you're runinng Hydrolix version 5.1 or greater
Havekubectl and/or k9s configured for your Hydrolix instance
Ensure you have at least 6 GB memory and 6 CPUs available in your Kubernetes cluster

Installation

Modify the Hydrolix configuration

Edit your Hydrolix cluster's configuration to include the Grafana in-cluster integration.

Add these two lines to the spec: section of your Hydrolix configuration
```
  data_visualization_tools:
    - grafana
```
For example, here's what it could look like in your hydrolixcluster.yaml file.
```
spec:
  acme_enabled: true
  admin_email: [email protected]
  data_visualization_tools:
    - grafana
  db_bucket_endpoint: https://us-sea-1.linodeobjects.com
  db_bucket_region: us-sea-1
```
Load the configuration changes into your Hydrolix cluster with kubectl apply -f hydrolixcluster.yaml or by saving your changes in k9s.
Wait for the Grafana Kubernetes pod to start. This can take several minutes. You can tell when it's ready for use by spotting the Running indicator in the output of the kubectl command.
```
% kubectl get pods | grep -E 'grafana'       

grafana-864958596c-wnv6x                 2/2     Running     0               8m37s  
```
Verify the web interface is working.
1. Make sure you're logged into your Hydrolix cluster's UI by visiting https://{hostname}.hydrolix.live in a web browser.
2. Visit https://{hostname}.hydrolix.live/grafana in your web browser to see Kibana's Web UI. You should be directed to a login page.

If the UI isn't available after a few minutes, see the troubleshooting section below.

Obtain and use your Grafana administrative password

The built-in Grafana instance comes pre-configured with an automatically generated administrative username and password.

Using k9s
1. K9s, in “:secrets”
2. General, decode with ‘x’
3. Find the value of GRAFANA_ADMIN_PASSWORD

Using kubectl

% kubectl get secret general -o jsonpath="{.data.GRAFANA_ADMIN_PASSWORD}" | base64 --decode

View the Grafana Interface

Visit https://{hostname}.hydrolix.live/grafana in your web browser.
Use the admin username and the password you decoded in the last step.

Verification

Verify metrics are flowing into Grafana by viewing them on Grafana's Explore page.

Visit http://{hostname}.hydrolix.live/grafana/explore in your web browser.
Use the Metric drop-down menu to select a metric to view.
Select the Run query button in the upper right corner.

The value of the metric should be displayed on a graph and in raw form below.

Troubleshooting

Not enough resources

If the grafana-* pod isn't starting and the kubectl or k9s utility is showing the pod in a pending state, check the resources on the nodes backing your Hydrolix installation. It's possible to lower the amount of memory so Grafana (and its associated renderer service) doesn't request as much from Kubernetes. More information about scaling pods and services can be found in Scale your Cluster. Here's a sample of a low-overhead configuration in the Hydrolix spec file:

spec:
  scale:
    grafana:
      cpu: 2
      memory: 2Gi
    renderer:
      cpu: 1
      memory: 1Gi

Postgres not available

Grafana uses the same Postgres instance as the Hydrolix catalog database, storing Grafana configuration data in a separate database called grafana. This is where Grafana dashboard definitions, users, alerts, and query history are stored.

The configuration provided by Hydrolix assumes that this Postgres instance is only available through SSL connections. If you're using an insecure configuration of Postgres, the built-in Grafana instance will refuse to connect.

If you're unable to use an SSL-enabled Postgres, perhaps for development or demo purposes, configure Grafana to make non-SSL connections with the kubectl utility.

Set pg_ssl_mode to disable
```
kubectl edit hdx
```
Use the file editor to add this line to the Hydrolix spec:
```
pg_ssl_mode: disable
```
Save the file.
Disable Grafana, then re-enable Grafana by removing, then adding lines to the Hydrolix spec:

data_visualization_tools:
- grafana