Kibana

Hydrolix includes an in-cluster Kibana integration that deploys Kibana, Kibana Gateway, and Elasticsearch together. It translates Elasticsearch queries to SQL and routes them to Hydrolix, making Hydrolix tables queryable as Elasticsearch indices.

This page covers installing, configuring, and deactivating the Hydrolix integration. For Kibana guidance, see the Kibana documentation.

Kibana Gateway was previously called Quesma

Hydrolix acquired the database proxy software from Quesma in September 2025. Older versions of Hydrolix may have artifacts containing the word quesma.

Before you begin

Ensure you have these prerequisites:

• Kubernetes cluster-admin privileges as described in the Kubernetes documentation
• The names of the Hydrolix tables you want to query in Kibana
• Sufficient resources in your Kubernetes cluster to accommodate new Kibana, Kibana Gateway, and Elasticsearch pods

Install Kibana Gateway

1. Install the Elasticsearch Custom Resource Definitions (CRD) and operator. With kubectl configured for your Hydrolix cluster, run these commands:

   kubectl create -f https://download.elastic.co/downloads/eck/2.15.0/crds.yaml
   kubectl apply -f https://download.elastic.co/downloads/eck/2.15.0/operator.yaml
   

   See the Elasticsearch documentation for details.

2. Edit the HydrolixCluster spec configuration to include the Kibana in-cluster integration.

   spec:
     data_visualization_tools:
       - Kibana
   

3. Wait for the pods to start. The elasticsearch, kibana-gateway, and kibana-kb pods can take several minutes to start. Verify the pods are running:

   kubectl get pods | grep -E 'elastic|kibana'
   

   Expected output showing Running status:

   elasticsearch-es-default-0            1/1     Running            3 (74m ago)       7h25m
   kibana-kb-85bdfdffd6-6x9n7            1/1     Running            0                 7h25m
   kibana-gateway-596b76b665-fhbpz       1/1     Running            0                 7h25m
   

4. Verify the Kibana web interface:

   1. Sign in to your Hydrolix cluster UI at https://hostname.hydrolix.live.
   2. Navigate to https://hostname.hydrolix.live/kibana.

Use Kibana and Kibana Gateway

The in-cluster Kibana integration uses the hydro.logs table as a data source by default.

1. Visit your Kibana installation at https://hostname.hydrolix.live/kibana.
2. Select Stack Management from the main menu.
3. Select Kibana > Data Views.
4. Create a data view:
   • If this is your first data source, select the link in the text "You can also create a data view against hidden, system, or default indices" at the bottom of the introduction page.
   • If this isn't your first data source, select the Create data view button.
5. Enter a name, index pattern, and timestamp in the form.
6. Select Save data view to Kibana.

Configure Kibana Gateway

Kibana Gateway can expose additional Hydrolix tables and communicate with Kibana instances running outside the cluster.

Additionally, it detects all summary tables in configured projects by querying the Hydrolix Config API. Once the integration is active, Kibana can query those summary tables without additional configuration. This feature was introduced in Hydrolix version 5.10.

Query additional projects and tables

This feature was introduced in Hydrolix version 5.9.

By default, Kibana Gateway exposes data in the hydro.logs table to Kibana. To expose additional tables, add their projects to the HydrolixCluster config spec. For example, to expose the hydro.monitor and cloudflare.logs tables in Hydrolix, the cluster configuration would be:

spec:
  kibana_gateway_config:
     projects: ["hydro", "cloudflare"]

Kibana Gateway automatically discovers all tables in the listed projects.

Table auto discovery limitation

Tables with name collisions across multiple projects are automatically ignored with a corresponding WARN log in the kibana-gateway container. For example, if both hydro and cloudflare happen to have a table called logs, neither would be exposed through auto discovery alone. See also Why differentiate the table name and index name.

Customize each table using the following fields in the HydrolixCluster config spec:

spec:
  kibana_gateway_config:
     projects: ["{project_name}", "{second_project_name}"]  # (list of strings) Hydrolix project names. Kibana Gateway discovers and exposes all tables in the listed projects as Elasticsearch indexes. Defaults to [].
     indexes:                         # (dict) Hierarchical index mapping for explicit per-table configuration. Each key is the Elasticsearch index name that Kibana sees.
        {index_name}:                           # Index name visible to Kibana. Can differ from the actual Hydrolix table name.
           project: {project_name}            # (string) The Hydrolix project (database) that contains the table.
           table: {table_name}       # (string) The Hydrolix table name within the project. Use when the index name differs from the actual table name. Defaults to the index name.
           ignore: true                 # (boolean) Hides a table from auto-discovery. Defaults to false.

Example configuration using the hydro.logs, sample_project.sample_table, and cloudflare.logs tables:

spec:
  kibana_gateway_config:
    projects: ["hydro", "sample_project", "cloudflare"]
    indexes:
      hydrologs:
        project: hydro
        table: logs
      sample_table:
        project: sample_project
        table: sample_table
        ignore: true
      cflogs:
        project: cloudflare
        table: logs

Why differentiate the table name and index name

Mapping table names to different index names can resolve table name collisions across projects. When accessing multiple projects using Kibana, two different projects can have tables with the same name such as hydro.logs and visordb.logs. In Elasticsearch/Kibana, index names must be unique, so you can't have two indexes both called logs. The index-name-to-table-name mapping lets you present them under distinct names:

spec:
  kibana_gateway_config:
    indexes:
      hydrologs:              # Kibana sees this as "hydrologs"
        project: hydro
        table: logs           # actual Hydrolix table is hydro.logs
      visorlogs:              # Kibana sees this as "visorlogs"
        project: visordb
        table: logs           # actual Hydrolix table is visordb.logs

Enable public access to Kibana Gateway

To allow external clients, such as Kibana running outside the cluster, to communicate with Kibana Gateway, enable public access using this configuration:

spec:
  kibana_gateway_config:
    enable_public_access: true
  issue_wildcard_cert: true

This configuration exposes Kibana Gateway at kibana-gateway.hostname.hydrolix.live. The hdx-elastic-user Kubernetes secret holds the username and password. See Managing Secrets using kubectl for more information.

Deactivate Kibana

To remove the Kibana integration:

1. Edit your hydrolixcluster.yaml file and remove the data_visualization_tools and kibana_gateway_config sections.

2. Apply the configuration changes:

   kubectl apply -f hydrolixcluster.yaml
   

3. Remove the Elasticsearch CRDs and operator:

   kubectl delete -f https://download.elastic.co/downloads/eck/2.15.0/crds.yaml
   kubectl delete -f https://download.elastic.co/downloads/eck/2.15.0/operator.yaml