Set up Kibana

Hydrolix includes an in-cluster Kibana integration that deploys Kibana, Kibana Gateway, and Elasticsearch together. It translates Elasticsearch queries to SQL and routes them to Hydrolix, making Hydrolix tables queryable as Elasticsearch indices.

This page covers enabling, configuring, and deactivating the integration on a cluster. It's intended for cluster administrators. To access and query Kibana once it's enabled, see Use Kibana.

Kibana Gateway was previously called Quesma

Hydrolix acquired the database proxy software from Quesma in September 2025. Older versions of Hydrolix may have artifacts containing the word quesma.

Before you begin

Ensure you have these prerequisites:

• Kubernetes cluster-admin privileges as described in the Kubernetes documentation
• The names of the Hydrolix tables you want to query in Kibana
• Sufficient resources in your Kubernetes cluster to accommodate new Kibana, Kibana Gateway, and Elasticsearch pods

Install Kibana Gateway

1. Install the Elasticsearch Custom Resource Definitions (CRD) and operator. With kubectl configured for your Hydrolix cluster, run these commands:

   kubectl create -f https://download.elastic.co/downloads/eck/2.15.0/crds.yaml
   kubectl apply -f https://download.elastic.co/downloads/eck/2.15.0/operator.yaml
   

   See the Elasticsearch documentation for details.

2. Enable the integration by adding Kibana to the operator's data_visualization_tools array field in the HydrolixCluster spec. The operator deploys the Kibana, Kibana Gateway, and Elasticsearch pods only when Kibana is present in this array.

   spec:
     data_visualization_tools:
       - Kibana
   

3. Wait for the pods to start. The elasticsearch, kibana-gateway, and kibana-kb pods can take several minutes to start. Verify the pods are running:

   kubectl get pods | grep -E 'elastic|kibana'
   

   Expected output showing Running status:

   elasticsearch-es-default-0            1/1     Running            3 (74m ago)       7h25m
   kibana-kb-85bdfdffd6-6x9n7            1/1     Running            0                 7h25m
   kibana-gateway-596b76b665-fhbpz       1/1     Running            0                 7h25m
   

Once the pods are running, see Use Kibana to access Kibana and create data views.

Configure Kibana Gateway

Kibana Gateway can expose additional Hydrolix tables and communicate with Kibana instances running outside the cluster.

Additionally, it detects all summary tables in configured projects by querying the Hydrolix Config API. Once the integration is active, Kibana can query those summary tables without additional configuration, the same as any other index. See Use Kibana. This feature was introduced in Hydrolix version 5.10.

Query additional projects and tables

This feature was introduced in Hydrolix version 5.9.

By default, Kibana Gateway exposes data in the hydro.logs table to Kibana. To expose additional tables, add their projects to the HydrolixCluster config spec. For example, to expose the hydro.monitor and cloudflare.logs tables in Hydrolix, the cluster configuration would be:

spec:
  kibana_gateway_config:
    projects:
      - hydro
      - cloudflare

Kibana Gateway discovers all tables in the listed projects.

Table auto discovery limitation

Auto discovery ignores identically-named tables in different projects. It writes a WARN log in the kibana-gateway container for the colliding table names and doesn't expose any corresponding Elasticsearch index. For example, if both hydro and cloudflare projects hold a table called logs, neither is exposed.

To expose a table with a colliding name, create a mapping to a unique index name in the indexes config.

Customize each table using the following fields in the HydrolixCluster config spec:

spec:
  kibana_gateway_config:
    projects:                        # Kibana Gateway discovers and exposes all tables in the listed projects as Elasticsearch indexes. Defaults to ["hydro"]
      - {project_name}               # (string) Hydrolix project name
      - {second_project_name}        #
    indexes:                         # (dict) Hierarchical index mapping for explicit per-table configuration. Each key is the Elasticsearch index name that Kibana sees.
      {index_name}:                  # (string) Index name visible to Kibana. Can differ from the actual Hydrolix table name.
        project: {project_name}      # (string) The Hydrolix project (database) that contains the table.
        table: {table_name}          # (string) The Hydrolix table name in the project. Use when the index name differs from the actual table name. Defaults to the index name.
        ignore: true                 # (boolean) Hides a table from auto-discovery. Defaults to false.

Example configuration using the hydro.logs, sample_project.sample_table, and cloudflare.logs tables:

spec:
  kibana_gateway_config:
    projects:
      - hydro
      - sample_project
      - cloudflare
    indexes:
      hydrologs:
        project: hydro
        table: logs
      sample_table:
        project: sample_project
        table: sample_table
        ignore: true
      cflogs:
        project: cloudflare
        table: logs

See Why differentiate the table name and index name to understand why the logs table is mapped to a cflogs index rather than a logs index.

Tunables

The Kibana Gateway supports additional configuration properties.

default_query_time_range

Type: string | Added v6.0

The default time range applied to queries that don't already specify one. This property sets the global default for all tables unless a table overrides it with the query_time_range tunable.

Default: 24h

Example

spec:
  kibana_gateway_config:
     default_query_time_range: 12h

---

query_time_range

Type: string | Added v6.0

A table-level override for default_query_time_range. When set on a specific table, this value takes precedence over the global default for that table. Like default_query_time_range, it applies only to queries that don't already specify a time range.

Default: null (falls back to default_query_time_range)

Example

spec:
  kibana_gateway_config:
    default_query_time_range: 24h
    indexes:
      my_project.my_table:
         query_time_range: 7d

---

Why differentiate the table name and index name

When two or more projects contain tables with the same name, such as hydro.logs and akamai.logs, you must give each table a unique index name in the indexes config. In Elasticsearch and Kibana, index names must be unique, so two logs tables can't both appear as a logs index, and auto discovery drops both unless you disambiguate them. Map each table to a distinct index name to present them under separate names:

spec:
  kibana_gateway_config:
    indexes:
      hydrologs:              # Kibana sees this as "hydrologs"
        project: hydro
        table: logs           # actual Hydrolix table is hydro.logs
      akamailogs:             # Kibana sees this as "akamailogs"
        project: akamai
        table: logs           # actual Hydrolix table is akamai.logs

Enable public access to Kibana Gateway

To allow external clients, such as Kibana running outside the cluster, to communicate with Kibana Gateway, enable public access using this configuration:

spec:
  kibana_gateway_config:
    enable_public_access: true
  issue_wildcard_cert: true

This configuration exposes Kibana Gateway at kibana-gateway.hostname.hydrolix.live. The hdx-elastic-user Kubernetes secret holds the username and password. See Managing Secrets using kubectl for more information.

Deactivate Kibana

To remove the Kibana integration:

1. Edit your hydrolixcluster.yaml file and remove the data_visualization_tools and kibana_gateway_config sections.

2. Apply the configuration changes:

   kubectl apply -f hydrolixcluster.yaml
   

3. Remove the Elasticsearch CRDs and operator:

   kubectl delete -f https://download.elastic.co/downloads/eck/2.15.0/crds.yaml
   kubectl delete -f https://download.elastic.co/downloads/eck/2.15.0/operator.yaml