Documentation
ProductPricingDocsBlog

Enable TLS

Suggest Edits

Overview

Hydrolix provides multiple ways to manage certificates on your cluster:

  • Generate and manage the certificate entirely within Hydrolix
  • Upload your own certificate to your cluster

Use TLS to securely communicate with Hydrolix endpoints. The traefik component manages TLS. By default, Hydrolix uses a self-signed certificate.

To use cert-manager to generate and manage certificates, see Add a Custom Certificate .

Enable TLS

To enable TLS, update the protocol section of the hydrolix_url component to https in your hydrolixcluster.yaml configuration file:

  hydrolix_url: https://demo.hydrolix.net -> Forces Hydrolix cluster to use TLS
  hydrolix_url: http://demo.hydrolix.net -> Forces Hydrolix cluster to use plain HTTP

📘

Native port change from 9000 to 9440

When TLS is enabled, the native ClickHouse interface listens on port 9440.

Hydrolix-managed certificate

Use Hydrolix to generate and manage a TLS certificate for you.

Prerequisites

Before you begin:

  • Create an active DNS record for your Hydrolix cluster's URL. This is usually an A record.
  • Ensure public access to the IP that hosts your cluster for the certificate challenge

Generate a certificate in Hydrolix

  1. Set the configuration option acme_enabled to true in hydrolixcluster.yaml.
  2. Load the configuration changes to your Hydrolix cluster.
    Hydrolix automatically generates a certificate for your cluster and stores it in a Kubernetes secret named traefik-tls.

Generate a certificate with an allowlist

If your cluster uses an allowlist, Hydrolix adds the Buypass ACME certificate provider IP address to the allowlist.

Hydrolix adds the IP automatically to the spec.ip.allowlist cluster configuration. Check the hydrolixcluster.yaml file to verify that the correct IP address is present.

Generate a certificate without an allowlist

If your cluster doesn't use an allowlist, Hydrolix uses the Let's Encrypt certificate provider.

Hydrolix refreshes the certificate weekly, and the certificate is valid for 90 days.

Confirm certificate generation

To confirm certificate deployment, run the following command to view details of the certificate.

 $ kubectl -n <namespace> get secret traefik-tls -o yaml

For example:

$ kubectl -n hydrolix get secret traefik-tls -o yaml  
apiVersion: v1  
data:  
  tls.crt: someCertificateData  
  tls.key: someCertificateData  
kind: Secret  
metadata:  
  creationTimestamp: "2023-06-27T15:27:12Z"  
  name: traefik-tls  
  namespace: hydrolix  
  resourceVersion: "32029846"  
  uid: 33c06aa2-b81f-44a9-9123-c24e1af94bb5  
type: kubernetes.io/tls

Set the acme_enabled value to true in the hydrolixcluster.yaml file, and apply the changes. Hydrolix starts the init job in the cluster, and creates a cron job called acme-renewal.

Verify the init job runs

To find the init job, run:

$ kubectl -n <namespace> get job --field-selector status.successful=1 | grep acme
init-acme-509c50f0              1/1           21s        24h

This job may disappear if the service has been running for more than a day. Hydrolix clears successful job notifications automatically.

Verify Kubernetes cron job runs

A Kubernetes cron job is also created at startup. To verify, find the acme-renewal job:

$ kubectl -n <namespace> get cronjob
NAME                SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
acme-renewal        0 2 * * SUN   False     0        <none>          24h

📘

Hydrolix-generated certificates update automatically

If you're having issues with renewal, please contact Hydrolix Support.

Use or renew your own certificate

  1. To add or renew a certificate:

    kubectl create secret tls traefik-tls --key=certificates.key --cert=fullchain.pem

    This creates a secret in Kubernetes called traefik-tls and stores your certificate and private key in that
    secret. Traefik automatically checks for the traefik-tls secret, and uses it if valid.

  2. To apply the changes, restart traefik:

    kubectl rollout restart deployment traefik

Kubernetes certificate chain order

Kubernetes requires full chain certificates. The certificate chain should begin with your certificate, continue with intermediate certificates down the chain, and end with the root certificate:

\-----BEGIN CERTIFICATE-----  
{ Your issued Certificate }  
-----END CERTIFICATE-----  
----BEGIN CERTIFICATE-----  
{ Intermediate Certificate }  
-----END CERTIFICATE-----  
-----BEGIN CERTIFICATE-----  
{ Root Certificate }  
-----END CERTIFICATE-----

Troubleshoot certificates

The certificate request process retries six times before it stops. To check the status and locate possible problems, look at these logs:

  • The init-acme job logs
  • The acme pod logs

No cron job listed

If the cron job is not running, the initial init-script may have failed.

How to fix:

  1. Delete the init-acme job if it exists.

    kubectl -n <namespace> get jobs | grep init-acme
kubectl -n <namespace> delete job <job-name>

  2. Delete the acme-renewal cron job if it exists:

    kubectl -n <namespace> get cronjobs | grep acme-renewal
kubectl -n <namespace> delete cronjob acme-renewal

  3. Restart the operator to trigger the init job.

      kubectl rollout restart deployment operator -n <namespace>

LetsEncrypt is busy

Check the acme pod logs for a line similar to this:

 acme: error: 0 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rateLimited :: Service busy; retry later., url:

This can happen due to the number of requests going to the Let's Encrypt service.

How to fix:

Wait 30 minutes for the request backlog and try again.

AWS-specific allowlist

The VPC security groups for the AWS load balancer have a maximum of 60 rules. If your AWS load balancer hits the limit, Hydrolix can't add the required IP addresses for certificate renewal.

How to fix:

  1. Temporarily set the allowlist to 0.0.0.0/0.
  2. Apply the configuration to the hydrolixcluster.yaml file.
    kubectl apply -f hydrolixcluster.yaml --namespace <namespace>
  3. Wait for the init-acme job to complete.
    kubectl -n <namespace> get job --field-selector status.successful=1 | grep acme
  4. Verify that the job completed successfully, and that a traefik-tls secret exists.
  5. Revert the changes made to hydrolixcluster.yaml and verify that the acme-renewal cronjob exists.

If this option isn't feasible, use a custom certificate method instead.

Updated 13 days ago