Documentation
ProductPricingDocsBlog

Enable TLS

Suggest Edits

Adding a Certificate

Hydrolix provides multiple ways to manage certificates on your cluster. You can:

  • generate and manage the certificate entirely within Hydrolix
  • upload your own certificate to your cluster

📘

Native port change from 9000 to 9440

When TLS is enabled, the native clickhouse interface listens on port 9440.

You should use TLS when communicating with Hydrolix endpoints. The traefik component manages TLS. By default, Hydrolix uses a self signed certificate.

If you want to use cert-manager to generate and manage certificates for you, see Add a Custom Certificate .

To enable TLS, update the protocol section of the hydrolix_url component to https in your hydrolixcluster.yaml configuration file:

.......

  hydrolix_url: https://demo.hydrolix.net -> Will force Hydrolix cluster to use TLS
  hydrolix_url: http://demo.hydrolix.net -> Will force Hydrolix cluster to use plain HTTP.

......

Hydrolix-Generated Certificate

Hydrolix can handle TLS certificate generation for you.

You'll need:

  • an active DNS record for your Hydrolix cluster's URL
  • public access to the IP that hosts your cluster (for the certificate challenge).

Installation Steps

  1. Set the configuration option acme_enabled to true in hydrolixcluster.yaml.
  2. Load the configuration changes to your Hydrolix cluster. Hydrolix will automatically generate a certificate for your cluster and store it in a Kubernetes secret named traefik-tls.

If your cluster uses an allowlist, Hydrolix will add the Buypass ACME certificate provider IP address to the allowlist. If your cluster does not use an allowlist, Hydrolix uses the Let's Encrypt certificate provider.

Hydrolix refreshes the certificate weekly, but the certificate itself is valid for 90 days.

Confirming the Certificate Generation

The first step is to check a certificate is deployed, the following command will provide details of the certificate.

 $ kubectl -n <namespace> get secret traefik-tls -o yaml

For example:

$ kubectl -n hydrolix get secret traefik-tls -o yaml  
apiVersion: v1  
data:  
  tls.crt: someCertificateData  
  tls.key: someCertificateData  
kind: Secret  
metadata:  
  creationTimestamp: "2023-06-27T15:27:12Z"  
  name: traefik-tls  
  namespace: hydrolix  
  resourceVersion: "32029846"  
  uid: 33c06aa2-b81f-44a9-9123-c24e1af94bb5  
type: kubernetes.io/tls

When the acme_enabled is set to true you should find you have an init job run in the cluster as well as a new cronjob created.

To see that the init job has run you can run the following command.

$ kubectl -n <namespace> get job --field-selector status.successful=1 | grep acme
init-acme-509c50f0              1/1           21s        24h

Note: this may disappear if you've had the service running for a while. Successful job notifications are cleared down.

A Kubernetes cronjob is also created at startup. To check the cronjob has been created you can run, a job called acme-renewal should be created.

$ kubectl -n <namespace> get cronjob
NAME                SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
acme-renewal        0 2 * * SUN   False     0        <none>          24h

Certificate issues

If you find you have no Certificate deployed their can be various reasons. The certificate request process will re-try 6 times before halting. To check the status looking at the logs of the init-acme job or acme pods which will typically show where something has failed.

Some things to check include:

  • AWS Specific Allowlist - If you have IP Allowlists enabled be aware that VPC security groups for the underlying AWS load-balancer can have a maximum of 60 rules. If this limit has been reached the IP's for the certificate renewal service will not be added to the Loadbalancer not allowing the http call back from the CA. A quick fix to this is to set the allowlist to 0.0.0.0/0 to get the certificate deployed. If this is not possible using an alternative method for certificates maybe needed - Route53 Certificates, Cloudflare Certificates, Google CloudDNS Certificates.
  • LetsEncrypt is busy - This is usually seen with a log line similar to the below, this is usually due to the number of requests going to the Letsencrypt service. To resolve it, it is best to wait for 30 minutes and re-start the service.
 acme: error: 0 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rateLimited :: Service busy; retry later., url:
  • No Cronjob - if you see there is no cronjob listed it is likely the initial init-script has not been run. Follow the instructions below to re-run the init-job.

Restarting the Certificate Generation

To re-setup the certificate generation service you should

  1. Delete the init-acme job
$ kubectl -n <namespace> get jobs | grep init-acme

kubectl -n <namespace> delete job <job name>
  1. Delete the cronjob (if exsits)
$ kubectl -n <namespace> get cronjobs | grep acme-renewal

kubectl -n <namespace> delete cronjob acme-renewal
  1. Restart the operator
kubectl rollout restart deployment operator

Loading Your Own Certificate

To specify a certificate, load one with kubectl:

kubectl create secret tls traefik-tls --key=certificates.key --cert=fullchain.pem

This creates a secret in Kubernetes called traefik-tls and stores your certificate and private key in that secret. traefik automatically checks for the traefik-tls secret, and uses it if a valid configuration exists.

🚧

Certificate Chain Order

Kubernetes requires full chain certificates. Certificate chain should begin with your certificate, continue with intermediate certificates down the chain, and end with the root certificate:

-----BEGIN CERTIFICATE-----
{ Your issued Certificate }
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
{ Intermediate Certificate }
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
{ Root Certificate }
-----END CERTIFICATE-----

To put your changes into effect, restart traefik with the following command:

kubectl rollout restart deployment traefik

Renewing your own Certificate

📘

Hydrolix Generated Certificates should be updated automatically

Hydrolix Generated Certificate is renewed automatically. If you are having issues with the renewal please reach out to Hydrolix Support

To renew a certificate:

  • Delete the existing traefik-tls secret: 
    kubectl delete secret traefik-tls
  • Create a new traefik-tls secret containing the new certificate: 
    kubectl create secret tls traefik-tls --key=privkey.pem --cert=fullchain.pem
  • Restart traefik to put your changes into effect: 
    kubectl rollout restart deployment traefik

Updated about 2 months ago