Deploy Hydrolix

Hydrolix deployments follow the Kubernetes operator pattern. To deploy Hydrolix, generate an operator configuration (operator.yaml) and a custom resource Hydrolix configuration (hydrolixcluster.yaml). You'll use these files to deploy Hydrolix on your Kubernetes cluster.

Prerequisites

You must complete all of the steps outlined in the Prepare Your Cluster guide before you can deploy Hydrolix in EKS.

Then, ensure that you've loaded the environment variables defined in env.sh:

source env.sh

Switch kubectl context to use your Hydrolix cluster namespace:

kubectl config set-context --current --namespace=$HDX_KUBERNETES_NAMESPACE

Deploy Operator

The Hydrolix operator resources API generates all of the Kubernetes resource definitions required to deploy the operator, including service accounts and role permissions. Once deployed, the operator manages your Hydrolix cluster deployment.

You generated the operator configuration in the previous section, so now run the following kubectl command to deploy an operator with the configuration stored in the file:

kubectl apply -f operator.yaml

Deploy Hydrolix

Hydrolix provides a number of scale profiles that allocate all cluster resources based on a single expected load value. Specify a profile using the spec.scale-profile field in your cluster configuration. If omitted, Hydrolix uses the default minimal profile. The following instructions create a dev scale deployment and apply it to your cluster. Write the following into a file named hydrolixcluster.yaml:

apiVersion: hydrolix.io/v1
kind: HydrolixCluster
metadata:
  name: hdx
  namespace: ${HDX_KUBERNETES_NAMESPACE}
spec:
  admin_email: ${HDX_ADMIN_EMAIL}
  db_bucket_region: ${HDX_BUCKET_REGION}
  db_bucket_url: ${HDX_DB_BUCKET_URL}
  env: {}
  hydrolix_name: hdx
  hydrolix_url: ${HDX_HYDROLIX_URL}
  ip_allowlist:
  - 0.0.0.0/0 #TODO: Replace this with your IP address in CIDR notation, eg. 12.13.14.15/32
  kubernetes_namespace: ${HDX_KUBERNETES_NAMESPACE}
  overcommit: false
  scale:
    intake-head:
      replicas: 1
  scale_profile: dev

The above config will deploy, among other things, a default, internal Postgres instance that is non-HA. If you want to run a more resilient version, read our Deploy Production Postgres guide .

Use the following command to replace the environment variables above with their values:

eval "echo \"$(cat hydrolixcluster.yaml)\"" > hydrolixcluster.yaml

Don't forget to add your IP address to the allowlist. You can get your IP address by running the following command:

curl -s ifconfig.me

Finally, deploy the configuration:

kubectl apply -f hydrolixcluster.yaml

Create the DNS Record

Next, create the DNS record so you can access your cluster via the hostname. To retrieve the traefik CNAME, use the following command:

kubectl get service/traefik

If you can't find the service, check to make sure you're using the correct Kubernetes namespace for your cluster.

👍

Enable IP and SSL/TLS access

This may also be a good time to set-up IP access control and a TLS certificate. You can find instructions in the Enable Access & TLS section.

Check Deployment

You can now check the status of your deployment. You can use kubectl or via the EKS console. For example, to see the status of each pod, run the following kubectl command:

kubectl get pods --namespace $HDX_KUBERNETES_NAMESPACE

You should see output similar to the following:

NAME                           READY   STATUS    RESTARTS   AGE
batch-head-79cb89b844-b4wpb    1/1     Running   2          80m
batch-peer-7cc54bfb76-282p8    1/1     Running   0          72m
intake-api-675bfd4854-98x88    1/1     Running   0          51m
keycloak-5c99c6549b-gvjqr      1/1     Running   0          13d
merge-head-774fdcc468-gznxb    1/1     Running   0          80m
merge-peer-76ccf8d966-sfz45    1/1     Running   0          72m
operator-9957b786d-hww9x       1/1     Running   0          47m
postgres-0                     1/1     Running   0          10d
query-head-5bcd8989c5-wfq89    1/1     Running   0          72m
query-peer-7b85c54bb6-7x7g8    1/1     Running   0          72m
query-peer-7b85c54bb6-zjtwp    1/1     Running   0          33m
rabbitmq-66847bbfc7-wdm4x      1/1     Running   0          72m
redpanda-0                     1/1     Running   0          43m
redpanda-1                     1/1     Running   0          43m
stream-head-7f4dcdc89c-fr899   1/1     Running   0          39m
stream-peer-667c6ddbdd-knrps   1/1     Running   0          51m
traefik-8444885db7-g2vhm       1/1     Running   0          61m
traefik-8444885db7-tp58j       1/1     Running   0          80m
turbine-api-566c9f59bd-w98fq   1/1     Running   0          72m
ui-b8844cc76-5cbgp             1/1     Running   0          80m
version-84466c6dbb-cqgvk       1/1     Running   0          80m
zookeeper-6d5f66fbb6-gwxn6     1/1     Running   0          13d

👍

Admin Email

When you create a new Hydrolix cluster, your cluster sends an email to HDX_ADMIN_EMAIL with instructions and a link to create a password. If you do not receive this email, contact us at [email protected].