Deploying Hydrolix

We provide a small toolkit to provision a Hydrolix deployment. Hydrolix is deploy based on the common Kubernetes Operator concept. You need to download a version of hkt which will generate the operator.yaml and also the crd hydrolix.yaml

Deploying Hydrolix

Setup

Ensure you have the following environment variable in scope

export HDX_BUCKET_REGION=<i.e. us-east-2>                                                                   
export HDX_HYDROLIX_URL=<i.e https://my.domain.com>                                                       
export HDX_KUBERNETES_NAMESPACE=<i.e. production-service> 
export HDX_DB_BUCKET_URL=s3://$HDX_KUBERNETES_NAMESPACE
export HDX_ADMIN_EMAIL=<i.e. [email protected]>
export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query "Account" --output text)"
export AWS_STORAGE_ROLE="arn:aws:iam::${AWS_ACCOUNT_ID}:role/${HDX_KUBERNETES_NAMESPACE}-bucket"

Switch kubectl context to use your namespace if you have not done so already.

kubectl config set-context --current --namespace=$HDX_KUBERNETES_NAMESPACE

Install the Hydrolix kubernetes CLI tool hkt

# download
curl -s -o hkt -L https://docs.hydrolix.io/download/hkt/current && chmod +x hkt

# move to a bin folder
sudo mv hkt ~/.local/bin/ 
OR
sudo mv hkt /usr/local/bin/

# verify installation
hkt versions
$3.24.0

Deploy the Hydrolix operator

The operator-resources command generates all the kubernetes resource definitions required for deploying the operator, service accounts, and role permissions. Once deployed, the operator will manage all Hydrolix cluster deployments. To upgrade a deployments version, download a newer version of hkt and repeat this step.

# generator the operator yaml file
hkt operator-resources > operator.yaml

# deploy the operator
kubectl apply -f operator.yaml

Deploy the Hydrolix platform

The hydrolix-cluster command generates the hydrolix.yaml deployment file. We have provided a number of scale profiles for various cloud providers and deployment sizes. You can specify a profile using the scale-profile flag, and if omitted a minimal profile is selected by default. You can also edit the hydrolix.yaml to tune each deployment to your resource requirements. The following instructions create a dev scale deployment and apply it to your cluster.

# generate the hydrolix yaml file
hkt hydrolix-cluster --scale-profile dev --ip-allowlist 0.0.0.0/0 > hydrolix.yaml

# deploy hydrolix
kubectl apply -f hydrolix.yaml

Create your DNS record

The final step in your deployment should be creating the DNS record so you are able to access the services. To retrieve the traefik CNAME you can use the kubectl get service command.

kubectl get service/traefik

NAME          TYPE           CLUSTER-IP       EXTERNAL-IP                                                                     PORT(S)                                AGE                                                                          8089/TCP                               68m
traefik       LoadBalancer   10.100.150.31    a1245678-abcdefg.elb.us-east-1.amazonaws.com  

👍

Enabling IP and SSL/TLS access

This may also be a good time to set-up the IP Access control and TLS certificate. You can find instructions in the Enabling Access & TLS section.

Check your deployment

You can now check the status of your deployment. This can all be done via the kubectl command or via the EKS console, for example to see the status of each pod running

kubectl get pods --namespace $HDX_KUBERNETES_NAMESPACE

NAME                           READY   STATUS    RESTARTS   AGE
batch-head-79cb89b844-b4wpb    1/1     Running   2          80m
batch-peer-7cc54bfb76-282p8    1/1     Running   0          72m
intake-api-675bfd4854-98x88    1/1     Running   0          51m
keycloak-5c99c6549b-gvjqr      1/1     Running   0          13d
merge-head-774fdcc468-gznxb    1/1     Running   0          80m
merge-peer-76ccf8d966-sfz45    1/1     Running   0          72m
operator-9957b786d-hww9x       1/1     Running   0          47m
postgres-0                     1/1     Running   0          10d
query-head-5bcd8989c5-wfq89    1/1     Running   0          72m
query-peer-7b85c54bb6-7x7g8    1/1     Running   0          72m
query-peer-7b85c54bb6-zjtwp    1/1     Running   0          33m
rabbitmq-66847bbfc7-wdm4x      1/1     Running   0          72m
redpanda-0                     1/1     Running   0          43m
redpanda-1                     1/1     Running   0          43m
stream-head-7f4dcdc89c-fr899   1/1     Running   0          39m
stream-peer-667c6ddbdd-knrps   1/1     Running   0          51m
traefik-8444885db7-g2vhm       1/1     Running   0          61m
traefik-8444885db7-tp58j       1/1     Running   0          80m
turbine-api-566c9f59bd-w98fq   1/1     Running   0          72m
ui-b8844cc76-5cbgp             1/1     Running   0          80m
version-84466c6dbb-cqgvk       1/1     Running   0          80m
zookeeper-6d5f66fbb6-gwxn6     1/1     Running   0          13d

👍

Admin Email

You should have received an email that will now allow you to set a password and login. If you do not receive this email, please feel free to contact us at [email protected] and we'll happily assist you.