ProductPricingCase StudiesSupportSTART FOR FREE!

Deploy Hydrolix

Suggest Edits

Hydrolix deployments follow the Kubernetes operator pattern. To deploy Hydrolix, use the Hydrolix Kubernetes Tool (hkt) to generate an operator configuration and a custom resource Hydrolix configuration.

Deploying Hydrolix

Setup

First, configure the following environment variables:

export HDX_BUCKET_REGION=<i.e. us-east-2>
export HDX_HYDROLIX_URL=<i.e https://my.domain.com>
export HDX_KUBERNETES_NAMESPACE=<i.e. production-service>
export HDX_DB_BUCKET_URL=s3://$HDX_KUBERNETES_NAMESPACE
export HDX_ADMIN_EMAIL=<i.e. [email protected]>
export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query "Account" --output text)"
export AWS_STORAGE_ROLE="arn:aws:iam::${AWS_ACCOUNT_ID}:role/${HDX_KUBERNETES_NAMESPACE}-bucket"

📘

Team Access

After deploying Hydrolix for the first time, we will send an email to HDX_ADMIN_EMAIL with instructions and a link to create a password. This will allow you to administer Hydrolix and invite others in your team.

Security: Hydrolix restricts IP access and is managed through the hydrolix.yaml ip_allowlist section. Please refer to the docs for more information.

Switch kubectl context to use your namespace if you have not done so already.

kubectl config set-context --current --namespace=$HDX_KUBERNETES_NAMESPACE

Deploy the Hydrolix operator

The Hydrolix operator resources API generates all the Kubernetes resource definitions required for deploying the operator, service accounts, and role permissions. Once deployed, the operator will manage all Hydrolix cluster deployments. To upgrade a deployment to a new version, repeat this step.

# generate the operator yaml file
curl "https://www.hydrolix.io/operator/latest/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}" > operator.yaml

# deploy the operator
kubectl apply -f operator.yaml

Deploy the Hydrolix platform

We have provided a number of scale profiles for various cloud providers and deployment sizes. You can specify a profile using the scale-profile flag, and if omitted a minimal profile is selected by default. You can also edit the hydrolix.yaml to tune each deployment to your resource requirements. The following instructions create a dev scale deployment and apply it to your cluster.

apiVersion: hydrolix.io/v1
kind: HydrolixCluster
metadata:
  name: hdx
  namespace: ${HDX_KUBERNETES_NAMESPACE}
spec:
  admin_email: ${HDX_ADMIN_EMAIL}
  db_bucket_region: ${HDX_BUCKET_REGION}
  db_bucket_url: ${HDX_DB_BUCKET_URL}
  env: {}
  hydrolix_name: hdx
  hydrolix_url: ${HDX_HYDROLIX_URL}
  ip_allowlist:
  - 42.78.92.98/32 // TODO: Replace this with your IP address!
  kubernetes_namespace: ${HDX_KUBERNETES_NAMESPACE}
  overcommit: false
  scale: {}
  scale_profile: dev

Use the following command to replace the environment variables above with their values:

eval "echo \"$(cat hydrolixcluster.yaml)\""

Don't forget to add your IP address to the allowlist. You can get your IP address by running curl -s ifconfig.me.

Create your DNS record

The final step in your deployment should be creating the DNS record so you are able to access the services. To retrieve the traefik CNAME you can use the kubectl get service command. 

kubectl get service/traefik

NAME          TYPE           CLUSTER-IP       EXTERNAL-IP                                                                     PORT(S)                                AGE                                                                          8089/TCP                               68m
traefik       LoadBalancer   10.100.150.31    a1245678-abcdefg.elb.us-east-1.amazonaws.com

👍

Enabling IP and SSL/TLS access

This may also be a good time to set-up the IP Access control and TLS certificate. You can find instructions in the Enabling Access & TLS section.

Check your deployment

You can now check the status of your deployment. This can all be done via the kubectl command or via the EKS console, for example to see the status of each pod running

kubectl get pods --namespace $HDX_KUBERNETES_NAMESPACE

NAME                           READY   STATUS    RESTARTS   AGE
batch-head-79cb89b844-b4wpb    1/1     Running   2          80m
batch-peer-7cc54bfb76-282p8    1/1     Running   0          72m
intake-api-675bfd4854-98x88    1/1     Running   0          51m
keycloak-5c99c6549b-gvjqr      1/1     Running   0          13d
merge-head-774fdcc468-gznxb    1/1     Running   0          80m
merge-peer-76ccf8d966-sfz45    1/1     Running   0          72m
operator-9957b786d-hww9x       1/1     Running   0          47m
postgres-0                     1/1     Running   0          10d
query-head-5bcd8989c5-wfq89    1/1     Running   0          72m
query-peer-7b85c54bb6-7x7g8    1/1     Running   0          72m
query-peer-7b85c54bb6-zjtwp    1/1     Running   0          33m
rabbitmq-66847bbfc7-wdm4x      1/1     Running   0          72m
redpanda-0                     1/1     Running   0          43m
redpanda-1                     1/1     Running   0          43m
stream-head-7f4dcdc89c-fr899   1/1     Running   0          39m
stream-peer-667c6ddbdd-knrps   1/1     Running   0          51m
traefik-8444885db7-g2vhm       1/1     Running   0          61m
traefik-8444885db7-tp58j       1/1     Running   0          80m
turbine-api-566c9f59bd-w98fq   1/1     Running   0          72m
ui-b8844cc76-5cbgp             1/1     Running   0          80m
version-84466c6dbb-cqgvk       1/1     Running   0          80m
zookeeper-6d5f66fbb6-gwxn6     1/1     Running   0          13d

👍

Admin Email

You should have received an email that will now allow you to set a password and login. If you do not receive this email, please feel free to contact us at [email protected] and we'll happily assist you.

Updated about 3 hours ago