Deploy Hydrolix
Hydrolix deployments follow the Kubernetes operator pattern. To deploy Hydrolix, use the Hydrolix Kubernetes Tool (hkt
) to generate an operator configuration and a custom resource Hydrolix configuration.
Deploying Hydrolix
- Setup
- Install the Hydrolix kubernetes CLI tool
hkt
- Deploy the Hydrolix operator
- Deploy the Hydrolix platform
- Create a DNS Record
- Check your Deployment
Setup
First, configure the following environment variables:
export HDX_BUCKET_REGION=<i.e. us-east-2>
export HDX_HYDROLIX_URL=<i.e https://my.domain.com>
export HDX_KUBERNETES_NAMESPACE=<i.e. production-service>
export HDX_DB_BUCKET_URL=s3://$HDX_KUBERNETES_NAMESPACE
export HDX_ADMIN_EMAIL=<i.e. [email protected]>
export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query "Account" --output text)"
export AWS_STORAGE_ROLE="arn:aws:iam::${AWS_ACCOUNT_ID}:role/${HDX_KUBERNETES_NAMESPACE}-bucket"
Team Access
After deploying Hydrolix for the first time, we will send an email to HDX_ADMIN_EMAIL with instructions and a link to create a password. This will allow you to administer Hydrolix and invite others in your team.
Security: Hydrolix restricts IP access and is managed through the
hydrolix.yaml
ip_allowlist section. Please refer to the docs for more information.
Switch kubectl
context to use your namespace
if you have not done so already.
kubectl config set-context --current --namespace=$HDX_KUBERNETES_NAMESPACE
Deploy the Hydrolix operator
The Hydrolix operator resources API generates all the Kubernetes resource definitions required for deploying the operator, service accounts, and role permissions. Once deployed, the operator will manage all Hydrolix cluster deployments. To upgrade a deployment to a new version, repeat this step.
# generate the operator yaml file
curl "https://www.hydrolix.io/operator/latest/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}" > operator.yaml
# deploy the operator
kubectl apply -f operator.yaml
Deploy the Hydrolix platform
We have provided a number of scale profiles for various cloud providers and deployment sizes. You can specify a profile using the scale-profile
flag, and if omitted a minimal
profile is selected by default. You can also edit the hydrolix.yaml
to tune each deployment to your resource requirements. The following instructions create a dev
scale deployment and apply it to your cluster.
apiVersion: hydrolix.io/v1
kind: HydrolixCluster
metadata:
name: hdx
namespace: ${HDX_KUBERNETES_NAMESPACE}
spec:
admin_email: ${HDX_ADMIN_EMAIL}
db_bucket_region: ${HDX_BUCKET_REGION}
db_bucket_url: ${HDX_DB_BUCKET_URL}
env: {}
hydrolix_name: hdx
hydrolix_url: ${HDX_HYDROLIX_URL}
ip_allowlist:
- 42.78.92.98/32 // TODO: Replace this with your IP address!
kubernetes_namespace: ${HDX_KUBERNETES_NAMESPACE}
overcommit: false
scale: {}
scale_profile: dev
Use the following command to replace the environment variables above with their values:
eval "echo \"$(cat hydrolixcluster.yaml)\""
Don't forget to add your IP address to the allowlist. You can get your IP address by running curl -s ifconfig.me
.
Create your DNS record
The final step in your deployment should be creating the DNS record so you are able to access the services. To retrieve the traefik CNAME you can use the kubectl get service
command.
kubectl get service/traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 8089/TCP 68m
traefik LoadBalancer 10.100.150.31 a1245678-abcdefg.elb.us-east-1.amazonaws.com
Enabling IP and SSL/TLS access
This may also be a good time to set-up the IP Access control and TLS certificate. You can find instructions in the Enabling Access & TLS section.
Check your deployment
You can now check the status of your deployment. This can all be done via the kubectl
command or via the EKS console, for example to see the status of each pod running
kubectl get pods --namespace $HDX_KUBERNETES_NAMESPACE
NAME READY STATUS RESTARTS AGE
batch-head-79cb89b844-b4wpb 1/1 Running 2 80m
batch-peer-7cc54bfb76-282p8 1/1 Running 0 72m
intake-api-675bfd4854-98x88 1/1 Running 0 51m
keycloak-5c99c6549b-gvjqr 1/1 Running 0 13d
merge-head-774fdcc468-gznxb 1/1 Running 0 80m
merge-peer-76ccf8d966-sfz45 1/1 Running 0 72m
operator-9957b786d-hww9x 1/1 Running 0 47m
postgres-0 1/1 Running 0 10d
query-head-5bcd8989c5-wfq89 1/1 Running 0 72m
query-peer-7b85c54bb6-7x7g8 1/1 Running 0 72m
query-peer-7b85c54bb6-zjtwp 1/1 Running 0 33m
rabbitmq-66847bbfc7-wdm4x 1/1 Running 0 72m
redpanda-0 1/1 Running 0 43m
redpanda-1 1/1 Running 0 43m
stream-head-7f4dcdc89c-fr899 1/1 Running 0 39m
stream-peer-667c6ddbdd-knrps 1/1 Running 0 51m
traefik-8444885db7-g2vhm 1/1 Running 0 61m
traefik-8444885db7-tp58j 1/1 Running 0 80m
turbine-api-566c9f59bd-w98fq 1/1 Running 0 72m
ui-b8844cc76-5cbgp 1/1 Running 0 80m
version-84466c6dbb-cqgvk 1/1 Running 0 80m
zookeeper-6d5f66fbb6-gwxn6 1/1 Running 0 13d
Admin Email
You should have received an email that will now allow you to set a password and login. If you do not receive this email, please feel free to contact us at [email protected] and we'll happily assist you.
Updated about 3 hours ago