Prepare a Cluster

In order to deploy the Hydrolix data platform on Microsoft Azure, you first need to set up a Microsoft Managed Kubernetes Service (AKS) cluster.

Set Environment Variables

To begin, set up some environment variables using an env.sh script. Replace the < > values below with your actual values:

export HDX_AZURE_REGION=<e.g. eastus>
export HDX_AZURE_RG=<e.g. dev>
export HDX_CLUSTER_NAME=<e.g. hdx>
export HDX_AZURE_STORAGE_ACCOUNT=<e.g. vaellushalu>
export HDX_HYDROLIX_URL=<e.g. https://testdomain.hydro59.com>
export HDX_KUBERNETES_NAMESPACE=<e.g. mytest>
export HDX_KUBERNETES_CLUSTER_NAME=<e.g. testcluster>
export HDX_ADMIN_EMAIL=<e.g. [email protected]>
export HDX_DB_BUCKET_URL=https://${HDX_AZURE_STORAGE_ACCOUNT}.blob.core.windows.net/${HDX_KUBERNETES_NAMESPACE}
export HDX_DB_SAMPLE_URL=https://${HDX_AZURE_STORAGE_ACCOUNT}.blob.core.windows.net/${HDX_KUBERNETES_NAMESPACE}/test_data

Load these environment variables with the following command:

source env.sh

The az commands below depend on these environment variables.

Login to Azure

Use this command to open up a browser login window:

az login --scope https://management.core.windows.net//.default

Upon success, the az tool will have logged into your account.

Create an Azure Resource Group

To contain the Hydrolix-related resources, create a new Resource Group with this command.

az group create --location ${HDX_AZURE_REGION} --resource-group $HDX_AZURE_RG

Create an Azure Storage Account

az storage account create --name $HDX_AZURE_STORAGE_ACCOUNT --resource-group $HDX_AZURE_RG --location $HDX_AZURE_REGION --sku STANDARD_LRS

Create an Azure Storage Container

👍

Already Created an Azure Storage Container?

If you've already created an Azure Storage Container for Hydrolix, skip this step.

Hydrolix needs access to an Azure Storage Container for data storage. Run the following command to create a storage container via the Azure CLI:

az storage container create -n $HDX_KUBERNETES_NAMESPACE --account-name $HDX_AZURE_STORAGE_ACCOUNT

Create a Kubernetes Cluster

Next, you'll deploy a Kubernetes Cluster into the Resource Group. The following command creates a single node pool with the following characteristics:

default-pool

• Count: Autoscaling from 2-10 is a reasonable choice to begin provisioning with.
• Type: The default node type is Standard_DS2_v2; it is recommended that you provision using at least Standard_DS4_v2 nodes.

This pool provides sufficient node and capacity sizing for a basic load. Note that this command generates new SSH keys if you don't have them on your workstation already.

az aks create -n $HDX_KUBERNETES_CLUSTER_NAME --resource-group $HDX_AZURE_RG --kubernetes-version 1.27.3 --enable-cluster-autoscaler  --min-count 2 --max-count 10 -s Standard_DS4_v2 --generate-ssh-keys

Ensure that you have the Kubeconfig credentials for this cluster:

az aks get-credentials --name $HDX_KUBERNETES_CLUSTER_NAME --resource-group $HDX_AZURE_RG

Create a Kubernetes Namespace

kubectl create namespace $HDX_KUBERNETES_NAMESPACE

Set your kubectl context to this new namespace:

kubectl config set-context --current --namespace="$HDX_KUBERNETES_NAMESPACE"