To prepare a Linode environment for a Hydrolix deployment, use the Linode Cloud console. You'll need to create a new object storage bucket with an access key, a Kubernetes cluster, and a PostgreSQL instance.

Variables

You'll need seven pieces of information to create environment variables for easy management of your Hydrolix Kubernetes cluster.

Collect this information as you set up services so you can put them in an env.sh script for easy access later in the process.

Create Your Bucket

1. Begin by logging in to the Linode Cloud console. Click the Create button. From the resulting list, select the "Bucket" option:

2. Enter a label (also known in the Linode UI as a "name") for the bucket. Hydrolix will store all of your row data in this bucket. Choose a region, preferring the region where you intend to configure your Kubernetes cluster:

3. Record the bucket URL for later use. You can find it in the "Buckets" list under the title of the bucket.
4. Record the bucket region identifier for later use in the Set Environment Variables section. A full list of region codes can be found in the Linode documentation. For example, Atlanta, GA's identifier is us-southeast-1.
5. Record the bucket endpoint. This is simply your bucket URL minus the first part of the bucket name. For example, if your bucket URL is https://hdx-company.us-southeast-1.linodeobjects.com, then the bucket endpoint is https://us-southeast-1.linodeobjects.com

Create a Bucket Access Key and Secret Key

Next, create an access key so Hydrolix can securely connect to your storage bucket. Within the Linode Cloud console, navigate to the "Object Storage" page. Click on "Access Keys" to view the access keys tab:

Click the "Create Access Key" button. Enter a name for the key. Enable the "Limited Access" toggle. Grant the key "Read/Write" access to the bucket you created in the previous step. Click the "Create Access Key" button to create the keys.

A dialog box will appear in the Linode Cloud console, displaying your access key and your secret key.

Store your secret key somewhere secure, like a password manager: Linode will only show you this key once. You can view the access key at any time, but it's best to save it now since you'll need it to create your cluster. Once you've recorded the keys, click the "I Have Saved My Secret Key" button to dismiss the dialog box.

Create the Kubernetes Cluster

Now it's time to create the Kubernetes cluster where you'll deploy Hydrolix services. In the Linode Cloud console, click the Create button. From the resulting list, select the "Kubernetes" option:

1. Enter a name for your cluster.
2. Select the same region used to create your bucket.
3. Use the latest version of Kubernetes available in the dropdown list. You can reference the Hydrolix release notes if you want to verify whether a version of Kubernetes is supported.
4. Choose the HA option, for high availability.
5. Use three Dedicated 32 GB nodes in the list below by clicking its "Add" button. (See the last line in the screenshot below.)
6. Click "Create Cluster" to instantiate your cluster.

A few minutes later, you'll see the cluster summary page show you that your three nodes are up and running.

Set Environment Variables

Command-line access is much easier if you set up the below environment variables using an env.sh script.

export HDX_HYDROLIX_URL=<Hydrolix URL>
export HDX_KUBERNETES_NAMESPACE=<Kubernetes Namespace>
export HDX_BUCKET_REGION=<Bucket Region>
export HDX_DB_BUCKET_URL=<Bucket URL>
export HDX_DB_BUCKET_ENDPOINT=<Bucket Endpoint>

export HDX_ADMIN_EMAIL=<Administrative E-mail>

For instance, following the values from the examples above:

export HDX_HYDROLIX_URL=https://my.domain.com
export HDX_KUBERNETES_NAMESPACE=hdx-company
export HDX_BUCKET_REGION=us-southeast-1
export HDX_DB_BUCKET_URL=https://hdx-company.us-southeast-1.linodeobjects.com
export HDX_DB_BUCKET_ENDPOINT=https://us-southeast-1.linodeobjects.com

export [email protected]

Load these environment variables with the following command:

source env.sh

Configure the Kubernetes Cluster

After creating the cluster, you should see the following cluster summary page:

First, download your kubeconfig.yaml file. Save it to your local computer. You'll use this file to connect to your cluster on the command line.

Next, configure autoscaling for your cluster:

1. Click "Autoscale Pool" to open a dialog where you can configure autoscaling.
2. Enable the "Autoscaler" toggle.
3. For the "Max" setting, enter a value of "20".
4. Click the "Save Changes" button to update your cluster configuration.

You can now connect to your cluster with kubectl:

kubectl --kubeconfig <configfile> get nodes

For easier access, export the configuration filename in the KUBECONFIG environment variable. Following our example where you downloaded a hdx-company-kubeconfig.yaml file:

export KUBECONFIG=${PWD}/hdx-company-kubeconfig.yaml
kubectl get nodes

Create a k8s Namespace

Create a dedicated namespace in your Kubernetes Cluster:

kubectl create namespace $HDX_KUBERNETES_NAMESPACE

For ease of use, set your new namespace as a default:

kubectl config set-context --current --namespace="$HDX_KUBERNETES_NAMESPACE"

Set Bucket Access Key and Secret within Kubernetes

Run the following using the bucket access key and secret key you obtained from the corresponding step in which you generated them.

 kubectl create secret generic curated \
    --from-literal=AWS_ACCESS_KEY_ID='{YOUR_ACCESS_KEY_ID}' \
    --from-literal=AWS_SECRET_ACCESS_KEY='{YOUR_SECRET_ACCESS_KEY}'

This sets the bucket access key and secret key in a Kubernetes secret. The operator pod will aggregate all the values from the curated Kubernetes secret and merge them into the dynamically generated general secret. This general secret is accessible for all resources within your given namespace, giving your Hydrolix cluster's pods and their containers access to the bucket credentials.