Creating a Hydrolix Environment

After installing hdxctl, the next task is creating a Hydrolix environment.

The act of creating a new environment is called create-cluster, however, many components will be activated. The components are explained in the architecture section.

The create-cluster command requires a region and a Hydrolix Client_ID. When hdxctl was installed, an AWS region was chosen and a Hydrolix Client_ID was generated. The region that was chosen will need to be the same as the region chosen for cluster deployment.

hdxctl --region <region> create-cluster <Client_ID>

For example:

user@hdxctl:~$ hdxctl --region us-east-2 create-cluster hdxcli-ze26j666

You may be asked for an email address if this is the first time you have run the command, this is so you can login into the Configuration API and the Portal. This email address is only used for login purposes.

When this command runs, an AWS Cloudformation stack from a static template, provided and maintained by Hydrolix, is created that will stand up and initialize the environment with sample data. This command is asynchronous. To execute it synchronously to watch status updates live, add the --wait command. The output will look similar to:

ubuntu@ip-172-31-14-164:~$ hdxctl --region eu-west-1 create-cluster hdxcli-6tb6llo6
creating self_deploy stack
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
updating self_deploy stack
Waiting for stack update; sleeping 100 seconds
....
Waiting for stack update; sleeping 100 seconds
creating hydrolix stack
initiated creation of hdx-ezdgz44a

In this case, hdx-ezdgz44a is the cluster_id that is referenced in hdxctl cluster commands.

Note: This process can take up to 20 minutes.

To see the clusters you have created, run the clusters command:

$ hdxctl clusters 

ubuntu@ip-172-31-14-164:~$ hdxctl clusters
CLIENT_ID        CLUSTER_ID    CREATED              HOST                           STATUS           WHO      REGION
---------------  ------------  -------------------  -----------------------------  ---------------  -------  ---------
hdxcli-6tb6llo6  hdx-ezdgz44a  2020-08-20 09:57:17  myhyd.hydrolix.live.  CREATE_COMPLETE  user1  eu-west-1

If you do not see your cluster, issue the --sync command

$ hdxctl clusters --sync

CLIENT_ID        CLUSTER_ID    CREATED              HOST                   STATUS           WHO      REGION
---------------  ------------  -------------------  --------------------- ---------------  -------  ---------
hdxcli-6tb6llo6  hdx-ezdgz44a  2020-08-20 09:57:17  myhyd.hydrolix.live.  CREATE_COMPLETE  user1  eu-west-1

If you now check in your cloud console you should see you have a new cluster built. Note, the new cluster is not running as its scale is 0. To scale the cluster please take a look at the instructions for cluster scaling

Enabling Access to your Cluster

Each new Hydrolix cluster is created with network access restrictions that limit connectivity to the systems end-points (Config API, Ingest Streaming API, Query API). This is achieved through the use of IP allows lists and with IP’s or ranges of IP’s specified to be ‘allowed’ access to those end-ponts.

To specify the IP address’s that can access the these end-points you should run the following command:

hdxctl update <client_id> <cluster_id> --ip-allowlist "CIDR IP1" --ip-allowlist "CIDR IP2"

Each IP or range should be specified under its own option.

For example:

hdxctl update hdxcli-6tb6llo6 hdx-ezdgz44a --ip-allowlist "111.222.333.444/32" 

If you want to simply set the cluster as having public accessibility you can use `0.0.0.0/0’

For example:

hdxctl update hdxcli-6tb6llo6 hdx-ezdgz44a --ip-allowlist "0.0.0.0/0" 

Enabling Access to your Bucket.

If you are to import data from an existing bucket you will need to update the bucket allow-list. This will enable the batch-archiecture to see the bucket and retrieve data from those locations.

To enable access to a bucket the following command should be used:

hdxctl update <client_id> <cluster_id> --bucket-allowlist "<bucketname>" 

For example:

hdxctl update hdxcli-6tb6llo6 hdx-ezdgz44a --bucket-allowlist "mybucket" 

Final Step

You should have recieved an email that will now allow you to set a password and login. If you do not recieve this email, please feel free to contact us at support@hydrolix.io and we’ll happily assist you.