Create Cluster!
It's happening!
To deploy the platform into your cloud account you will be using the hdxctl
tool you installed earlier with the use of a sub-command called create-cluster
.
First time use of the hdxctl create-cluster
command will create all the components needed and scale the system to a basic level. The command will require the AWS Region and the Hydrolix client_id
you registered for earlier.
If this is the first time you have run the command you will also be asked for an email address, this is so you can login into the Configuration API and the Portal, simply add a working email address.
hdxctl --region <region> create-cluster <client_Id> --wait
$ hdxctl --region eu-west-1 create-cluster hdxcli-12345678 --wait
creating self_deploy stack
Waiting for stack creation sleeping 5 seconds
Waiting for stack creation sleeping 5 seconds
.............
creating hydrolix stack
initiated creation of hdx-abcdefgh
Waiting for stack update; sleeping 100 seconds
Waiting for stack update; sleeping 100 seconds
.............
stack creation complete
ubuntu@ip-172-31-14-164:~$
The hdxctl
CLI uses predefined AWS Cloudformation templates and a lambda to build the stack. All you need to do is sit back and wait.
To learn more about what components are created take a look at the platform section.
To see the status of your cluster (if you haven't used the --wait
flag) you can use the following command, this will show the status of your cluster build:
hdxctl clusters --sync
$ hdxctl clusters --sync
CLIENT_ID CLUSTER_ID CREATED HOST STATUS WHO REGION
--------------- ------------ ------------------- ----------------------------- --------------- ------- ---------
hdxcli-12345678 hdx-abcdefgh 2020-08-20 09:57:17 myhost.hydrolix.live. CREATE_COMPLETE user1 eu-west-1
The clusters
command is useful as it can provide information on the overall state of updates and scaling changes you may have implemented. In this instance, CREATE_COMPLETE
means everything has been created successfully.
If you are having problems with the install, this usually comes in the form of permissions to deploy components. To see where the deployment has failed look at the Cloud Formation console under the events for the two Stacks created.
We can now take a look at the current scale of the cluster:
hdxctl scale <client_id> <cluster_id>
$ hdxctl scale hdxcli-12345678 hdx-abcdefgh
SERVICE/POOL COUNT FAMILY SIZE DISK
-------------- ------- -------- ------ ------
bastion 1 t2 micro 30
batch-peer0 1 r5 xlarge 30
grafana 0 t2 small 30
head 1 c5n xlarge 30
intake-misc 1 t3 small 30
merge-peer0 1 r5 xlarge 30
prometheus 0 t2 medium 50
query-peer0 1 c5n xlarge 30
stream-head 1 r5 xlarge 30
stream-peer0 1 r5 xlarge 30
superset 0 t2 medium 30
web 1 t3 small 30
zookeeper 3 t2 medium 30
If you also check in your cloud console you should see you have a bunch of EC2 instances running a d a new S3 bucket named after your client_id
If for some reason you seeing all your counts as 0 you can use this command to get it to a minimal state.
hdxctl scale --minimal <client_id> <cluster_id>
Now, we need to get Access..... onward!
Updated 3 months ago