Deploying to GKE

It's happening!

To deploy you will use the hkt tool to create the YAML required for kubectl and then apply it.

The first step is to install the operator in your Kubernetes Cluster:

hkt operator-resources > operator.yaml && kubectl apply -f operator.yaml

Then you can create the resource definition using the hydrolix-cluster command in hkt, and apply it.

hkt hydrolix-cluster --ip-allowlist `curl -s ifconfig.me`/32 --env EMAIL_PASSWORD=$EMAIL_PASSWORD  > hydrolixcluster.yaml && kubectl apply -f hydrolixcluster.yaml

Within the hydrolixcluster.yaml a default scale is defined for the Hydrolix cluster. More information can be found here Scaling on GCP should you wish to deploy differently than the default.

The default setting applied will bring up a minimal size of infrastructure of stateful and stateless components

Stateful

ServiceDescriptionReplicasCPUMemoryStorageData Storage
postgresThe Core1416Gi5Gi100Gi
redpandaIngest226Gi5Gi1Ti
prometheusReporting and Control115Gi5Gi50GB

Stateless

ServiceDescriptionReplicasCPUMemoryStorageData Storage
alter-peerAlter0220Gi10Gi-
batch-headIngest1500m1Gi5Gi-
batch-peerIngest1-12210Gi10Gi-
decayAge111Gi5Gi-
intake-apiIngest1500m1Gi5Gi-
kafka-peerIngest0210Gi10Gi-
keycloakThe Core112Gi5Gi-
merge-headMerge1500m1Gi5Gi-
merge-peerMerge1-12230Gi10Gi-
operatorKubernetes Operator1500m1Gi5Gi-
query-headQuery1648Gi50Gi-
query-peerQuery3-121548Gi50Gi-
rabbitmqIngest114Gi10Gi-
reaperAge121Gi5Gi-
stream-headIngest1-12410Gi10Gi-
stream-peerIngest1-12210Gi10Gi-
turbine-apiQuery1500m1Gi5Gi-
traefikThe Core211Gi5Gi-
versionThe Core1500m1Gi1Gi-
zookeeperThe Core111Gi5Gi-

Create your DNS record

The final step in your deployment should be creating the DNS record so you are able to access the services.
To retrieve the traefik IP you can use the kubectl get services command.

kubectl get services --namespace=$CLIENT_ID
NAME          TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                                AGE
intake-api    ClusterIP      10.108.4.124    <none>           8080/TCP                               17d
keycloak      ClusterIP      10.108.6.86     <none>           8080/TCP                               17d
native        LoadBalancer   10.108.2.66     35.188.xxx.yyy   9000:30315/TCP                         17d
postgres      ClusterIP      None            <none>           5432/TCP                               16d
prometheus    ClusterIP      None            <none>           9090/TCP                               81m
query-head    ClusterIP      10.108.5.68     <none>           9000/TCP,8088/TCP                      17d
rabbitmq      ClusterIP      10.108.1.200    <none>           5672/TCP                               17d
redpanda      ClusterIP      None            <none>           9092/TCP,8082/TCP,33146/TCP,9644/TCP   48m
stream-head   ClusterIP      10.108.2.202    <none>           8089/TCP                               17d
traefik       LoadBalancer   10.108.14.118   104.155.xxx.yyy  80:31016/TCP                           81m
turbine-api   ClusterIP      10.108.13.125   <none>           3000/TCP                               17d
ui            ClusterIP      10.108.0.245    <none>           80/TCP                                 81m
version       ClusterIP      10.108.7.203    <none>           23925/TCP                              17d
zookeeper     ClusterIP      10.108.2.118    <none>           2181/TCP                               17d

πŸ‘

Enabling IP and SSL/TLS access.

This may also be a good time to set-up the IP Access control and TLS certificate. You can find instructions in the Enabling Access & TLS section.

Checking your deployment.

You can now check the status of your deployment. This can all be done via the Kubectl command or via the Google Cloud console, for example to see the status of each pod running:

kubectl get pods --namespace $CLIENT_ID
NAME                           READY   STATUS    RESTARTS   AGE
batch-head-79cb89b844-b4wpb    1/1     Running   2          80m
batch-peer-7cc54bfb76-282p8    1/1     Running   0          72m
intake-api-675bfd4854-98x88    1/1     Running   0          51m
keycloak-5c99c6549b-gvjqr      1/1     Running   0          13d
merge-head-774fdcc468-gznxb    1/1     Running   0          80m
merge-peer-76ccf8d966-sfz45    1/1     Running   0          72m
operator-9957b786d-hww9x       1/1     Running   0          47m
postgres-0                     1/1     Running   0          10d
query-head-5bcd8989c5-wfq89    1/1     Running   0          72m
query-peer-7b85c54bb6-7x7g8    1/1     Running   0          72m
query-peer-7b85c54bb6-zjtwp    1/1     Running   0          33m
rabbitmq-66847bbfc7-wdm4x      1/1     Running   0          72m
redpanda-0                     1/1     Running   0          43m
redpanda-1                     1/1     Running   0          43m
stream-head-7f4dcdc89c-fr899   1/1     Running   0          39m
stream-peer-667c6ddbdd-knrps   1/1     Running   0          51m
traefik-8444885db7-g2vhm       1/1     Running   0          61m
traefik-8444885db7-tp58j       1/1     Running   0          80m
turbine-api-566c9f59bd-w98fq   1/1     Running   0          72m
ui-b8844cc76-5cbgp             1/1     Running   0          80m
version-84466c6dbb-cqgvk       1/1     Running   0          80m
zookeeper-6d5f66fbb6-gwxn6     1/1     Running   0          13d

The Final Step

You should have received an email that will now allow you to set a password and login. If you do not receive this email, please feel free to contact us at [email protected] and we'll happily assist you.


Did this page help you?