There are occasions where a more resilient deployment of PostGres is needed or several Hydrolix clusters want to access the same bucket. The following describes how this is achieved within GKE.
If you are looking to migrate a cluster that already has data loaded please contact [email protected] before commencing this change. The catalog (PostGres) is a critical part of the Hydrolix service and you can cause irreparable damage to the cluster if it is moved incorrectly.
Hydrolix suggests using the Google Cloud SQL service to create your PostGres instance. It can be created as a high availability deployments, backups, deletion protection etc all built in.
Hydrolix would recommend sizing your instances based on the sizing criteria specified within Scale Profiles, For example Mega would be 100GB Disk, 6CPU, 24G of Memory.
In addition Hydrolix would recommend deploying it without a public IP. To ensure connectivity with the K8's cluster it should therefore be deployed within the same VPC in the same region (to ensure good performance). More information on how Kubernetes and Cloud SQL can work together can be found [here[(https://cloud.google.com/sql/docs/postgres/connect-kubernetes-engine#private-ip).
hydrolix-cluster command generates the
hydrolixcluster.yaml deployment file. We have provided a number of scale profiles for various cloud providers and deployment sizes. You can specify a profile using the
scale-profile flag. You can also edit the
hydrolixcluster.yaml to tune each deployment to your resource requirements. The following instructions create a
dev scale deployment and apply it to your cluster.
hkt hydrolix-cluster --scale-profile dev --ip-allowlist `curl -s ifconfig.me`/32 > hydrolixcluster.yaml
Once you have the basic file created you will need to edit it.
Open the hydrolixcluster.yaml in your favourite text editor and you will need to edit the following values::
--- apiVersion: hydrolix.io/v1 kind: HydrolixCluster metadata: name: <NameSpace> #<--- Should already be set spec: admin_email: <admin email> #<--- Should already be set db_bucket_url: <bucket path> #<--- Should already be set db_bucket_region: <region> #<--- Should already be set hydrolix_url: <hostname to use> #<--- Should already be set catalog_db_admin_user: postgres #<--- Add the admin user "postgres" to your config catalog_db_admin_db: postgres #<--- Add the admin db "postgres" to your config catalog_db_host: <YOU HOST/IP> #<--- Add the IP for your cluster pg_ssl_mode: <disable/require> #<--- Set whether you would like to use SSL. ip_allowlist: - 111.222.333.444/32 #<--- Should be already set scale_profile: dev #<--- Should be already set scale: postgres: replicas: 0 #<---- Don't forget to set the internal postgres to 0
--- apiVersion: hydrolix.io/v1 kind: HydrolixCluster metadata: name: myhdxdeployment spec: admin_email: [email protected] db_bucket_url: gs://myhdxdeployment db_bucket_region: us-central1 hydrolix_url: http://my.hdxdeploymente.com catalog_db_admin_user: postgres catalog_db_admin_db: postgres catalog_db_host: 184.108.40.206 pg_ssl_mode: disable ip_allowlist: - 111.222.333.444/32 scale_profile: dev scale: postgres: replicas: 0
In the Scale portion of your file make sure to set PostGres to 0 replicas in the scale section. This is so the postgres instance isn't started.
postgres: replicas: 0
The PostGres Secret should be held within a curated secret within Kubernetes.
--- apiVersion: v1 kind: Secret metadata: name: curated namespace: <namespace> stringData: ROOT_DB_PASSWORD: <the password to your postgres> type: Opaque
--- apiVersion: v1 kind: Secret metadata: name: curated namespace: myhdxdeployment stringData: ROOT_DB_PASSWORD: mysupersecretpassword type: Opaque
The following commands will apply these settings.
kubectl apply -f secrects.yaml kubectl apply -f hydrolixcluster.yaml
Already Running Cluster
If you have created this after you have already deployed, you will need to do a
kubectl rollout restart deploymentfor the cluster.
Updated 2 months ago