Deploy Production PostgreSQL

Hydrolix provisions a single internal PostgreSQL pod to store the catalog. This default configuration has no high availability or automated backups and isn't suitable for production.

For production deployments, use CloudNativePG to manage a high-availability PostgreSQL cluster inside the LKE cluster.

New deployments don't require migration

For new Hydrolix installations, complete all steps on this page after deploying the Operator and HydrolixCluster resource, but before ingesting any data. The Operator creates the required databases, users, and permissions on the CNPG instance. No migration is needed.

Existing deployments require migration

Switching an existing deployment from the internal PostgreSQL pod to an external database requires a catalog migration. Catalog loss can lead to data becoming unrecoverable. Contact Hydrolix support and review Migrate to External PostgreSQL before proceeding.

Prerequisites

• A Hydrolix deployment on LKE. See Deploy Hydrolix on LKE.
• kubectl configured to access the LKE cluster.
• Object storage for CloudNativePG WAL archiving and backups.

Deploy high-availability PostgreSQL in Kubernetes

Use CloudNativePG (CNPG) to manage a high-availability PostgreSQL cluster in Kubernetes. CNPG is external to Hydrolix.

1. Install the CNPG operator. See Installation and upgrades - CloudNativePG for instructions.

2. Create a catalog.yaml file with this minimal configuration:

   apiVersion: postgresql.cnpg.io/v1
   kind: Cluster
   metadata:
     name: catalog
     namespace: {namespace}
   spec:
     backup: {backup}      # -- Specify the object storage path here
     bootstrap:
       initdb:
         database: catalog
         owner: query_api
     enableSuperuserAccess: true
     imageName: ghcr.io/cloudnative-pg/postgresql:15.12
     instances: 3
     primaryUpdateMethod: switchover
     storage:
       size: 100Gi
   

   • backup: Specify an object storage path for archiving Write-Ahead Logging (WAL) files and backups. See Appendix C - Common object stores for backups - CloudNativePG for supported options.
   • owner: query_api: The PostgreSQL role that owns the catalog database. Hydrolix uses this role internally - don't change this value.
   • enableSuperuserAccess: Must be true so Hydrolix can log in with the root user to create the Keycloak and Config API databases and users.

3. Apply the Cluster object and wait for the status to show as healthy.

   kubectl apply -f catalog.yaml
   kubectl -n "${namespace}" get cluster catalog
   

   A healthy cluster shows Cluster in healthy state in the STATUS column with all instances ready:

   NAME      AGE   INSTANCES   READY   STATUS                     PRIMARY
   catalog   5m    3           3       Cluster in healthy state   catalog-1
   

Define the external PostgreSQL connection

Disable the internal PostgreSQL instance and configure Hydrolix to connect to the external PostgreSQL endpoint.

1. Edit the HydrolixCluster resource.

   kubectl edit hydrolixcluster hdx -n {namespace}
   

2. Fill in the values for catalog_db_admin_user, catalog_db_admin_db, and catalog_db_host. Set scale.postgres.replicas to 0.

   • For CloudNativePG, use catalog-rw as the catalog_db_host value. This is the CNPG read-write service endpoint that routes to the primary instance.
   • For an external managed PostgreSQL service, use the endpoint the cloud provider supplies.

   spec:
     catalog_db_admin_user: postgres      #<--- Admin user
     catalog_db_admin_db: postgres        #<--- Database for initial admin connection
     catalog_db_host: <postgresql-host>   #<--- catalog-rw for CNPG, or the external endpoint
   
     scale:
       postgres:
         replicas: 0                      #<---- Disable the internal PostgreSQL pod
     scale_profile: prod
   

Create the secret

Store the PostgreSQL credentials in a curated Kubernetes secret.

If using CloudNativePG, retrieve the auto-generated passwords first.

ROOT_PWD=$(kubectl -n {namespace} get secret catalog-superuser -o jsonpath='{.data.password}' | base64 -d)
CATALOG_PWD=$(kubectl -n {namespace} get secret catalog-app -o jsonpath='{.data.password}' | base64 -d)

1. Edit the curated secret.

   kubectl edit secret curated -n {namespace}
   

2. Add the stringData property with the required credentials. Kubernetes encodes values from stringData and stores them in data. When reading the curated secret, only the data key is present.

   For CloudNativePG, include both passwords:

   stringData:
     ROOT_DB_PASSWORD: ${ROOT_PWD}
     CATALOG_DB_PASSWORD: ${CATALOG_PWD}
   

   For an externally managed PostgreSQL service, include only the admin password set when creating the instance:

   stringData:
     ROOT_DB_PASSWORD: <postgresql-admin-password>
   

New and existing deployments

The Operator picks up the secret on first deploy for new deployments.

If the Hydrolix cluster is already running, restart all deployments to apply the new credentials. Secret changes don't trigger automatic restarts.

kubectl rollout restart deployment