Documentation
ProductPricingDocsBlog

Upgrade to v4.14

This guide shows you how to upgrade Hydrolix, tells you about what's new with the ClickHouse upgrade, and also shows you how to roll back to version 4.12 if needed.

Suggest Edits

Upgrade

Upgrading is as simple as following the steps in the Release Notes:

Upgrade on GKE

kubectl apply -f "https://www.hydrolix.io/operator/v4.14.6/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

Upgrade on EKS

kubectl apply -f "https://www.hydrolix.io/operator/v4.14.6/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

ClickHouse Upgrade Benefits

When upgrading to this version of Hydrolix, you'll start using ClickHouse version v23.8.10. New features include query caching, parametrized views, a host of new functions, and performance improvements.

Parameterized Views

Views can now have parameters, as specified in the ClickHouse documentation. For example, this SQL statement creates a view:

CREATE VIEW test_parameterized AS SELECT timestamp, cost FROM sample_project.raw_data WHERE (cost < {cost_limit:double}) LIMIT 100

When you use the view, include the variable name in your SELECT statement:

SELECT * FROM test_parameterized(cost_limit = 65518)

Query Caching -- Experimental

This is an experimental feature, so use it with caution.

When using SETTINGS use_query_cache = true at the end of your query, ClickHouse will cache the results of the query for future use. Subsequent query results issued within a predetermined amount of time will be served from a local cache, rather than from cloud storage. Read more about this feature in the ClickHouse documentation.

Using the example query from above, 

SELECT * FROM test_parameterized(cost_limit = 65518)

a test cluster shows this much elapsed time:

Longer Query

However, appending SETTINGS use_query_cache = true to the query drastically reduces the query response time if the results are in the cache:

Shorter Query

Other New Functions

These new SQL functions are available:

  • GenerateRandomStructure
  • ToDecimalString
  • JSONArrayLength
  • UTCTimestampTransform
  • AESDecryptMysql
  • AESEncryptMysql
  • CurrentSchema
  • Age
  • Decrypt
  • Encrypt
  • EndsWithUTF8
  • FirstLine
  • hasSubsequence
  • hasSubsequenceCaseInsensitive
  • hasSubsequenceCaseInsensitiveUTF8
  • hasSubsequenceUTF8
  • Initcap
  • InitcapUTF8
  • IsNotDistinctFrom
  • ParseDateTime
  • RegexpExtract
  • ServerTimezone
  • Soundex
  • Space
  • StartsWithUTF8
  • StructureToCapnProtoSchema
  • StructureToProtobufSchema
  • SubstringIndex
  • TupleConcat
  • WidthBucket
  • HashingMurmur
  • dateDiff with 'ms' and 'ns'

Breaking Changes and Compatibility Notices

Unified Authentication is Default

If you're upgrading from version 4.8 or earlier, note that this version of Hydrolix uses Unified Authentication by default, which is a breaking change for existing customers using Basic Authentication. To maintain the current behavior when upgrading a cluster from 4.8 or earlier, ensure that unified_auth: false is set in your cluster's configuration.

Invitation URL API Change

As of version 4.10, the /config/v1/inviteurl API endpoint has been changed to /config/v1/invites/invite_url. Refer to the API documentation to check this and other invitation system API changes.

Continuous Control Loop Operator

Rather than running just once per deployment, the operator now runs continuously, as of version 4.10. Temporary manual changes to deployments will now be overwritten by this control loop. When making manual changes, scale the operator to 0 with kubectl scale --replicas 0 deployment/operator.

Rolling Back to 4.8 with Batch Ingest

If you are using batch ingest, and need to roll back to 4.8.x after upgrading to this version, contact Hydrolix Customer Success before rolling back.

Recent Versions of PostgreSQL

This version of Hydrolix requires PostgreSQL version 11 or higher. We strongly recommend you use PostgreSQL 13 or higher for easy upgrading. If you are using PostgreSQL version 11 or 12, enable the ltree extension as superuser, and consider upgrading to an up-to-date version of PostgreSQL.

v4.14 to v4.12 Downgrade Procedure

Overview

Downgrading Hydrolix from v4.14 to v4.12 is much like upgrading, with one extra step to do if you've configured multi-cred for GCP storage.

Remove GCP storages

Before rolling back to v4.12, delete any GCP storages that use the new multi-cred configuration.

Perform the Rollback

This procedure is the same as the upgrade step normally published in the release notes, but with v4.12.3 as the target version. For example:

Downgrade on GKE

kubectl apply -f "https://www.hydrolix.io/operator/v4.12.3/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

Downgrade on EKS

kubectl apply -f "https://www.hydrolix.io/operator/v4.12.3/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

Updated about 2 months ago