Upgrade to v4.14
This guide shows you how to upgrade Hydrolix, tells you about what's new with the ClickHouse upgrade, and also shows you how to roll back to version 4.12 if needed.
Upgrade
Upgrading is as simple as following the steps in the Release Notes:
Upgrade on GKE
kubectl apply -f "https://www.hydrolix.io/operator/v4.14.6/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"
Upgrade on EKS
kubectl apply -f "https://www.hydrolix.io/operator/v4.14.6/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"
ClickHouse Upgrade Benefits
When upgrading to this version of Hydrolix, you'll start using ClickHouse version v23.8.10. New features include query caching, parametrized views, a host of new functions, and performance improvements.
Parameterized Views
Views can now have parameters, as specified in the ClickHouse documentation. For example, this SQL statement creates a view:
CREATE VIEW test_parameterized AS SELECT timestamp, cost FROM sample_project.raw_data WHERE (cost < {cost_limit:double}) LIMIT 100
When you use the view, include the variable name in your SELECT statement:
SELECT * FROM test_parameterized(cost_limit = 65518)
Query Caching -- Experimental
This is an experimental feature, so use it with caution.
When using SETTINGS use_query_cache = true
at the end of your query, ClickHouse will cache the results of the query for future use. Subsequent query results issued within a predetermined amount of time will be served from a local cache, rather than from cloud storage. Read more about this feature in the ClickHouse documentation.
Using the example query from above,
SELECT * FROM test_parameterized(cost_limit = 65518)
a test cluster shows this much elapsed time:
However, appending SETTINGS use_query_cache = true
to the query drastically reduces the query response time if the results are in the cache:
Other New Functions
These new SQL functions are available:
- GenerateRandomStructure
- ToDecimalString
- JSONArrayLength
- UTCTimestampTransform
- AESDecryptMysql
- AESEncryptMysql
- CurrentSchema
- Age
- Decrypt
- Encrypt
- EndsWithUTF8
- FirstLine
- hasSubsequence
- hasSubsequenceCaseInsensitive
- hasSubsequenceCaseInsensitiveUTF8
- hasSubsequenceUTF8
- Initcap
- InitcapUTF8
- IsNotDistinctFrom
- ParseDateTime
- RegexpExtract
- ServerTimezone
- Soundex
- Space
- StartsWithUTF8
- StructureToCapnProtoSchema
- StructureToProtobufSchema
- SubstringIndex
- TupleConcat
- WidthBucket
- HashingMurmur
- dateDiff with 'ms' and 'ns'
Breaking Changes and Compatibility Notices
Unified Authentication is Default
If you're upgrading from version 4.8 or earlier, note that this version of Hydrolix uses Unified Authentication by default, which is a breaking change for existing customers using Basic Authentication. To maintain the current behavior when upgrading a cluster from 4.8 or earlier, ensure that unified_auth: false
is set in your cluster's configuration.
Invitation URL API Change
As of version 4.10, the /config/v1/inviteurl
API endpoint has been changed to /config/v1/invites/invite_url
. Refer to the API documentation to check this and other invitation system API changes.
Continuous Control Loop Operator
Rather than running just once per deployment, the operator now runs continuously, as of version 4.10. Temporary manual changes to deployments will now be overwritten by this control loop. When making manual changes, scale the operator to 0 with kubectl scale --replicas 0 deployment/operator
.
Rolling Back to 4.8 with Batch Ingest
If you are using batch ingest, and need to roll back to 4.8.x after upgrading to this version, contact Hydrolix Customer Success before rolling back.
Recent Versions of PostgreSQL
This version of Hydrolix requires PostgreSQL version 11 or higher. We strongly recommend you use PostgreSQL 13 or higher for easy upgrading. If you are using PostgreSQL version 11 or 12, enable the ltree
extension as superuser, and consider upgrading to an up-to-date version of PostgreSQL.
v4.14 to v4.12 Downgrade Procedure
Overview
Downgrading Hydrolix from v4.14 to v4.12 is much like upgrading, with one extra step to do if you've configured multi-cred for GCP storage.
Remove GCP storages
Before rolling back to v4.12, delete any GCP storages that use the new multi-cred configuration.
Perform the Rollback
This procedure is the same as the upgrade step normally published in the release notes, but with v4.12.3 as the target version. For example:
Downgrade on GKE
kubectl apply -f "https://www.hydrolix.io/operator/v4.12.3/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"
Downgrade on EKS
kubectl apply -f "https://www.hydrolix.io/operator/v4.12.3/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"
Updated 12 days ago