Upgrade to v5.10

Do not skip minor versions when upgrading or downgrading

Skipping versions when upgrading or downgrading Hydrolix can result in database schema inconsistencies and cluster instability. Always upgrade or downgrade sequentially through each minor version.

Example:
Upgrade from 5.7.9 → 5.8.6 → 5.9.5, not 5.7.9 → 5.9.5.

Overview

Hydrolix v5.10 introduces several breaking changes that require attention before upgrading. This page covers required pre-upgrade actions, automatic migrations, and the standard upgrade procedure.

For the complete list of changes, see the v5.10.3 release notes.

Pre-upgrade checklist

Complete these checks before applying the v5.10 operator:

• [ ] Verify Kafka brokers support TLS 1.3 (if using SASL_SSL sources)
• [ ] Upgrade Grafana to v12.3.1 or later (if operator-managed)
• [ ] Remove any automation that uses CREATE DICTIONARY or CREATE FUNCTION SQL statements
• [ ] Update shadow table configurations to use transform instead of transform_id
• [ ] Update dashboards and alerts that reference the efficiency metric
• [ ] Review any automation that explicitly configures merge-head or disables merge-controller
• [ ] Verify SIEM source credentials work correctly after upgrading (automatic migration)

Required actions before upgrading

These changes require action before or immediately after upgrading. Items are ordered by severity.

Kafka sources now require TLS 1.3

Kafka sources using SASL_SSL now require the broker to support TLS 1.3 or later. Connections to brokers that only support TLS 1.2 will fail, which can cause ingestion failures and data loss.

Before upgrading, verify that all Kafka brokers in your environment support TLS 1.3. Contact your Kafka administrator or check the broker's ssl.enabled.protocols configuration.

See Kafka sources for configuration details.

Grafana minimum version requirement

Hydrolix v5.10 requires Grafana v12.3.1 or later for operator-managed instances. This applies to both Grafana Enterprise and OSS.

The operator now uses the non-deprecated plugin installation method available in Grafana v12.1.0+. The previous method using the GF_PLUGINS_INSTALL environment variable is no longer supported. Plugin configuration now uses the grafana_config.plugins dictionary.

The grafana-clickhouse-datasource plugin is no longer installed by default. The hydrolix-hydrolix-datasource plugin is installed by default instead.

Before upgrading:

1. Confirm your operator-managed Grafana instance is running v12.3.1 or later.
2. Review and update any Grafana plugin configuration that relies on GF_PLUGINS_INSTALL.
3. If your dashboards depend on the grafana-clickhouse-datasource plugin, add it to your grafana_config.plugins configuration.

See Grafana for configuration details.

CREATE DICTIONARY and CREATE FUNCTION SQL statements blocked

Hydrolix now returns QUERY_NOT_ALLOWED for CREATE DICTIONARY and CREATE FUNCTION SQL statements. This is a security fix for a server-side request forgery (SSRF) vulnerability (CWE-918).

Before upgrading, remove any automation, scripts, or test suites that issue these SQL statements directly. Use the Hydrolix API or UI to manage dictionaries and user-defined functions instead.

Shadow table field rename

The shadow_table configuration field transform_id has been renamed to transform. The transform field now accepts either a valid UUID or a transform name, enabling reusable Configuration as Code (CaC) definitions across clusters.

Before upgrading, update any automation, scripts, or API calls that use transform_id in shadow table configurations. The table_id field has also been renamed to table.

efficiency metric replaced

Three new metrics replace the merge controller efficiency metric:

• row_weighted_efficiency
• mem_weighted_efficiency
• harmonic_efficiency

Before upgrading, update any dashboards or alerts that reference the efficiency metric to use the new metric names. See Advanced merge controller for details.

Automatic changes

These changes take effect automatically during the upgrade. No action is required unless your environment has custom configuration that overrides default behavior.

Merge controller is now the default

The merge-controller component is now the default, replacing merge-head. Clusters with no explicit merge configuration will automatically use merge-controller. This also removes the dependency on RabbitMQ for merge operations.

If your environment has automation or configuration that explicitly disables merge-controller or relies on merge-head behavior, review it before upgrading. See Advanced merge controller for details.

Shard key automatic migration

On upgrade, Hydrolix automatically migrates existing tables that have a shard_key configured. The migration sets shard_key_algo to strict and marks the table with legacy_sharding = true. The enable_sharding flag is set to true.

This migration preserves existing sharding behavior. No action is required unless you want to switch to the new performance sharding mode after upgrading. See Table settings for details on the new sharding options.

SIEM source credential migration

On upgrade, Hydrolix automatically migrates SIEM sources that have access details and secret keys exposed directly on the API. The migration transfers these credentials to a new SIEM credential object associated with the source.

After upgrading, verify that SIEM source ingestion is working correctly. If a SIEM source had an empty secret value before the upgrade, the migration falls back to a safe default value. SIEM source credentials can be updated after migration if needed.

Protected system resources

The hydro.logs table and the megaTransform transform are now protected system resources. The API blocks attempts to delete, rename, or update these resources, except for settings fields.

Protected resources:

• Project: hydro
• Tables: logs, monitor
• Transforms: monitor_ingest_transform, megaTransform
• Source: hydrologs

Remove any automation that modifies or deletes these resources. Table settings can still be updated.

Apply the new operator

For self-managed installations, apply the new operator with the appropriate command for your cloud provider. For installations managed by Hydrolix-supplied tools, follow the procedure prescribed by those tools.

GKE

kubectl apply -f "https://www.hydrolix.io/operator/v5.10.3/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

EKS

kubectl apply -f "https://www.hydrolix.io/operator/v5.10.3/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

LKE and AKS

kubectl apply -f "https://www.hydrolix.io/operator/v5.10.3/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}"

Monitor the upgrade

Kubernetes jobs named init-cluster and init-turbine-api will automatically run to upgrade your entire installation to match the new operator's version number. This will take a few minutes, during which time you can observe your pods' restarts with your Kubernetes monitor tool.

Ensure both the init-cluster and init-turbine-api jobs have completed successfully and that the turbine-api pod has restarted without errors. After that, view the UI and use the API of your new installation as a final check.

If the turbine-api pod doesn't restart successfully, or other functionality is missing, check the logs of the init-cluster and init-turbine-api jobs for details about failures. This can be done using the k9s utility or with the kubectl command:

% kubectl logs -l app=init-cluster
% kubectl logs -l app=init-turbine-api

For additional help, contact Hydrolix support.

Rollback to v5.9.x

Downgrading from v5.10.3 to v5.9.x requires a database schema migration. After applying the v5.9.x operator, open a shell in the turbine-api container and run:

./manage.py release_5_9

Then apply the v5.9.x operator using the same command pattern shown in Apply the new operator, replacing the version number with your target v5.9.x version.

For additional help, contact Hydrolix support.