Skip to content

v6.1.12

Support multiple Traefik basic auth passwords, tighten Kubernetes roles and role bindings for the workload reloader, and log original client IP whenever possible.

This release contains bug fixes to Hydrolix v6.1. Refer to the release notes to see other notable feature announcements and information for this version.

Upgrade instructions⚓︎

Install the latest Hydrolix v6.1 patch release.

Don't skip minor versions when upgrading or downgrading

Skipping versions when upgrading or downgrading Hydrolix can result in database schema inconsistencies and cluster instability. Always upgrade or downgrade sequentially through each minor version, moving to every intermediate minor release in turn rather than jumping directly from the starting version to the target.

Point releases are safe to skip. Upgrade through each minor version, using the latest point release of each one. Don't install every point release along the way.

For example, to go from v5.10 to v6.1, upgrade sequentially to v5.11.10 -> v6.0.19 -> v6.1.10. Downgrading works the same way, in reverse.

Step 1: Set tunable⚓︎

Set the tunable turbine_api_feature_migrate_projects_to_orgs to true if you want each of the projects in the cluster to be elevated to its own organization, or skip this step and accept the default.

1
2
spec:
  turbine_api_feature_migrate_projects_to_orgs: false

Default behavior: For a single-tenant cluster, after upgrading to v6.1, three Customer objects are created. One Customer object each is created for the projects hydro and sample_project. All other projects are associated with a third Customer object.

This is an example of explicit declaration of the default setting.

1
2
spec:
  turbine_api_feature_migrate_projects_to_orgs: true

Set this for [multi-tenant clusters][multi-tenant clusters]. This tunable must be explicitly set in the Hydrolix cluster spec before upgrading to v6.1.

A distinct Customer object is created for every existing project.

The tunable setting controls a background job that creates the [parallel hierarchy][parallel hierarchy] and runs for the lifetime of Hydrolix v6.1.

Step 2: Apply the new Hydrolix operator⚓︎

If you have a self-managed installation, apply the new operator directly with the kubectl command examples. If you're using Hydrolix-supplied tools to manage your installation, follow the procedure prescribed by those tools.

1
2
3
4
VERSION=v6.1.14
BASEURL="https://www.hydrolix.io/operator/${VERSION}/operator-resources"

kubectl apply -f "${BASEURL}?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

1
2
3
4
VERSION=v6.1.14
BASEURL="https://www.hydrolix.io/operator/${VERSION}/operator-resources"

kubectl apply -f "${BASEURL}?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

1
2
3
4
VERSION=v6.1.14
BASEURL="https://www.hydrolix.io/operator/${VERSION}/operator-resources"

kubectl apply -f "${BASEURL}?namespace=${HDX_KUBERNETES_NAMESPACE}"

Step 3: Monitor the upgrade process⚓︎

Kubernetes jobs named init-cluster and init-turbine-api will automatically run to upgrade your entire installation to match the new operator's version number. This will take a few minutes, during which time you can observe your pods' restarts with your Kubernetes monitor tool.

Ensure both the init-cluster and init-turbine-api jobs have completed successfully and that the turbine-api pod has restarted without errors. After that, view the UI and use the API of your new installation as a final check.

If the turbine-api pod doesn't restart successfully, or other functionality is missing, check the logs of the init-cluster and init-turbine-api jobs for details about failures. This can be done using the k9s utility or with the kubectl command:

1
2
% kubectl logs -l app=init-cluster
% kubectl logs -l app=init-turbine-api

If you still need help, contact Hydrolix support.

Post-upgrade instructions⚓︎

This one-time post-upgrade step removes the orphaned objects, if they still exist.

1
2
kubectl delete clusterrolebinding "reloader-${HDX_KUBERNETES_NAMESPACE}"
kubectl delete clusterrole "reloader-${HDX_KUBERNETES_NAMESPACE}"

The operator reconciliation logic can't delete the role and binding which become unnecessary after upgrade to v6.1.12 or later.

Downgrade instructions⚓︎

Use these instructions when downgrading from v6.1 to v6.0.

  1. Connect to the Kubernetes cluster using k9s and select the namespace holding the Hydrolix cluster.
  2. Select the turbine-api pod and turbine-api container.
  3. Invoke a shell by pressing s

  4. Execute each of the following commands, which will produce STDERR to terminal. Each must exit cleanly.

    1
    ./manage.py release_6_0

  5. Exit the k9s utility by pressing Ctrl+c.

  6. Follow the upgrade instructions for the latest Hydrolix v6.0 release.

Changelog⚓︎

Improvements⚓︎

Cluster operations improvements⚓︎

  • Traefik now accepts multiple basic authentication passwords in one Kubernetes secret, where it previously accepted one. Use commas to separate passwords, or the character set in the traefik_password_delimiter tunable.

Bug fixes⚓︎

Config API fixes⚓︎

  • Audit records now take the client IP from the first X-Forwarded-For entry, falling back to the REMOTE_ADDR header, the X-Real-Ip header, then the connection's remote address.
  • Fixed storage creation failing when every bucket access check passed. The Config API looked for the first failing check and errored when there wasn't one.

Cluster operations fixes⚓︎

  • Scoped the reloader Kubernetes RBAC role to its own namespace, replacing a ClusterRole that granted cluster-wide access to secrets, configmaps, and workloads. On a shared cluster, that let a Hydrolix service account read any secret and restart another tenant's workloads. The scheduler, descheduler, and OpenTelemetry roles were scoped the same way.