v2.17.6

Release Location

wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v2.17.6 && chmod +x hdxctl

Features

Ingest - Ingest: Automatically confirm SNS subscription requests sent to Autoingest

Query - Query: Improved Query retry functionality when a peer fails. - Query: Improved Query Peer failover functionality - Materialized View: Add initial ability write query output to a new table (INSERT.... SELECT...). More info

Portal/API - Portal: Add drawer functionality to clusters page. - Portal: Update Fluid Table information page to a new drawer. - Portal: Add additional error handling in the clusters pages. - API: Add UI additional instance type options. - API: Add Avro API capabilities.

Life Cycle - Merge: Merge Queue optimisation to provide a merge queue per merge pool - Merge: Move Merge Queues to Core stack. - Merge: Move Merge Peers to a pod instance. - Merge: Added shard key awareness for optimised merging.

Observability - Grafana: Change the name of dashboard folder in Grafana to Hydrolix Monitoring - Grafana: Move all dashboard into Hydrolix Monitoring folder - Superset: Vanity redirect /superset even without slash.

Command and Control - Scale: Extend Spot instance capabilities to make it percentage spot instance configurable. - Bastion: Upgrade bastion PostgreSQL version

Bug Fixes

Ingest - Ingest: Bugfix to datetime handling limiter functionality - Ingest: Remove unrecognized messages from the queue for Autoingest - Autoingest: Issue resolved where unrecognized messages in the Autoingest queue were left in-flight until process restart.

Query - Query-Head: Bug fix where head would fail with full disk with very large query responses. - Query: Update query stats with stats being obtained directly from Clickhouse.

Portal/API - API: Bug fix to add users with no permissions to the viewable users list. - API: Bug fix for default pool behaviour. - API: Initialisation fixes to API service. - Portal/API: Bug fix for Source Page where reapplying changes caused a failed update. - Portal: Bug fix for Kafka updates.

Command and Control - Bug fix: Ensure pool name is non-empty when locating queue - Bug fix: Fixed SSH Key multiline error with authorized keys - Bug fix: Fixed RSA SSH Key support error. - Bug fix: Respect AWS public key in addition to ssh_authorized_keys.

Observability - Bug fix to enable Superset to change the Server type.

Lifecycle - Merge: Merge-peer bug fix for filling up disk On failure - Merge-Head: Issue resolved where the merge head is pool-aware, but the config was not been updated with pool information.