added

13 September 2021 - v.2.17.6

Download: hdxctl-v2.17.6

Release Location

wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v2.17.6 && chmod +x hdxctl

Features

Ingest

  • Ingest: Automatically confirm SNS subscription requests sent to Autoingest

Query

  • Query: Improved Query retry functionality when a peer fails.
  • Query: Improved Query Peer failover functionality
  • Materialized View: Add initial ability write query output to a new table (INSERT.... SELECT...). More info

Portal/API

  • Portal: Add drawer functionality to clusters page.
  • Portal: Update Fluid Table information page to a new drawer.
  • Portal: Add additional error handling in the clusters pages.
  • API: Add UI additional instance type options.
  • API: Add Avro API capabilities.

Life Cycle

  • Merge: Merge Queue optimisation to provide a merge queue per merge pool
  • Merge: Move Merge Queues to Core stack.
  • Merge: Move Merge Peers to a pod instance.
  • Merge: Added shard key awareness for optimised merging.

Observability

  • Grafana: Change the name of dashboard folder in Grafana to Hydrolix Monitoring
  • Grafana: Move all dashboard into Hydrolix Monitoring folder
  • Superset: Vanity redirect /superset even without slash.

Command and Control

  • Scale: Extend Spot instance capabilities to make it percentage spot instance configurable.
  • Bastion: Upgrade bastion PostgreSQL version

Bug Fixes

Ingest

  • Ingest: Bugfix to datetime handling limiter functionality
  • Ingest: Remove unrecognized messages from the queue for Autoingest
  • Autoingest: Issue resolved where unrecognized messages in the Autoingest queue were left in-flight until process restart.

Query

  • Query-Head: Bug fix where head would fail with full disk with very large query responses.
  • Query: Update query stats with stats being obtained directly from Clickhouse.

Portal/API

  • API: Bug fix to add users with no permissions to the viewable users list.
  • API: Bug fix for default pool behaviour.
  • API: Initialisation fixes to API service.
  • Portal/API: Bug fix for Source Page where reapplying changes caused a failed update.
  • Portal: Bug fix for Kafka updates.

Command and Control

  • Bug fix: Ensure pool name is non-empty when locating queue
  • Bug fix: Fixed SSH Key multiline error with authorized keys
  • Bug fix: Fixed RSA SSH Key support error.
  • Bug fix: Respect AWS public key in addition to ssh_authorized_keys.

Observability

  • Bug fix to enable Superset to change the Server type.

Lifecycle

  • Merge: Merge-peer bug fix for filling up disk On failure
  • Merge-Head: Issue resolved where the merge head is pool-aware, but the config was not been updated with pool information.