added

28 August 2021 - v.2.16.4

Download: hdxctl-v2.16.4

Release Location

wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v2.16.4 && chmod +x hdxctl

Features

Ingest

  • Add additional Shard key support for additional shard key support.
  • Add max_future_days to table settings. The feature is used to parse timestamps to have a MAX number of days ahead of now().
  • Add Avro support.
  • Transform execution reliability updates and logging.
  • Remove Lambda support for intake components (now using on-box microservices for better reliability)
  • Additional "limiter" field support added for fields in transforms.

Query

  • Additional Service Pool to Table mapping support.

Portal/API

  • API Updates for table settings and new features (Avro etc).
  • API updates for Merge Pools feature and Web instance types.
  • Add Portal, API to Web instance.
  • Update to Clusters Page, page structure and functionality.
  • Backward compatibility updates to API structure.
  • API Bugfix for typo that stopped Kafka deployments

Life Cycle

  • Added the ability for merge-pools, so merge servers can be specified for projects and tables.
  • Deleted Table functionality updates.

Observability

  • Grafana Dashboard improvements and consolidation.
  • Default Alert Dashboard

Command and Control

  • Cloudformation updates.
  • Removal of non-used promdisco tool

Bug Fixes

Ingest

  • Bug fixes on the handling of Epoch timestamp values and conversion to DateTime/DateTime64.
  • Bad Transform protections added with additional logging.
  • Sort key ordering fixes when NULL fields are encountered in input stream.

Portal/API

  • Portal Sort Key to be made optional over required.