28 August 2021 - v.2.16.4

Download: hdxctl-v2.16.4

Release Location

wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v2.16.4 && chmod +x hdxctl

Features

Ingest

• Add additional Shard key support for additional shard key support.
• Add max_future_days to table settings. The feature is used to parse timestamps to have a MAX number of days ahead of now().
• Add Avro support.
• Transform execution reliability updates and logging.
• Remove Lambda support for intake components (now using on-box microservices for better reliability)
• Additional "limiter" field support added for fields in transforms.

Query

• Additional Service Pool to Table mapping support.

Portal/API

• API Updates for table settings and new features (Avro etc).
• API updates for Merge Pools feature and Web instance types.
• Add Portal, API to Web instance.
• Update to Clusters Page, page structure and functionality.
• Backward compatibility updates to API structure.
• API Bugfix for typo that stopped Kafka deployments

Life Cycle

• Added the ability for merge-pools, so merge servers can be specified for projects and tables.
• Deleted Table functionality updates.

Observability

• Grafana Dashboard improvements and consolidation.
• Default Alert Dashboard

Command and Control

• Cloudformation updates.
• Removal of non-used promdisco tool

Bug Fixes

Ingest

• Bug fixes on the handling of Epoch timestamp values and conversion to DateTime/DateTime64.
• Bad Transform protections added with additional logging.
• Sort key ordering fixes when NULL fields are encountered in input stream.

Portal/API

• Portal Sort Key to be made optional over required.