v2.16.4

Release Location

wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v2.16.4 && chmod +x hdxctl

Features

Ingest - Add additional Shard key support for additional shard key support. - Add max_future_days to table settings. The feature is used to parse timestamps to have a MAX number of days ahead of now(). - Add Avro support. - Transform execution reliability updates and logging. - Remove Lambda support for intake components (now using on-box microservices for better reliability) - Additional "limiter" field support added for fields in transforms.

Query - Additional Service Pool to Table mapping support.

Portal/API - API Updates for table settings and new features (Avro etc). - API updates for Merge Pools feature and Web instance types. - Add Portal, API to Web instance. - Update to Clusters Page, page structure and functionality. - Backward compatibility updates to API structure. - API Bugfix for typo that stopped Kafka deployments

Life Cycle - Added the ability for merge-pools, so merge servers can be specified for projects and tables. - Deleted Table functionality updates.

Observability - Grafana Dashboard improvements and consolidation. - Default Alert Dashboard

Command and Control - Cloudformation updates. - Removal of non-used promdisco tool

Bug Fixes

Ingest - Bug fixes on the handling of Epoch timestamp values and conversion to DateTime/DateTime64. - Bad Transform protections added with additional logging. - Sort key ordering fixes when NULL fields are encountered in input stream.

Portal/API - Portal Sort Key to be made optional over required.