26 November 2021 - v2.23.12

Release Location

wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v2.23.12 && chmod +x hdxctl

Features

Deployment
HDXCTL: Upgrade boto3 to 1.20.6
HDXCTL: Added support for setting a boundary policies and ARN for roles used in deployment process.

Configuration and Control
Scaling: Spot Instance percentages now based on the default pool settings.
Security: Security Update to RDS applied with newest patches and version updated to 11.13.

Security
AWS - Enable Customer-owned KMS Key support - hdxctl update --use-s3-kms-key true client_id clusterid. Note requires TLS support (below) turned on as well.
AWS - Enable TLS Endpoint Support - hdxctl update --use-https-with-s3 true client_id clusterid

Alter
Alter-Peer: Placeholders enabled to allow for row deletion and other database/table mechanics to change data once encode.
Alter-Peer: Scaling and Control integration.

Ingest
Transform: Allow nulls in csv quote field
Batch: Change batch max_minutes_per_partition setting default to 360

Observability
Superset: Increase default max header size for superset proxy

Query
Query: Improved I/O scheduling for faster, more reliable performance between compute and cloud storage.
Query: Resiliency enhancements added to Peer restarts.
Query: Added missing timerange_sec, max_partitions and new admin_comment settings/circuit breakers
Query: Added support for custom sql settings via SETTINGS hdx_ within SQL
Query: Be able to use a special header X-HDX-query-settings as query settings for a query

Bug Fixes

Configuration and Control
HDXCTL: Fix incompatibility issue with ubuntu 18.04 vs botocore

Ingest
Ingest: Improve automatic datetime value handling
Ingest: Reduce log noise for greater clarity. Reduce log rejection noise.

Query
Write to S3: Fix for output results to S3 file
Pools: Pool_name Bugfix for quoted strings.