Releases
ProductPricingDocsBlog
Back to All
added

11 November 2021 - v2.22.6

about 3 years ago by Jason Turner

Download: hdxctl-v2.22.6

Release Location

wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v2.22.6 && chmod +x hdxctl

Features

Deployment

  • Support for AWS ap-southeast-2 and ap-northeast-1
  • Replace Amazon Load Balancers with Nginx and NLB's
  • Move Load balancers to the Core side of the stack.
  • Publish native protocol port for Query access on load balancers, now able to use DBeaver type tools

Configuration and Control

  • Improve availability of docker repos.
  • Increase IAM Policy granularity
  • Update to the hdxctl scale to have --minimal to bring up 1 instance of Grafana, Superset and Prometheus
  • Add service column to hdxctl instances command for improved instance definition.
  • hdx project name is now a reserved name and can not be used.

Ingest

  • Add support for schema-less AVRO support.
  • Improve Merge Logging
  • Improve and reorganize stream peer shutdown sequence for faster and more reliable shutdown.
  • Stream Rejects rows on failure instead of whole batches when limits are hit.
  • Add ability within transform to specify more than one potential input source field.

Observability

  • Collect internal Grafana metrics in Prometheus
  • Grafana World Map plugin installation
  • Grafana Tree Map plugin installation
  • Grafana Diagram plugin installation
  • Expose all catalog column metadata on tables.
  • Add version endpoint accessible via NGINX
  • Add NGINX metrics to Prometheus

Portal/API

  • Add AWS t2 family server type to intake-misc supported types.
  • Change API definition to OpenAPIv3 format with various improvements.
  • Add Servers field to the OpenAPIv3 definition for better documentation support.
  • Keycloak performance improvements.
  • Portal: Add drawer to Users page in the UI.
  • Portal: Add error messages to Job detail drawer.

Query

  • query.timeout_secs - Introduce a query timeout circuit breaker Query API HTTP Options
  • storage.read_max_partitions - Maximum number of partitions to be read for a query circuit breaker Query API HTTP Options
  • storage.max_timerange_secs - Maximum time range a query can span circuit breaker Query API HTTP Options
  • Improve connection handling and reliability when a peer is removed or added to a query pool.

Bug Fixes

Configuration and Control

  • Bugfix for Manifest syncing failures when null resolution types are experienced.
  • HDXCTL: Fix silently failing bootstrap template build
  • Fix missing permissions for cached docker images that could affect deployment and update.
  • Fix issue causing the need to run route twice when updating a cluster.
  • Fix connection errors on webs instances during rolling restarts
  • Bug fix for new client stacks not getting DNS entries
  • Bug fix for old ClientID's, where old ClientID's could not be re-used in a new deployment

Ingest

  • Fix stream double counting of some CSV-in-JSON rows.
  • Resolves an issue where Batch Peer can deadlock and take no more work after certain failure modes

Observability

  • Constrain Journald data to 1G
  • Limit size of container logs to 10M for instances running Podman.

Portal/API

  • Allow for null "resolution" in non datetime types in transform definition.
  • Various Portal functionality fixes

Query

  • Various networking and reliability Bug Fixes.