20 March 2023 - v3.34.5

You can download version 3.34.5 here.

Hydrolix Kubernetes Tool (HKT) hkt-v3.34.5

wget -O hkt https://hdx-infrastructure.s3.amazonaws.com/hkt/hkt-v3.34.5 && chmod +x hkt

Upgrade on GKE:

kubectl apply -f "https://www.hydrolix.io/operator/v3.34.5/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

Upgrade on EKS:

kubectl apply -f "https://www.hydrolix.io/operator/v3.34.5/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

General

  • UI: Updated table health widget to show the partition count, shard count, raw data size, compression ratio, and ingestion latency for a table.
  • API: Added support for data storage in Microsoft Azure Storage.
  • API: Improved messaging for batch job errors, including error messages for authentication, indexer, heartbeat, and file read failures.
  • Config - added support for tolerations in Kubernetes.
  • UI: Added labels for days in lifecycle management options.
  • UI: Added units to table, batch job, and ingestion source data.
  • UI: Added batch job errors to the sidebar.
  • API: Added option to manually specify UUID when creating projects and tables.
  • API: Added option to manually specify query output format at the organization level.
  • API: Added guarantee that a table always has a default transform, as long as at least one transform exists for that table.
  • API: Added validation for stream duration settings.
  • API: Added intake support for HTTP auth tokens.
  • API: For queries that do not use the entire query pool, randomized peers chosen to better distribute load.
  • API: Added an endpoint to upload an entire HDX partition, including a catalog entry.
  • Intake: Added support for HDX_SKIP_REJECTS and HDX_LOG_REJECTS environment variables to control reject recording and logging.
  • Intake: Added support for specifying a field with from_json_pointers.
  • Ingest: Removed support for deprecated legacy ingestion settings layout.

Services

Notable Bug Fixes

  • UI: Removed spurious false value from certain query results.
  • UI: Removed SQL field from table merge settings.
  • UI: Prevented users from modifying the auto_view directly.
  • Query: Removed "obselete setting" ClickHouse warnings on client connection.
  • UI: Allowed UI to specify a transform with an empty SQL field.
  • API: Stopped treating user emails as case-sensitive.
  • API: Fixed bug that prevented summary table creation when raw table content included a dash.

Platforms

  • API: Added support for custom load balancer tags on AWS.
  • Intake: Added ability to configure separate AWS credentials for separate Kinesis streams.
  • Intake: Added ability to configure Kinesis credentials with an environment variable.
  • Kinesis Ingest: Changed format for offset storage (breaking change).