10 June 2024 - v4.14.1

ClickHouse upgrade to v23.8.10.43 LTS, improved ingest reliability

Notable New Features

  • ClickHouse upgrade to v23.8.10.43 LTS
    • Supports parameterized views, query caching (experimental), and other features listed in our v4.14 Upgrade Guide.
  • Improved Ingest Reliability
    • We've included DNS fixes and Traefik improvements.
  • Multi-Cloud Credentials
    • Connect to different cloud accounts in Google Cloud, AWS, Azure, and Linode by defining sets of credentials. Currently, this can help alleviate problems with cloud storage rate limits. It also paves the way for more features based on this flexibility.

Upgrade on GKE:

kubectl apply -f "https://www.hydrolix.io/operator/v4.14.1/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

Upgrade on EKS:

kubectl apply -f "https://www.hydrolix.io/operator/v4.14.1/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

General

  • API: Multiple sets of cloud credentials are now supported
  • API: Created new endpoint to export and import catalogs
  • Core: Upgraded to ClickHouse v23.8.10.43 LTS, Clang18 + libc++15, C++23 standard, and AWS C++ SDK 1.11.285
  • Data: Batch import now supports summary table generation
  • Data: Added intake-head 429 responses when backed up
  • UI: New summary table analysis form

Bug Fixes

  • API: Conflicting transforms are now rejected
  • API: Keycloak database is backed up every update, not every hour
  • API: Fixed table breakage related to cold_data_max_age_days
  • API: PUT method on a storage doesn't require a new storage name
  • API: Job status is now updated via 1-minute cron job
  • API: Updated KeyCloak backup script to stream data, fixing large transfer errors
  • API: Fix 500 error when updating summary max_partitions
  • Core: Improved ip removal logic from dns cache
  • Core: ZooKeeper disconnect is now handled gracefully
  • Data: Determine S3 file size before download to save memory
  • UI: query editor: Fix formatting for dashes in column names in query editor
  • UI: Invalid transform fields are now shown
  • UI: Alter jobs "commit" button is not shown until alter job is done