18 October 2023 - v3.50.9

You can download version 3.50.9 here.

Upgrade on GKE:

kubectl apply -f "https://www.hydrolix.io/operator/v3.50.9/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

Upgrade on EKS:

kubectl apply -f "https://www.hydrolix.io/operator/v3.50.9/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

General

  • Service log level can be modified with the log_level tunable
  • Add stream_concurrency_limit tunable to adjust stream head concurrent connections
  • Gracefully fallback to default merge pools if configured pool cannot be found
  • Distribution of secrets is limited to pods on a "need-to-know" basis
  • Node type targeting now works for Crunchy Data pods
  • Keycloak pods have been scaled up in prod and mega profiles
  • Enabled brute force protection by default on Keycloak realm
  • Always 404 when UUIDs in URL are incorrect
  • Add primary_key to table API endpoint
  • Add support for pushing metrics and integrate it into new merge
  • Removed datetime64 as a type on transforms
  • Check traefik events as part of operator readiness probe
  • Add autovalue for Kinesis stream ARN and name
  • Make hydrolix pods run as turbine user
  • GOMEMLIMIT environment variable is now applied to all pods for better scaling
  • Ingest pods now support the /config-version endpoint
  • Add vector pod options for disabling logging
  • New project naming UI allows project name change
  • UI now has the ability to delete projects
  • Add timerange and limit params to audit trail UI
  • Update franz-go kafka library to 1.14.4
  • Improved logging for Akamai SIEM

Notable Bug Fixes

  • Mitigate dropping of MESSAGE_QUEUE_MAX_ROWS value, defaults to 100
  • More carefully handle writing large events to internal message queue
  • Fixed merge not reporting raw_size and uncompressed_mem_size
  • Merge now considers both mem_size and uncompressed_mem_size when merging summary tables
  • Fixed broken CTEs and type matching when timerange filter is on
  • Fixed ingest dictionary lookups broken on scale-up: wait for dictionaries and config to be loaded
  • Fixed secret and env related problems affecting kinesis and siem on v3.50.x
  • Fixed redpanda data races and update default write limit
  • Fixed operator crash creating kinesis
  • Error gracefully on compound type mismatch
  • Fixed UI bug in CSV transform generation
  • Fixed UI query reference tab highlighting
  • Shard keys can no longer be changed after they are set
  • Updates to pools no longer fail on incorrect validation
  • Fixed Operator crashes when using target cpu utilization scale setting
  • Handle 412 from Akamai SIEM