Releases
ProductPricingDocsBlog
Back to All

24 July 2024 - v4.16.2

about 2 months ago by Rich Vanderwal

Selective dictionary loading, intake pooling, UI authentication logs, and custom views in the UI

Notable New Features

  • Selective Dictionary Loading
    • Services will only load dictionaries they need, saving service start-up costs and ongoing memory use.
  • Pooled Intake and Stream Heads
    • Separate pools of intake services can now be defined to isolate different intake workloads.
  • Authentication Logs in the UI
    • View authentication logs in the Hydrolix UI via the new "Auth Logs" tab on the Security page.
  • Custom SQL Views in the UI

Upgrade on GKE:

kubectl apply -f "https://www.hydrolix.io/operator/v4.16.2/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&gcp-storage-sa=${GCP_STORAGE_SA}"

Upgrade on EKS:

kubectl apply -f "https://www.hydrolix.io/operator/v4.16.2/operator-resources?namespace=${HDX_KUBERNETES_NAMESPACE}&aws-storage-role=${AWS_STORAGE_ROLE}"

General

  • API: Views now cannot be created containing non-existent fields.
  • API: To support selective dictionary loading, the API now supports dictionary load levels
  • API: Query circuit breakers can now be applied at the project and table levels
  • API: Pre-baked transform templates can be submitted and used via the HTTPS API
  • API: Removed authorization from /v1/login endpoint schema
  • API: SIEM access_details settings can now include a limit field
  • Control: intake-heads and stream-heads can now be pooled with separate routes to each
  • Control: Added silence_linode_alerts tunable
  • Core: Services selectively load dictionaries only as needed
  • Core: Query peer now truncate long lists of partitions in the logs
  • Core: Added hdx_query_catalog_timeout_ms timeout to postgres queries
  • Core: Dictionaries without dictionary_load_level setting are allowed
  • Data: libwurfl updated to v1.13.1.0
  • Data: Added support for auto ingesting data from Azure via ServiceBus
  • Data: Support added for batch imports from any of the four cloud providers to any of them using multicred
  • Data: Intake heads now return 429 responses when intake_head_max_outstanding_requests threshold is exceeded
  • UI: New audit trail page has been added to the Security tab
  • UI: New custom view creation, editing, and deleting in the UI.

Bug Fixes

  • API: Transform conflicts no longer reported when primary property is omitted
  • API: Transform conflicts due to summary SQL changes are now handled properly
  • API: Transform datatypes are no longer accidentally converted
  • API: Customer-set circuit breakers are overridden when API makes internal Clickhouse querys, avoiding operational errors
  • API: Jobs must now have non-active status before deletion
  • API: API now returns a 400 error when it can't validate summary SQL
  • API: hdx_query_max_result_bytes minimum value changed from 0 to 10,000
  • API: hdx_query_unlimited_cnf setting now works at org-level
  • API: Fixed return messages for /v1/users/{user_id}/remove/roles/ when user doesn't have role
  • API: activate_user in /v1/invites endpoint works as expected
  • API: The populate catalog endpoint /v1/orgs/{org_id}/catalog/upload/ now specifies a unique set of root_path, data_path and storage_id
  • API: Storage endpoint behavior fixed by enforcing unique storage names
  • API: Incorrect transform conflict validation fixed
  • Control: Merge peers no longer restart when max_partitions_per_candidate is changed
  • Core: Summary table parent can't be changed by modifying the summary query
  • Core: Query retrieves only partitions needed when SQL contains IN
  • Some query stats were not being reported in queries with LIMIT. This has been fixed.
  • Data: S3 region is only looked up when we have no endpoint
  • Data: Fixed pathing issues with Azure batch import
  • Data: Fixed issue with single Azure file download in batch
  • UI: Query options sidebar is no longer pre-populated with options from the query_options endpoint
  • UI: Linode region options are fixed for bucket creation
  • UI: CSV transforms that incorrectly have JSON sample data are now handled properly.
  • UI: Endpoint is no longer required in bucket configuration form
  • UI: SQL text after comments isn't ignored