v2.18.5
Release Location
Ingest - Ingest: Add max_future_days limiting for tables. Enables the ability to filter date times that are presented too far in the future. Default is this is turned off. - Ingest (Batch): Add additional information into the log to include the the input path. - Ingest (Batch): Ensure Batch Peers always remove SQS entries from the queue. - Ingest (Kafka): Add offset for Kafka peer readers.
Query - Query: Enable ability to see active queries. - Query: Add query server metrics to Prometheus
Portal/API - Portal: Additional input validation and error handling. - Portal: Enable Shard Key in Table settings. - Portal: Add retry behaviour in Batch Jobs in the UI. - Portal: Add delete stopped jobs from the UI. - Portal: Add Kafka section to Cluster UI. - API: Change behaviour of Transforms and views PATCH to PUT.
Observability - Grafana: New Query Dashboard. Deployed in Grafana. Will be deployed with build later.
Bug Fixes
Ingest - Ingest: Intake will no longer create SQS queues if they do not exist, instead logging an error. - Ingest (Batch): Fix a goroutine leak when batch ingesting JSON that can exhaust memory.
Query - Query: Fixed segfault in query engine where unindexed partitions were queried with Indexed ones. - Query: Add partition option to query API (for debugging purposes). - Query: HTTP client bug fix where at high load the HTTP client could fail when collecting S3. - Query: Do not include internal queries in hdx.active_queries
Lifecycle - Merge: Partitions are sorted by modification date (most recent first) prior to being provided to the Merge SDK. The Merge SDK will use the first partition as the prototype for what a merged partition's schema should look like. - Merge: Bug fix where a non-indexed column is changed to a column caused the Merge-peer to crash. - Merge: Resolve an issue where merge components were unable to locate the queue when running without merge pools