25 July 2022 - v3.15.6
over 2 years ago by Alan Hardy
AWS Cloud Formation (HDXCTL) / Kubernetes (HKT)
Download : hdxctl-v3.15.6 / hkt-v3.15.6
Release Locations
AWS Cloud Formation
wget -O hdxctl https://hdx-infrastructure.s3.amazonaws.com/hdxctl-v3.15.6 && chmod +x hdxctl
Kubernetes (GKE)
wget -O hkt https://hdx-infrastructure.s3.amazonaws.com/hkt/hkt-v3.15.6 && chmod +x hkt
Query
- Add: circuit breakers to control agg/sort disk spilling thresholds
- BugFix: default query pool mismatch
- BugFix: Query Timeout Circuit Breaker
- BugFix: Open ended upper time range incorrectly triggered max time range circuit breaker
Core
- Add: Namespace dictionaries and functions to Project (backwards-compatible)
- BugFix: Improper JSON escaping in logs - handle double quote and backslash
- Add: make S3 Endpoint configurable
- Enhancement: unified metrics and traces
- BugFix: truncate logs issue
- BugFix: Logs timestamp standardization
- Bugfix: Logs and Metrics for ClickHouse exceptions
- Add: support configuring external otel trace server
Ingest
- BugFix: Handle floating points from JSON more carefully for indexers
- BugFix: Indexer partition rejected metric
- Add: double-checks on every storage upload, increase max retries to 10
- Enhancement: Stream validator endpoint
- Enhancement: Batch to use Turbine as a service
- Add: streaming http query param support
- K8s: batch head handle special folders in GCS
- BugFix: Ignore extra values in content type header
- BugFix: JSON flattening depth deserialization
- Removal: autoingest assets that are no longer used
- Enhancement: Support multiple sources and transform per table for autoingest
- Add: hot and cold partition count metrics to stream peer
- K8s: Disable automatic decompression of gzip contents when reading from GCS
- Add: Support custom null values
Portal/API
- Enhancement: update summary pages UI
- Enhancement: updates transform UI page to use validator
- Move jobs page up to the top level UI
- BugFix: create a new table UI
- BugFix: remove hdx and hydrologs tables in UI select
- BugFix: sort key & shard key updates don't stick
- Enhancement: Custom Dictionary and Custom Function API endpoints
- Modify: Standardize circuit breaker naming - prefix
hdx_
- Add: use cli dns service for named template email provider in API
- Enhancement: add global query options to org level API
- Modify: table defaults for stream and autoingest
- Add: table handles multiple autoingest settings
Infrastructure/Command & Control
- BugFix: create cluster fails after client rollback
- Add:
merge_head_batch_size
tunable and adjust default to 10000 from 500 - Add: support for non-standard s3 endpoints (minio, aliyun...)
- Add: extra metadata for indexer metrics
- Add:
batch_head_concurrency
&batch-peer-threads
tunables - Add:
superset_worker_memory
tunable - K8s: AWS web_identity Auth Mode
- K8s: fix traefik-cfg breaks on empty allowlist
- K8s: update scale defaults
- K8s: add vector logging-to-buckets in kubernetes
- K8s: adds retries to the init-turbine-api job
- K8s: catch error apply resources and show what is changing
- K8s: add json log parsing to vector config
- Modify: bucket logging layout and add pool annotations