Hydrolix Tunables List
A listing of Hydrolix tunables
A listing of HTN tunables used by Hydrolix. These tunables are set in the hydrolixcluster.yaml configuration file, under spec:.
| Tunable Name | Description | Default | Examples |
|---|---|---|---|
| acme_enabled | Automatically generate and renew SSL certs for your Hydrolix domain. Will override any existing Kubernetes secret named traefik-tls. | False | None |
| admin_email | The email address of the Hydrolix cluster administrator. | None | None |
| autoingest_unique_file_paths | Enable unique file paths from object store by ignoring duplicate paths. | False | None |
| aws_credentials_method | DEPRECATED: Use db_bucket_credentials_method. | None | ["static", "instance_profile"] |
| aws_load_balancer_subnets | Subnets to assign to the load balancer of the Traefik service when running in EKS. | ["subnet-xxxx,mySubnet"] | |
| aws_load_balancer_tags | Additional tags to be added to the load balancer of the Traefik service when running in EKS. | ["Environment=dev,Team=test"] | |
| traefik_service_annotations | Additional annotations for Traefik service. | {} | None |
| azure_blob_storage_account | The storage account to access an Azure blob storage container. | None | None |
| basic_auth | A list of Hydrolix services that should be protected with basic auth when accessed over HTTP. | [] | None |
| batch_peer_heartbeat_period | How frequently a batch peer should heartbeat any task it's working on as a duration string. | 5m | |
| bucket | DEPRECATED: Use db_bucket_url. | None | None |
| client_id | DEPRECATED: Use hydrolix_name and db_bucket_url. | None | None |
| catalog_db_admin_user | The admin user of the PostgreSQL server where Hydrolix metadata is stored. | turbine | None |
| catalog_db_admin_db | The default database of the admin user on the PostgreSQL server where Hydrolix metadata is stored. | turbine | None |
| catalog_db_host | The PostgreSQL server where Hydrolix metadata is stored. | postgres | None |
| catalog_db_port | The PostgreSQL server port where Hydrolix metadata is stored. | 5432 | None |
| catalog_intake_connections | Connection pool settings for intake services that connect to the PostgreSQL server where Hydrolix metadata is stored.Available options: 1. max_lifetime - The max duration that a connection can live before being recycled.2. max_idle_time - The max duration that a connection can be idle before being closed.3. max - The max number of connections that can be opened by each intake service that connects to the PostgreSQL server.4. min - The minimum number of connections to keep open to the PostgreSQL server.5. check_writable - If set to true, when a connection is opened to the PostgreSQL server, ensure the server can handle writes. | {"max_lifetime": "10m", "max_idle_time": "1m"} | None |
| clickhouse_http_port | The dedicated port for the ClickHouse HTTP interface. | 8088 | None |
| data_service_termination_grace_period | Termination grace period for most data services. | 120 | None |
| db_bucket_credentials_method | The method Hydrolix uses to acquire credentials for connecting to cloud storage. | web_identity | ["static", "ec2_profile", "web_identity"] |
| db_bucket_endpoint | The endpoint URLfor S3 compatible object storage services.Not required if using AWS S3 or if db_bucket_url is provided. | None | None |
| db_bucket_name | The name of the bucket for Hydrolix to store data in. Not required if db_bucket_url is provided. | None | None |
| db_bucket_region | Not required if it can be inferred from db_bucket_url. | None | ["us-east-2", "us-central1"] |
| db_bucket_type | The object storage type of the bucket you would like Hydrolix to store data in. Not required if db_bucket_url is provided. | None | ["gs", "s3"] |
| db_bucket_url | The URL of the cloud storage bucket you would like Hydrolix to store data in. | None | ["gs://my-bucket", "s3://my-bucket", "https://my-bucket.s3.us-east-2.amazonaws.com", "https://s3.us-east-2.amazonaws.com/my-bucket", "https://my-bucket.us-southeast-1.linodeobjects.com", "https://minio.local/my-bucket"] |
| db_bucket_use_https | If true use HTTPS when connecting to the cloud storage service.Inferred from db_bucket_url if possible. | True | None |
| default_query_pool | Name of the default query pool. | query-peer | None |
| dns_server_ip | The IP address of the DNS server used for performance-critical purposes. | None | None |
| use_hydrolix_dns_resolver | If true, use Hydrolix DNSResolver. If false, use system resolver | True | None |
| dns_gcs_max_ttl_secs | Max DNS TTL for GCS storage. It is the longest period of time for which the DNS resolver can cache a DNS record before it expires and needs to be refreshed.max_ttl=0 means DNS cache strictly respects the TTL from the DNS query response. | 0 | None |
| dns_aws_max_ttl_secs | Max DNS TTL for AWS and S3-compatible storages.It is the longest period of time for which the DNS resolver can cache a DNS record before it expires and needs to be refreshed. max_ttl=0 means DNS cache strictly respects the TTL from the DNS query response. | 0 | None |
| dns_azure_max_ttl_secs | Max DNS TTL for Azure storage. It is the longest period of time for which the DNS resolver can cache a DNS record before it expires and needs to be refreshed. max_ttl=0 means DNS cache strictly respects the TTL from the DNS query response. | 0 | None |
| dns_gcs_max_resolution_attempts | Maximum number of attempts made by the DNs Resolver for GCS storage in a given DNS refresh cycle. | 1 | None |
| dns_aws_max_resolution_attempts | Maximum number of attempts made by the DNS Resolver for AWS and all s3 compatible storages in a given DNS refresh cycle. | 1 | None |
| dns_azure_max_resolution_attempts | Maximum number of attempts made by the DNS Resolver for Azure storage in a given DNS refresh cycle | 1 | None |
| domain | DEPRECATED: Use hydrolix_url. | None | None |
| disable_disk_cache | If true, query peers will immediately delete partition metadata from disk after use | False | None |
| disk_cache_cull_start_perc | Percentage of cache disk space used before starting to remove files. | 75 | None |
| disk_cache_cull_stop_perc | Percentage of cache disk space used before stopping removing files. | 65 | None |
| disk_cache_redzone_start_perc | Minimum percentage of cache disk space used to be considered as redzone. | 90 | None |
| disk_cache_entry_max_ttl_minutes | Max TTL for a cache disk entry. It is the longest period of time for which the LRU disk cache can save an entry before it expires. | 360 | None |
| max_http_retries | Maximum times to retry any query-related HTTP requests that fail. | 3 | None |
| max_exp_backoff_seconds | Cap for exponentially back off sleep time. | 20 | None |
| initial_exp_backoff_ms | Sleep time starts from this value and exponentially grows with retry count. | 0 | None |
| eks_product_code | EKS product code for use with Amazon Marketplace. | 6ae46hfauzadikp9f8npdbh9v | None |
| exp_backoff_growth_factor_ms | Every sleep will use this as multiplicative factor, ie 2^i * (growth_factor)ms. | 50 | None |
| exp_backoff_additive_jitter | True: (growthfactor)(1 + jitter), False: growthfactor(jitter). | True | None |
| enable_traefik_access_logging | If set to true, Traefik will log all access requests.WARNING This will produce a very high and potentially unmanageable amount of logs | False | None |
| hdx_traefik_auth_workers | Number of async workers gunicorn will create for services requests. Default - 1 | 1 | None |
| enable_traefik_hsts | If set to true, Traefik will enforce HSTS on all its connections.WARNING: This may lead to hard-to-diagnose persistent SSL failures if there are any errors in SSL configuration, and cannot be turned off later. | False | None |
| enable_password_complexity_policy | If set to true, uses the default password policy:Minimum length: 8 charactersUppercase characters: 1Lowercase characters: 1Digits: 1Special characters: 1Not recently used: Past 24 passwordsExpire password: 90 daysNot usernameNot email | False | None |
| password_expiration_policy | Number of days to expire password | None | None |
| traefik_hsts_expire_time | Expiration time for HSTS caching in seconds. | 315360000 | None |
| http_connect_timeout_ms | Maximum time to wait for socket connection to cloud storage to complete | 300 | None |
| http_ssl_connect_timeout_ms | Maximum time to wait for ssl handshake during connection to cloud storage to complete | 1000 | None |
| http_response_timeout_ms | Maximum time to wait for receiving HTTP headers to complete while reading from cloud storage | 1000 | None |
| http_read_timeout_ms | Maximum time to wait between a socket read and cloud storage having data ready to be read | 1000 | None |
| http_write_timeout_ms | Maximum time to wait before uploading partition to cloud is complete | 10000 | None |
| io_perf_mappings | Internally used presets for io_perf_mode. Parsed as JSON Array(Array(Int)) | [[2097152, 256, 256], [6291456, 128, 128], [12582912, 64, 64]] | None |
| disable_traefik_http_port | If true the load balancer will not forward to Traefik on port 80. When TLS is enabled, this port is only used to redirect to HTTPS. Otherwise this is the main way to access all services. | False | None |
| disable_traefik_https_port | If true the load balancer will not forward to Traefik on port 443. Only relevant if TLS is enabled | False | None |
| disable_traefik_native_port | If true the load balancer will not forward to Traefik on the ClickHouse native protocol port. This is port 9440 when TLS is enabled or 9000 if not. | False | None |
| disable_traefik_mysql_port | If true the load balancer will not forward to Traefik on the ClickHouse MySQL interface port. This is port 9004. | False | None |
| disable_traefik_clickhouse_http_port | If true the load balancer will not forward to Traefik on port 8088. This port provides a ClickHouse compatible query interface rooted at the root of the service rather than at a subpath. | False | None |
| enable_query_auth | When enabled requests to the query service, URLpaths starting with /query will require authentication. | False | None |
| user_acl_refresh_interval_secs | Frequency at which user ACL permissions are refreshed (in secs) | 30 | None |
| user_token_refresh_interval_secs | Frequency at which user tokens are refreshed (in secs) | 240 | None |
| user_token_expiration_secs | user token expiration period (in secs) | 1800 | None |
| auth_http_response_timeout_ms | Maximum time to wait for receiving HTTP headers from auth endpoint (turbine-api) in response to user permission requests | 2000 | None |
| auth_http_read_timeout_ms | Maximum time to wait for a socket read for user-permission data from auth endpoint (turbine-api) | 2000 | None |
| enable_vector | Run vector to send Kubernetes pod logs to json files in a bucket and to the internal logs topic. Default inferred from the value of scale_off. | None | None |
| disable_vector_kafka_logging | Prevent vector from emitting logs to Redpanda. | False | None |
| disable_vector_bucket_logging | Prevent vector from sending logs to the bucket. | False | None |
| env | Environment variables to set on all Kubernetes pods that are part of the Hydrolix cluster. | {} | None |
| force_container_user_root | Set the initial user for all containers to 0 (root). | False | None |
| host | DEPRECATED: Use hydrolix_url | None | None |
| http_port | The port to serve Hydrolix plain HTTP on. | None | None |
| https_port | The port to serve Hydrolix HTTPS on. | None | None |
| hydrolix_name | The name you would like to assign your Hydrolix cluster. Will be the same as the namespace name if not specified | None | None |
| hydrolix_url | The URLyou would like to use to access your Hydrolix cluster. | None | ["https://my-host.hydrolix.live", "https://my-host.mydomain.com", "http://my-host.local"] |
| ip_allowlist | A list of CIDR ranges that should be allowed to connect to the Hydrolix cluster load balancer. | ["127.0.0.1/32"] | None |
| intake_head_index_backlog_enabled | Whether to absorb received buckets in a backlog prior to indexing in intake-head to allow for more buffer for absorption in the face of spikes of traffic or throughput disruptions in indexing or uploading of partitions. If enabled, the newest data received will indexed ahead of older data when the backlog grows. | False | None |
| intake_head_index_backlog_max_mb | Controls the maximum size in MB that the indexing backlog on intake-head is allowed to grow before either dropping data or slowing new entries depending on the configured value of intake_head_index_backlog_trim_enabled. Only applicable if intake_head_index_backlog_enabled is true. | 256 | None |
| intake_head_index_backlog_purge_concurrency | Controls the number of workers used to purge buckets from the intake-head backlog when the max size is breached. Only applicable if intake_head_index_backlog_enabled is true. | 1 | None |
| intake_head_index_backlog_max_accept_batch_size | Controls the maximum number of buckets accepted from ingestion and added to the backlog at a time. Only applicable if intake_head_index_backlog_enabled is true. | 50 | None |
| intake_head_max_outstanding_requests | Configures the maximum number of requests that an intake-head pod will allow to be outstanding and in process before rejecting new requests with a 429 status code response. If not configured or set to 0, intake-head pods will never reject new requests. | 0 | None |
| intake_head_accept_data_timeout | Configures the maximum duration that intake-head will wait for a request to be accepted into the partition creation pipeline. If the timeout is reached, the request will be rejected with a 429 status code response. If not configured or set to 0, intake-head pods will not timeout. | 0s | None |
| intake_head_raw_data_spill_config | Provides configuration of the spill functionality for raw data in intake-head where ingested data is spilled to object storage when partition generation is slowed on a particular intake-head pod. Supported keys are:enabled``max_concurrent_fetch``fetch_lock_expire_duration``max_concurrent_spill``max_attempts_spill``num_partitions``empty_fetch_pause_duration | {"enabled": "false", "max_concurrent_fetch": "1", "fetch_lock_expire_duration": "10m", "max_concurrent_spill": "20", "max_attempts_spill": "5", "num_partitions": "10", "empty_fetch_pause_duration": "30s"} | None |
| intake_head_catalog_spill_config | Provides configuration of the spill functionality for catalog adds in intake-head whereby catalog adds are spilled to object storage when catalog interactions are slowed or fail on a particular intake-head pod. Supported keys are:enabled``max_concurrent_fetch``fetch_lock_expire_duration``max_concurrent_spill``max_attempts_spill``num_partitions``empty_fetch_pause_duration | {"enabled": "false", "max_concurrent_fetch": "1", "fetch_lock_expire_duration": "10m", "max_concurrent_spill": "20", "max_attempts_spill": "5", "num_partitions": "10", "empty_fetch_pause_duration": "30s"} | None |
| kafka_careful_mode | False | None | |
| kafka_tls_ca | A CA certificate used by the kafka_peer to authenticate Kafka servers it connects to. | None | |
| kafka_tls_cert | The PEM format certificate the kafka_peer will use to authenticate itself to a Kafka server. | None | |
| kafka_tls_key | The PEM format key the kafka_peer will use to authenticate itself to a Kafka server. | None | None |
| kinesis_coordinate_strategy | The strategy to use for coordinating Kinesis peers for a Kinesis source. Possible values are EXTERNAL_COORDINATOR or ZOOKEEPER | EXTERNAL_COORDINATOR | None |
| kinesis_coordinate_period | For Kinesis sources, how often the coordination process runs which checks for the available shards and peers and distributes consuming amongst available peers | 10s | None |
| kubernetes_cloud | DEPRECATED: Use kubernetes_profile. | None | ["aws", "gcp"] |
| kubernetes_premium_storage_class | The storage class to use with persistent volumes created in Kubernetes for parts of a Hydrolix cluster where throughput is most critical. | None | None |
| kubernetes_profile | Use default settings appropriate to this type of Kubernetes deployment. | generic | ["gke", "eks", "lke"] |
| kubernetes_storage_class | The storage class to use with persistent volumes created in Kubernetes as part of a Hydrolix cluster. | None | None |
| logs_sink_type | Type of logs sink. | kafka | None |
| logs_sink_local_url | The full URI to make local HTTP request to. | http://stream-head:8089/ingest/event | None |
| logs_sink_remote_url | The full URI to make remote HTTP request to. | None | |
| logs_sink_remote_auth_enabled | When enabled, remote HTTP will use basic auth from curated secret. | False | None |
| logs_http_remote_table | An existing Hydrolix <project.table> where the data should land in remote cluster. | hydro.logs | None |
| logs_http_remote_transform | A transform schema for ingest in remote cluster. | megaTransform | None |
| logs_http_table | An existing Hydrolix <project.table> where the data should land. | hydro.logs | None |
| logs_http_transform | A transform schema for ingest. | megaTransform | None |
| logs_kafka_bootstrap_servers | A comma separated list of Kafka bootstrap servers to send logs to. | redpanda | None |
| logs_kafka_topic | A Kafka topic to send logs to. | logs | None |
| logs_topic_partition_count | The number of partitions to assign to the logs topic for stream processing. | 81 | None |
| merge_head_batch_size | Number of records to pull from the catalog per request by the merge head. | 10000 | None |
| merge_interval | The time the merge process waits between checking for mergeable partitions. | 15s | None |
| merge_max_partitions_per_candidate | The maximum number of partitions per merge candidate. | 100 | None |
| merge_max_candidates | Number of candidates to produce per merge target each cycle. | 100 | None |
| merge_min_mb | Size in megabytes of the smallest merge tier. All other merge tiers are multiples of this value. | 1024 | None |
| merge_dispatch_frequency | How often a slot should be checked for exceeding max_idle. Expressed as duration string (e.g. 5s) | 5s | None |
| merge_first_era_frequency | How often merge candidates should be constructed for the first era. | 10s | None |
| merge_second_era_frequency | How often merge candidates should be constructed for the second era. | 60s | None |
| merge_third_era_frequency | How often merge candidates should be constructed for the third era. | 60m | None |
| merge_streaming_selector | Whether or not to use the Streaming Candidate Selector | True | None |
| merge_primary_window_width | Specifies the interval used to further filter partition selection queries. Smaller values limit the number of records the database needs to produce, but can increase query count. | 1080h | None |
| merge_candidate_concurrency | Number of concurrent MergeCandidate construction queries to run. | 6 | None |
| merge_controller_enabled | Whether or not the next generation merge controller is enabled. | False | |
| native_port | The port to serve the ClickHouse plaintext native protocol on if applicable. | 9000 | None |
| native_tls_port | The port to serve the ClickHouse TLS native protocol on if applicable. | 9440 | None |
| mysql_port | The port to serve the ClickHouse MySQL interface on if applicable. | 9004 | None |
| mysql_port_disable_tls | When True, Traefik will not use TLS configuration on MySQL TCP route. | True | None |
| oom_detection | Configuration options for detecting indexing OOM scenarios and retry with smaller data sizes if possible for services that perform ingest. Outer keys are names of the ingest services. The supported services are:intake-head``kafka-peer``kinesis-peer``akamai-siem-peerAvailable keys under each service are:k8s_oom_kill_detection_enabled``k8s_oom_kill_detection_max_attempts``circuit_break_oom_detection_enabled``preemptive_splitting_enabled | None | None |
| otel_endpoint | Send OTLP data to the HTTP server at this URL. | None | None |
| overcommit | When true, turn off memory reservations and limits for Kubernetes pods. Useful when running on a single node Kubernetes cluster with constrained resources. | False | None |
| owner | DEPRECATED: this was previously used internally by Hydrolix. | None | None |
| pg_ssl_mode | Determines whether and with what priority an SSL connection will be negotiated when connecting to a PostgreSQL server. See https://bit.ly/3U9ao8O. | disable | ["disable", "require", "verify-ca", "verify-full"] |
| pools | A list of dictionaries describing pools to deploy as part of the Hydrolix cluster. | None | None |
| registry | A docker registry to pull Hydrolix containers from. | PUBLIC_REGISTRY | None |
| sample_data_url | The storage bucket URLto use to load sample data. | None | |
| sql_transform_max_ast_elements | The number of AST elements an SQL transform can contain. This limits the maximum complexity of a SQL transform. | None | [100000, 150000] |
| sql_transform_max_expanded_ast_elements | The number of expanded AST elements an SQL transform can contain. This limits the maximum complexity of a SQL transform. | None | [100000, 150000] |
| scale | A list of dictionaries describing overrides for scale related configuration for Hydrolix services. | None | None |
| scale_off | When true, override all deployment and StatefulSet replica counts with a value of 0 and disable vector. | False | None |
| scale_profile | Selects from a set of predefined defaults for scale | eval | None |
| sdk_timeout_sec | How many seconds the Merge SDK should be given to run before it is killed. | 300 | None |
| silence_linode_alerts | If true will run a DaemonSet that turns off Linode alerts for LKE nodes. | False | None |
| str_dict_enabled | Enable/disable multi-threaded string dictionary decoding. | True | None |
| str_dict_nr_threads | Sets the maximum number of concurrent vCPU used for decoding. | 8 | None |
| str_dict_min_dict_size | Controls the number of entries in each string dictionary block. | 32768 | None |
| stream_concurrency_limit | The number of concurrent stream requests per cpu allocated across all pods beyond which Traefik will return 429 busy error responses. If not set or set to null no limit is enforced. | None | None |
| stream_partition_count | The number of partitions to use on the default Redpanda topic for stream service. | 50 | None |
| stream_load_balancer_algorithm | The load balancer algorithm to use with stream-head and intake-head services. | round-robin | ["least-connections-p2c", "round-robin"] |
| stream_partition_block | The number of partitions to use on a non-default Redpanda stream topic per TB/day of usage. | 6 | None |
| stream_replication_factor | The replication factor for the internal Redpanda topic used by the stream service it must always be less than the number of Redpanda replicas. If it is not, the configuration will not change. | 3 | None |
| targeting | A dictionary to pass targeting related Kubernetes settings to resources according to what Hydrolix service they are part of. | {} | None |
| turbine_api_init_pools | If enabled, the turbine-api component initializes some pools. | False | None |
| turbine_api_require_table_default_storage | If enabled, turbine-api will require tables to have their storage_map be populated with a default_storage_id. Useful when use of the cluster's default bucket should be discouraged. | False | None |
| traefik_external_ips | Traffic that ingresses into the cluster with one of these IPs gets directed to the Traefik service. Useful in particular when deploying all on one node. | None | [["192.168.1.5", "192.16.1.4"], ["172.16.0.8"]] |
| traefik_keep_alive_max_time | The number of seconds a client HTTP connection can be reused before receiving a Connection: close response from the server. Zero means no limit. | 26 | None |
| traefik_service_type | The type of service to use for Traefik, the entry point to the cluster. | public_lb | ["public_lb", "private_lb", "node_port", "cluster_ip"] |
| use_https_with_s3 | DEPRECATED: Use db_bucket_url or db_bucket_http_enabled. | None | None |
| use_tls | DEPRECATED: inferred from hydrolix_url. | False | None |
| use_crunchydata_postgres | Use a PostgreSQL server managed by Crunchydata's operator instead of the default dev mode postgres. | False | None |
| postgrescluster_backup_schedules | Backup schedules for PostgreSQL server. Keys are the type of backup to perform, either full, differential or incremental. Values are cron expressions representing the schedule for that backup type. Only considered when use_crunchydata_PostgreSQL server is true. | {"full": "0 16 * * 0", "differential": "0 16 * * 1-6"} | None |
| postgrescluster_backup_retention | Depends on postgrescluster_backup_retention_type. Either the count, or the number of days to retain full PostgreSQL server backups before expiring them. All differential and incremental backups that depend on full backups are also expired. Only considered when use_crunchydata_postgres is true | 1 | None |
| postgrescluster_backup_retention_type | Allowed values are count (default) and time. If set to count, then postgrescluster_backup_retention represents the number of full backups to retain before expiring them. If set to time, then postgrescluster_backup_retention represents a number of days for retention. Only considered when use_crunchydata_postgres is true. | count | ["count", "time"] |
| postgrescluster_metrics_enabled | If true, collect Prometheus metrics from the PostgreSQL server cluster. Changing this value will trigger a rolling restart of the PostgreSQL server pods. Only considered when use_crunchydata_postgres is true. | False | [] |
| postgrescluster_settings | Settings to apply to PostgreSQL server. Changes to this value will be automatically reloaded without validation. Changes to certain parameters will cause the PostgreSQL server to restart. | {} | [] |
| prometheus_label_value_length_limit | If a label value is larger than the value configured, Prometheus will discard the entire scrape. | 512 | [] |
| prometheus_remote_write_url | A URL you wish to use to configure Prometheus's remote-write functionality. | None | [] |
| prometheus_remote_write_username | The username for Prometheus to use with basic auth to connect to a remote-write endpoint. Ignored if prometheus_remote_write_url is not set. | hdx | [] |
| prometheus_scrape_interval | How frequently to scrape targets by default. | 15s | [] |
| prometheus_curated_configmap | Custom curated Prometheus ConfigMap that will be mounted onto the Prometheus pod. | [] | |
| vector_bucket | Bucket where Vector should save JSON format pod logs. | None | None |
| vector_bucket_path | Prefix under which vector will save pod logs. | logs | None |
| decay_enabled | Whether or not the Decay CronJob should run. | True | |
| decay_schedule | CRON schedule for Decay CronJob | 0 0 * * * | |
| decay_batch_size | Number of entries to fetch for each request to the catalog. | 5000 | |
| decay_max_deactivate_iterations | Maximum number of deactivation iterations to execute per table. | None | |
| decay_reap_batch_size | Number of entries to fetch for each request when locating entries for reaping | 5000 | |
| decay_max_reap_iterations | Maximum number of reap iterations to execute per table. | None | |
| job_purge_enabled | Whether or not the Job Purge CronJob should run. | True | |
| job_purge_schedule | CRON schedule for Job Purge CronJob | 0 2 * * * | |
| job_purge_age | How old a terminal job must be before it's deleted expressed as a duration string | 2160h | |
| partition_cleaner_dry_run | If true, Partition Cleaner will only log it's intentions and take no action | True | |
| partition_cleaner_grace_period | Minimum age of a partition before it is considered for deactivation or deletion expressed as a duration string. | 24h | |
| prune_locks_enabled | Whether or not the Prune Locks CronJob should run. | True | |
| prometheus_retention_ratio | The amount of the volume to reserve for Prometheus data. Example: 0.7 | 0.7 | |
| prometheus_retention_time | When to remove old Prometheus data. Example: 15d | None | |
| prometheus_retention_size | The maximum number of bytes of Prometheus data to retain. Overrides prometheus_retention_ratio. Units supported: B, KB, MB, GB, TB, PB, EB | None | |
| prune_locks_schedule | CRON schedule for Prune Locks CronJob | 30 0 * * * | |
| prune_locks_grace_period | Minimum age of a lock before it is considered for removal expressed as a duration string. | 24h | |
| limit_cpu | If set, container cpu limits are set to match cpu requests in Kubernetes. | True | |
| log_level | A dictionary to specify logging verbosity. Keys are service names with the special value of * controlling the default. | {} | |
| merge_cleanup_enabled | Whether or not the Merge Clean-up CronJob should run. | True | |
| merge_cleanup_schedule | CRON schedule for Merge Clean-up CronJob | /5 * * * | |
| merge_cleanup_delay | How long before a merged partition should be deleted expressed as a duration string. | 15m | |
| merge_cleanup_batch_size | Number of entries to fetch for each request to the catalog. | 5000 | |
| monitor_ingest | If enabled, deploy a service to ingest a timestamp into the hydro.monitor table every second. | False | |
| monitor_ingest_timeout | Deprecated. Use monitor_ingest_request_timeout. | None | |
| monitor_ingest_request_timeout | The number in seconds for HTTP timeout in HTTP POST from monitor_ingest. | 1 | |
| monitor_ingest_retry_timeout | The deadline for one submission by monitor ingest including all retries. | 1 | |
| query_peer_liveness_check_path | The HTTP path used to configure a Kubernetes liveness check for query-peers.Set to none to disable. | ?query=select%20count%28id%29%20from%20hdx.liveliness%20SETTINGS%20hdx_log_query=false%2Chdx_query_timerange_required=0 | |
| query_peer_liveness_failure_threshold | How many times query liveness check can fail. | 5 | |
| query_peer_liveness_period_seconds | How often should query liveness check run, in seconds. | 60 | |
| query_peer_liveness_probe_timeout | Number of seconds after which the liveness probe times out | 10 | |
| query_peer_liveness_initial_delay | Time in seconds to wait before starting query liveness checks. | 300 | |
| query_readiness_initial_delay | Time in seconds to wait before starting query readiness checks. | 0 | |
| refresh_job_statuses_enabled | Whether or not the Refresh Job Statuses CronJob should run. | True | |
| refresh_job_statuses_schedule | CRON schedule for Refresh Job Statuses CronJob | *** | |
| siem_backoff_duration | Backoff duration when SIEM limit not hit, for politeness. | 1s | None |
| skip_init_turbine_api | Skips running database migrations in the init-turbine-api job. Set to true when running multiple clusters with a shared database | False | |
| stale_job_monitor_enabled | Whether or not the Stale Job Monitor CronJob should run. | True | |
| stale_job_monitor_schedule | CRON schedule for Stale Job Monitor | /5 * * * | |
| stale_job_monitor_batch_size | How many jobs to probe in a single request. | 300 | None |
| stale_job_monitor_limit | How many jobs in total StaleJob will process per cycle. | 3000 | None |
| task_monitor_enabled | Whether or not the Task Monitor CronJob should run. | True | |
| task_monitor_schedule | CRON schedule for Task Monitor. | /2 * * * | |
| task_monitor_start_timeout | How old a ready task should be (in seconds) before it is considered lost and timed out. | 21600 | None |
| task_monitor_heartbeat_timeout | How old a tasks heartbeat should be (in seconds) before it is timed out. | 600 | None |
| unified_auth | Use the same auth used with the API for all services. | True | None |
| usagemeter_preserve | Duration to hang onto old, already-reported usage meter data on local clusters. | 1440h | None |
| usagemeter_reporting_url | URL to send usage data to. | https://prometheus-us.trafficpeak.live/ingest | None |
| usagemeter_reporting_table | Hydrolix table to send usage to, in project.table format. | metering_project.metering_table | None |
| usagemeter_reporting_transform | Hydrolix transform name or UUID for usage reporting. | metering_transform | None |
| usagemeter_query_timeout | Maximum time to wait for query against catalog to complete. | 4m | None |
| usagemeter_request_timeout | Maximum time to wait for reporting HTTP request to complete. | 1m | None |
| usagemeter_schedule | CRON schedule for usage meter cron job. Defaults to every 10 minutes. | /10 * * * | |
| usagemeter_enabled | Whether or not the usage meter cron job should run. | True | |
| hdx_query_max_memory_usage_perc | Maximum amount of memory to use for running a query on a single server as a percentage of the total available memory. | 80 | None |
| max_concurrent_queries | Max limit on total number of concurrently executed queries. Zero means unlimited. | 0 | None |
| max_server_memory_usage_perc | Max % of total system memory that server can use and allocate for its operation. | 0 | None |
| hdx_node_enabled | Whether or not enable hdx-node DaemonSet. | False | |
| hdx_node_config | HDX-node YAML configuration. | {} | |
| spill_locks_cleanup_enabled | Whether or not the Spill Locks Clean-up CronJob should run. | True | |
| spill_locks_cleanup_schedule | CRON schedule for Spill Locks Clean-up CronJob. | 0 12 * * * | |
| quesma_config | Quesma config for Hydrolix data source parameters. | {"project": "hydro", "table": "logs"} | |
| data_visualization_tools | List of data visualization tools to deploy. For example, Grafana and Kibana. | [] | |
| rollout_strategy_max_surge | Configures the number of pods (represented as percentage) that can be created above the desired amount of pods during deployment rollout update. | 25 | None |
| rollout_strategy_max_unavailable | Ensures the number of pods (represented as integer) that can be unavailable during deployment rollout update. | 0 | None |
| grafana_image | Definition of Grafana image:tag to be used. | grafana/grafana-enterprise:11.5.0 | |
| grafana_config | Grafana configuration. NOTE: Ensure grafana is included in data_visualization_tools Tunable to enable Grafana deployment. admin_user: Grafana admin username. admin_email: Grafana admin user email. allow_embedding: Prevents embedding Grafana in frames to mitigate Clickjacking risks. db_user: Grafana database username. alert_eval_timeout: Timeout for alert evaluation when fetching data from a source. smtp_enabled: Enables email server settings. Requires GRAFANA_SMTP_PASSWORD in the curated secret. smtp_host: Email server host. smtp_user: Email server authentication username. rendering_timeout: Timeout for rendering reports (PDFs, embedded images, CSV attachments). is_enterprise: Enables Grafana Enterprise. Requires GRAFANA_LICENSE in the curated secret. google_auth_enabled: Enables Google OAuth authentication. Requires GOOGLE_CLIENT_SECRET in the curated secret. google_client_id: Client ID of the Google Auth App. inactive_timeout: Maximum inactive duration before a user must log in again. allow_sign_up: Controls Grafana user creation through OAuth. If false, only existing users can log in. | {"admin_user": "admin", "admin_email": "admin@localhost", "allow_embedding": false, "db_user": "grafana", "alert_eval_timeout": "30s", "smtp_enabled": false, "smtp_host": "smtp.sendgrid.net:587", "smtp_user": "apikey", "rendering_timeout": "120s", "is_enterprise": false, "google_auth_enabled": false, "google_client_id": null, "inactive_timeout": "7d", "allow_sign_up": false} |
Updated about 12 hours ago