Skip to content

Tunables List

A listing of HTN tunables used by Hydrolix. These tunables are set in the hydrolixcluster.yaml configuration file, under spec:.

Tunables⚓︎

acme_enabled⚓︎

Type: boolean | Added v4.20

Automatically generate and renew SSL certs for your Hydrolix domain. Overrides any existing Kubernetes secret named traefik-tls.

Default: False


admin_email⚓︎

Type: string | Added v4.20

The email address of the Hydrolix cluster administrator.


alt_names⚓︎

Type: array | Added v5.9.3

Provide a list of alternate-domain names to add to the SSL cert. Used when acme_enabled=True

Example
1
2
3
spec:
  alt_names:
  - customer-grafana.hydrolix.live

argus_fleet_table⚓︎

Type: string | Added v5.4

Hydrolix table to send fleet info to, in project.table format.

Default: fleet_inventory.fleet_table


argus_fleet_transform⚓︎

Type: string | Added v5.4

Hydrolix transform name or UUID for fleet reporting.

Default: fleet_transform


argus_fleet_url⚓︎

Type: string | Added v5.4

URL to send fleet data to.

Default: https://argus.hydrolix.live/ingest


ariadne_core⚓︎

Type: dict | Added v5.7.4

Core configuration parameters. The secret key should refer name of kubernetes secret that contains the Ariadne Core's config. The secret must contain 'config.yaml' key and should be created by the user. A version key can be passed to use a specific tag of the Ariadne Core image.

Default value
1
2
3
4
5
spec:
  ariadne_core:
    enabled: false
    secret: ariadne-core
    version: v0.1.1
Example
1
2
3
4
5
spec:
  ariadne_core:
    enabled: true
    secret: ariadne-core
    version: v0.1.1

audit_logs_max_age⚓︎

Type: string | Added v5.6.2

How long to retain an audit_log record, expressed as a duration string.


audit_logs_migration_job_enabled⚓︎

Type: boolean | Added v5.6.2

Enable audit_log records to be migrated to a Hdx table.

Default: True


audit_logs_migration_job_schedule⚓︎

Type: string | Added v5.6.2

CRON schedule to migrate audit_log recoreds to Hdx.

Default: */5 * * * *


audit_logs_purge_age⚓︎

Type: string | Added v5.6.2

When to delete an expired audit_log record, expressed as a duration string.

Default: 5d


auth_http_read_timeout_ms⚓︎

Type: integer | Added v4.20

Maximum time to wait for a socket read for user-permission data from auth endpoint (turbine-api)

Default: 2000


auth_http_response_timeout_ms⚓︎

Type: integer | Added v4.20

Maximum time to wait for receiving HTTP headers from auth endpoint (turbine-api) in response to user permission requests

Default: 2000


autoingest_unique_file_paths⚓︎

Type: boolean | Added v4.22

Enable unique file paths from object store by ignoring duplicate paths.

Default: False


aws_credentials_method⚓︎

Type: string | Added v4.20

DEPRECATED: Use db_bucket_credentials_method.

Examples
spec:
  aws_credentials_method: static
spec:
  aws_credentials_method: instance_profile

aws_load_balancer_subnets⚓︎

Type: string | Added v4.20

Subnets to assign to the load balancer of the Traefik service when running in EKS.

Example
spec:
  aws_load_balancer_subnets: subnet-xxxx,mySubnet

aws_load_balancer_tags⚓︎

Type: string | Added v4.20

Additional tags to be added to the load balancer of the Traefik service when running in EKS.

Example
spec:
  aws_load_balancer_tags: Environment=dev,Team=test

azure_blob_storage_account⚓︎

Type: string | Added v4.20

The storage account to access an Azure blob storage container.


basic_auth⚓︎

Type: array | Added v4.20

A list of Hydrolix services that should be protected with basic auth when accessed over HTTP.


batch_controller_enabled⚓︎

Type: boolean | Added v5.9.3

If true, batch controller will be enabled.

Default: False


batch_peer_heartbeat_period⚓︎

Type: string | Added v4.20

How frequently a batch peer should heartbeat any task it's working on as a duration string.

Default: 5m


bucket⚓︎

Type: string | Added v4.20

DEPRECATED: Use db_bucket_url.


catalog_db_admin_db⚓︎

Type: string | Added v4.20

The default database of the admin user on the Postgre SQL server where Hydrolix metadata is stored.

Default: turbine


catalog_db_admin_user⚓︎

Type: string | Added v4.20

The admin user of the Postgre SQL server where Hydrolix metadata is stored.

Default: turbine


catalog_db_host⚓︎

Type: string | Added v4.20

The Postgre SQL server where Hydrolix metadata is stored.

Default: postgres


catalog_db_port⚓︎

Type: integer | Added v4.21

The Postgre SQL server port where Hydrolix metadata is stored.

Default: 5432


catalog_intake_connections⚓︎

Type: dict | Added v4.20

Connection pool settings for intake services that connect to the Postgre SQL server where Hydrolix metadata is stored.

Available options:

  1. max_lifetime - The max duration that a connection can live before being recycled.

  2. max_idle_time - The max duration that a connection can be idle before being closed.

  3. max - The max number of connections that can be opened by each intake service that connects to the Postgre SQL server.

  4. min - The minimum number of connections to keep open to the Postgre SQL server.

  5. check_writable - If set to true, when a connection is opened to the Postgre SQL server, ensure the server can handle writes.

Default value
1
2
3
4
5
6
7
spec:
  catalog_intake_connections:
    check_writable: null
    max: null
    max_idle_time: 1m
    max_lifetime: 10m
    min: null

catalog_proxy_enabled⚓︎

Type: boolean | Added v5.10

Use the catalog proxy service for database interactions.

Default: False


clickhouse_http_port⚓︎

Type: integer | Added v4.20

The dedicated port for the Click House HTTP interface.

Default: 8088


client_id⚓︎

Type: string | Added v4.20

DEPRECATED: Use hydrolix_name and db_bucket_url.


containers⚓︎

Type: object | Added v5.7.4

This tunable allows specifying custom registry paths and version overrides for images. Either image or tag can be specified. If image is missing registry path will default to hydrolix registry.

Example
spec:
  containers:
    elasticsearch:
      image: docker.io/elasticsearch
    merge:
      image: docker.io/merge
      tag: v10.2.3
    merge-controller:
      tag: v10.2.3
    turbine:
      image: turbine
      tag: v10.2.3

data_service_termination_grace_period⚓︎

Type: integer | Added v4.20

Termination grace period for most data services.

Default: 120


data_visualization_tools⚓︎

Type: array | Added v4.23

List of data visualization tools to deploy. Supported options: Grafana, Kibana and Superset.


db_bucket_credentials_method⚓︎

Type: string | Added v4.20

The method Hydrolix uses to acquire credentials for connecting to cloud storage.

Default: web_identity

Examples
spec:
  db_bucket_credentials_method: static
spec:
  db_bucket_credentials_method: ec2_profile
spec:
  db_bucket_credentials_method: web_identity

db_bucket_endpoint⚓︎

Type: string | Added v4.20

The endpoint URL for S3 compatible object storage services. Not required if using AWS S3 or if db_bucket_url is provided.


db_bucket_name⚓︎

Type: string | Added v4.20

The name of the bucket for Hydrolix to store data in. Not required if db_bucket_url is provided.


db_bucket_region⚓︎

Type: string | Added v4.20

Not required if it can be inferred from db_bucket_url.

Examples
spec:
  db_bucket_region: us-east-2
spec:
  db_bucket_region: us-central1

db_bucket_type⚓︎

Type: string | Added v4.20

The object storage type of the bucket you would like Hydrolix to store data in. Not required if db_bucket_url is provided.

Examples
spec:
  db_bucket_type: gs
spec:
  db_bucket_type: s3

db_bucket_url⚓︎

Type: string | Added v4.20

The URL of the cloud storage bucket you would like Hydrolix to store data in.

Examples
spec:
  db_bucket_url: gs://my-bucket
spec:
  db_bucket_url: s3://my-bucket
spec:
  db_bucket_url: https://my-bucket.s3.us-east-2.amazonaws.com

db_bucket_use_https⚓︎

Type: boolean | Added v4.20

If true use HTTPS when connecting to the cloud storage service. Inferred from db_bucket_url if possible.

Default: True


decay_batch_size⚓︎

Type: integer | Added v4.20

Number of entries to fetch for each request to the catalog.

Default: 5000


decay_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the Decay Cron Job should run.

Default: True


decay_max_deactivate_iterations⚓︎

Type: integer | Added v4.20

Maximum number of deactivation iterations to execute per table.


decay_max_reap_iterations⚓︎

Type: integer | Added v4.20

Maximum number of reap iterations to execute per table.


decay_reap_batch_size⚓︎

Type: integer | Added v4.20

Number of entries to fetch for each request when locating entries for reaping

Default: 5000


decay_schedule⚓︎

Type: string | Added v4.20

CRON schedule for Decay Cron Job

Default: 0 0 * * *


default_query_pool⚓︎

Type: string | Added v4.20

Name of the default query pool.

Default: query-peer


disable_disk_cache⚓︎

Type: boolean | Added v4.20

If true, query peers will immediately delete partition metadata from disk after use.

Default: False


disable_traefik_clickhouse_http_port⚓︎

Type: boolean | Added v4.20

If true the load balancer will not forward to Traefik on port 8088. This port provides a Click House compatible query interface at the root of the service rather than at a subpath.

Default: False


disable_traefik_http_port⚓︎

Type: boolean | Added v4.20

If true the load balancer will not forward to Traefik on port 80. When TLS is enabled, this port is only used to redirect to HTTPS. Otherwise this is the main way to access all services.

Default: False


disable_traefik_https_port⚓︎

Type: boolean | Added v4.20

If true, the load balancer will not forward to Traefik on port 443. Only relevant if TLS is enabled

Default: False


disable_traefik_mysql_port⚓︎

Type: boolean | Added v5.0

If true the load balancer will not forward to Traefik on the Click House My SQL interface port. This is port 9004.

Default: False


disable_traefik_native_port⚓︎

Type: boolean | Added v4.20

If true the load balancer will not forward to Traefik on the Click House native protocol port. This is port 9440 when TLS is enabled or 9000 if not.

Default: False


disable_vector_bucket_logging⚓︎

Type: boolean | Added v4.20

Prevent vector from sending logs to the bucket.

Default: False


disable_vector_kafka_logging⚓︎

Type: boolean | Added v4.20

Prevent vector from emitting logs to Redpanda.

Default: False


disk_cache_cull_start_perc⚓︎

Type: integer | Added v4.20

Percentage of cache disk space used before starting to remove files.

Default: 75


disk_cache_cull_stop_perc⚓︎

Type: integer | Added v4.20

Percentage of cache disk space used before stopping removing files.

Default: 65


disk_cache_entry_max_ttl_minutes⚓︎

Type: integer | Added v4.20

Max TTL for a cache disk entry. It is the longest period of time for which the LRU disk cache can save an entry before it expires.

Default: 360


disk_cache_redzone_start_perc⚓︎

Type: integer | Added v4.20

Minimum percentage of cache disk space used to be considered as redzone.

Default: 90


dns_aws_max_resolution_attempts⚓︎

Type: integer | Added v4.20

Maximum number of attempts made by the DNS Resolver for AWS and all s3 compatible storages in a given DNS refresh cycle.

Default: 1


dns_aws_max_ttl_secs⚓︎

Type: integer | Added v4.20

Max DNS TTL for AWS and S3-compatible storages. It is the longest period of time for which the DNS resolver can cache a DNS record before it expires and needs to be refreshed. max_ttl=0 means DNS cache strictly respects the TTL from the DNS query response.

Default: 0


dns_azure_max_resolution_attempts⚓︎

Type: integer | Added v4.20

Maximum number of attempts made by the DNS Resolver for Azure storage in a given DNS refresh cycle.

Default: 1


dns_azure_max_ttl_secs⚓︎

Type: integer | Added v4.20

Max DNS TTL for Azure storage. It is the longest period of time for which the DNS resolver can cache a DNS record before it expires and needs to be refreshed. max_ttl=0 means DNS cache strictly respects the TTL from the DNS query response.

Default: 0


dns_gcs_max_resolution_attempts⚓︎

Type: integer | Added v4.20

Maximum number of attempts made by the DNS Resolver for GCS storage in a given DNS refresh cycle.

Default: 1


dns_gcs_max_ttl_secs⚓︎

Type: integer | Added v4.20

Max DNS TTL for GCS storage. It is the longest period of time for which the DNS resolver can cache a DNS record before it expires and needs to be refreshed. max_ttl=0 means DNS cache strictly respects the TTL from the DNS query response.

Default: 0


dns_server_ip⚓︎

Type: string | Added v4.20

The IP address of the DNS server used for performance-critical purposes.


domain⚓︎

Type: string | Added v4.20

DEPRECATED: Use hydrolix_url.


eks_product_code⚓︎

Type: string | Added v4.20

EKS product code for use with Amazon Marketplace.

Default: 6ae46hfauzadikp9f8npdbh9v


enable_password_complexity_policy⚓︎

Type: boolean | Added v4.20

If set to true, uses the default password policy: Minimum length: 8 characters, Uppercase characters: 1, Lowercase characters: 1, Digits: 1, Special characters: 1, Not recently used: Past 24 passwords, Expire password: 90 days, Not username, Not email.

Default: False


enable_query_auth⚓︎

Type: boolean | Added v4.20

When enabled requests to the query service, URLpaths starting with /query require authentication.

Default: True


enable_traefik_access_logging⚓︎

Type: boolean | Added v4.20

If set to true, Traefik will log all access requests.WARNING: This will produce a very high and potentially unmanageable amount of logs

Default: False


enable_traefik_authorization⚓︎

Type: boolean | Added v5.9.3

Setting this parameter to true will perform authorization validation for routes enabled with unified_auth.

Default: False


enable_traefik_hsts⚓︎

Type: boolean | Added v4.20

If set to true, Traefik will enforce HSTS on all its connections.WARNING: This may lead to hard-to-diagnose persistent SSL failures if there are any errors in SSL configuration, and cannot be turned off later.

Default: False


enable_vector⚓︎

Type: boolean | Added v4.20

Run vector to send Kubernetes pod logs to JSON files in a bucket and to the internal logs topic. Default inferred from the value of scale_off.


env⚓︎

Type: object | Added v4.20

Environment variables to set on all Kubernetes pods that are part of the Hydrolix cluster.


exp_backoff_additive_jitter⚓︎

Type: boolean | Added v4.20

True: (growth_factor)(1 + jitter). False: growth_factor(jitter).

Default: True


exp_backoff_growth_factor_ms⚓︎

Type: integer | Added v4.20

Every sleep will use this as multiplicative factor. For example, 2^i *(growth_factor)ms.

Default: 50


extra_loadbalancers⚓︎

Type: integer | Added v5.7.4

Additional load balancers to be provisioned in addition to the default load balancer

Default: 0


force_container_user_root⚓︎

Type: boolean | Added v4.20

Set the initial user for all containers to 0 (root).

Default: False


grafana_config⚓︎

Type: dict | Added v5.1

Grafana configuration.

NOTE: To enable Grafana deployment, include grafana in the data_visualization_tools tunable

  • admin_user: Grafana admin username.

  • admin_email: Grafana admin user email.

  • allow_embedding: Prevents embedding Grafana in frames to mitigate clickjacking risks.

  • database: Optional Specify an existing external database to use for grafana.

  • type: Database type - either "postgres" or "mysql" (default: "postgres")

  • host: Database host in format "hostname: port" - If specified, a grafana DB will not be created in the catalog database as part of the init-cluster job.

  • name: Database name (default: "grafana")

  • ssl_mode: SSL mode for the connection -- either "disable", "require" (default: "disable")

  • db_user: Grafana database username. If connecting to an existing DB, the password can be set by defining GRAFANA_DB_PASSWORD in the curated secret.

  • alert_eval_timeout: Timeout for alert evaluation when fetching data from a source.

  • smtp_enabled: Enables email server settings. Requires the GRAFANA_SMTP_PASSWORD secret.

  • smtp_host: Email server host.

  • smtp_user: Email server authentication username.

  • rendering_timeout: Timeout for rendering reports (PDFs, embedded images, or CSV attachments ).

  • is_enterprise: Enables Grafana Enterprise. Requires the GRAFANA_LICENSE secret.

  • google_auth_enabled: Enables Google OAuth authentication. Requires the GOOGLE_CLIENT_SECRET secret.

  • google_client_id: Client ID of the Google Auth app.

  • inactive_timeout: Maximum inactive duration before requiring login again.

  • allow_sign_up: Controls Grafana user creation through OAuth. If false, only existing users can log in.

  • settings: A dictionary of custom grafana settings that take precedence. The python toml library is used to merge this dictionary into the grafana ini file.

Default value
spec:
  grafana_config:
    admin_email: admin@localhost
    admin_user: admin
    alert_eval_timeout: 30s
    allow_embedding: false
    allow_sign_up: false
    database:
      host: null
      name: grafana
      ssl_mode: disable
      type: postgres
    db_user: grafana
    google_auth_allowed_domains: []
    google_auth_enabled: false
    google_client_id: null
    inactive_timeout: 7d
    is_enterprise: false
    plugins: {}
    rendering_timeout: 120s
    settings: {}
    smtp_enabled: false
    smtp_host: smtp.sendgrid.net:587
    smtp_user: apikey
    unsigned_plugins: []

grafana_image⚓︎

Type: string | Added v5.1

Definition of Grafana image: tag to be used.

Default: grafana/grafana-enterprise:12.3.1


hdx_anomaly_detection⚓︎

Type: dict | Added v5.3

Anomaly Detection configuration parameters.

Connection settings:

  • enabled: Set to True to enable anomaly detection (default: False).

  • version: Optional tag to pin a specific anomaly detection image version.

  • clickhouse_host: Click House native protocol host (default: query-head).

  • clickhouse_port: Click House native protocol port (default: 9000).

  • clickhouse_secure: Use secure connection for Click House (default: false).

  • clickhouse_verify_ssl: Verify SSL certificates for Click House (default: false).

  • http_host: HTTP API host (default: traefik).

  • http_port: HTTP API port (default: 443).

  • http_secure: Use secure connection for HTTP API (default: true).

  • http_verify_ssl: Verify SSL certificates for HTTP API (default: false).

  • timeout: API request timeout in seconds (default: 10).

  • batch_size: Data ingestion batch size (default: 10000).

  • retry_count: Number of retry attempts (default: 3).

Application settings:

  • check_interval: Job config check interval in seconds (default: 60).

Config polling settings:

  • config_polling_enabled: Enable config polling from cloud storage (default: true).

  • config_polling_base_key: Base object key for configs in cloud storage (default: adconfig/v1/).

  • config_polling_manifest_filename: Manifest filename (default: ad_manifest.json).

  • config_polling_interval: Config polling interval in seconds (default: 60).

  • config_polling_retry_delay: Retry delay in seconds (default: 5).

  • config_polling_max_retries: Max retry attempts (default: 3).

Storage settings:

  • storage_retry_attempts: Storage operation retry attempts (default: 3).

  • storage_retry_delay: Storage retry delay in seconds (default: 1.0).

  • storage_timeout: Storage operation timeout in seconds (default: 30).

  • storage_max_retry_delay: Max storage retry delay in seconds (default: 60.0).

  • storage_backoff_factor: Storage retry backoff factor (default: 2.0).

Default value
spec:
  hdx_anomaly_detection:
    batch_size: 10000
    check_interval: 60
    clickhouse_host: query-head
    clickhouse_port: 9000
    clickhouse_secure: false
    clickhouse_verify_ssl: false
    config_polling_base_key: adconfig/v1/
    config_polling_enabled: true
    config_polling_interval: 60
    config_polling_manifest_filename: ad_manifest.json
    config_polling_max_retries: 3
    config_polling_retry_delay: 5
    configs: null
    enabled: false
    http_host: traefik
    http_port: 443
    http_secure: true
    http_verify_ssl: false
    retry_count: 3
    storage_backoff_factor: 2.0
    storage_max_retry_delay: 60.0
    storage_retry_attempts: 3
    storage_retry_delay: 1.0
    storage_timeout: 30
    timeout: 10
    version: v1.3.0
Example
1
2
3
4
5
6
7
8
9
spec:
  hdx_anomaly_detection:
    configs:
      tenant1:
        db_bucket_name: bucket-1
      tenant2:
        clickhouse_host: custom-ch
        db_bucket_name: bucket-2
    enabled: true

hdx_anomaly_rca⚓︎

Type: dict | Added v5.8.6

Anomaly Root Cause Analysis configuration parameters.

  • enabled: Set to True to enable anomaly RCA.

  • version: Optional tag to pin a specific anomaly RCA image version.

  • port: Port for the RCA API service.

  • config: Configuration settings including Config Map reference.

Default value
1
2
3
4
5
6
7
spec:
  hdx_anomaly_rca:
    config:
      configMap: null
    enabled: false
    port: 8080
    version: v1.0.0
Example
1
2
3
spec:
  hdx_anomaly_rca:
    enabled: true

hdx_ariadne_janus⚓︎

Type: dict | Added v5.6.2

Janus configuration parameters.

A version key can be passed to use a specific tag of the Janus image. The proxy_config_file_path key specifies the path where Janus Proxy stores its configuration file on the persistent volume (defaults to /data/config.json). Janus' Guardrails Service leverage Presidio provider. Presidio Components versions can be passed to use a specific image tag (not all are available and the default values are recommended ones). Presidio components leverage thread-based scaling model. Number of worker threads can be set as 'None' to be calculated automatically (CPU * 2 + 1).

Default value
spec:
  hdx_ariadne_janus:
    enabled: false
    presidio_analyzer_workers: null
    presidio_anonymizer_workers: 13
    presidio_enabled: false
    proxy_config_file_path: /data/config.json
    version: v0.3.1
    version_presidio_analyzer: 2.2.360
    version_presidio_anonymizer: 2.2.360
    version_proxy: v0.2.0
Example
1
2
3
4
spec:
  hdx_ariadne_janus:
    enabled: false
    proxy_config_file_path: /data/config.json

hdx_node_config⚓︎

Type: dict | Added v5.1

HDX Node configuration for the hdx-node Daemon Set.

hdx-node runs on every cluster node to monitor health, silence Linode alerts, and optionally block ports. Enable with hdx_node_enabled: true in the HDX spec.

All fields are optional to preserve sparse/merge behavior - only specified fields appear in the generated Config Map. hdx-node applies its own defaults.

Example minimal config:

1
2
3
4
5
hdx_node_enabled: true
hdx_node_config:
one_off_tasks:
silence_linode_alerts:
enabled: true

Prometheus metrics are exposed at /metrics per node:

  • hdx_node_leadership_changes_total: Number of times this node became leader

  • hdx_node_known_nodes: Count of known (discovered) nodes

  • hdx_node_unreachable_nodes: Count of nodes unreachable from this node

  • hdx_node_to_be_removed_nodes: Count of nodes marked for removal by leader

  • hdx_node_status: 1 if this node is leader, 0 if follower

Default: {"nodepulse":null,"one_off_tasks":null}

Example
1
2
3
4
5
6
spec:
  hdx_node_config:
    nodepulse: null
    one_off_tasks:
      silence_linode_alerts:
        enabled: true

hdx_node_enabled⚓︎

Type: boolean | Added v5.1

Whether or not enable hdx-node Daemon Set.

Default: False


hdx_pg_monitor⚓︎

Type: dict | Added v5.6.2

HDX PG Monitor configuration parameters.

Default value
spec:
  hdx_pg_monitor:
    config_map: hdx-pg-monitor-cfg
    constant_labels: ''
    default_interval: 60s
    default_query_timeout: 5m
    enabled: false
    metric_prefix: ''
    pool_max: '10'
    pool_min: '1'

hdx_pod_metrics_enabled⚓︎

Type: boolean | Added v5.8.6

If true, adds an ultra light-weight side-car that exports pod-level metrics to deployments that are monitored by HDX-Scaler's Vertical Autoscaler. If hdx-node is enabled then this tunable has no effect as HDX-Scaler will scrape pod-level metrics from hdx-node instead of a side-car.

Default: False


hdx_query_max_memory_usage_perc⚓︎

Type: integer | Added v4.20

Maximum amount of memory to use for running a query on a single server as a percentage of the total available memory.

Default: 0


hdx_query_max_perc_before_external_group_by⚓︎

Type: integer | Added v5.2

Maximum amount of memory to use for running a summary merge query as a percentage of the total available memory. Zero deactivates the restriction.

Default: 0


hdx_traefik_auth_workers⚓︎

Type: integer | Added v5.1

Number of async workers gunicorn will create for services requests. Defaults to number of CPU for hdx-traefik-auth container unless specified in this tunable.


hdx_vpa_metrics⚓︎

Type: object | Added v5.9.3

HDX VPA Metrics configuration parameters for aggregating node stats.

Default value
1
2
3
4
5
6
spec:
  hdx_vpa_metrics:
    enabled: false
    filter_monitored_pods: true
    metrics_port: 9100
    poll_interval: 10

health_check_default_ignored_resources⚓︎

Type: array | Added v5.9.3

Default list of Kubernetes resource patterns to ignore when evaluating cluster health. These resources will be treated as non-critical and won't affect the cluster's Ready status. Patterns support regex matching (e.g., 'Job/load-sample-project.*'). To completely override this list, set 'health_check_override_default_ignored_resources' to true.

Default: ["Job/load-sample-project.*"]

Example
1
2
3
4
spec:
  health_check_default_ignored_resources:
  - Job/load-sample-project.*
  - Deployment/turbine-api

health_check_ignored_resources⚓︎

Type: array | Added v5.9.3

Additional Kubernetes resource patterns to ignore when evaluating cluster health. By default, these are combined with 'health_check_default_ignored_resources'. To use only this list, set 'health_check_override_default_ignored_resources' to true. Patterns support regex matching (e.g., 'Deployment/my-optional-service').

Example
1
2
3
4
spec:
  health_check_ignored_resources:
  - Deployment/merge-peer
  - Job/merge-cleanup-.*

health_check_override_default_ignored_resources⚓︎

Type: boolean | Added v5.9.3

If true, replaces the default ignored resources list with only the user specified ones. If false (default), user-specified ignored resources are added to the defaults.

Default: False


host⚓︎

Type: string | Added v4.20

DEPRECATED: Use hydrolix_url


http_connect_timeout_ms⚓︎

Type: integer | Added v4.20

Maximum time to wait for socket connection to cloud storage to complete

Default: 300


http_port⚓︎

Type: integer | Added v4.20

The port to serve Hydrolix plain HTTP on.


http_proxy⚓︎

Type: dict | Added v5.2 | Reference

HTTP-proxy configuration. NOTE: disabled by default.

  • enabled: Set to True to enable the HTTP proxy.

  • version: Use to run an alternative proxy version.

  • port: Configures the port for incoming connections

  • server: Defines timeouts for incoming requests.

  • users: Specifies max time for query execution.

  • heartbeat: Sets Hydrolix health check parameters.

  • cache: Configures the storage type for query results; defaults to file_system. To use redis, provide configuration (addresses/username/password/TLS/certs).

Curated secrets are required:

  • HTTP_PROXY_REDIS_USERNAME and HTTP_PROXY_REDIS_PASSWORD are mandatory.

  • HTTP_PROXY_REDIS_TLS_KEY and HTTP_PROXY_REDIS_TLS_CERT are optional. "

Default value
spec:
  http_proxy:
    allow_ping: false
    cache:
      dir: /tmp/http-proxy/cache
      expire: 1m
      max_size: 150M
      mode: file_system
    enabled: false
    heartbeat:
      interval: 5s
      request: /query?query=SELECT%201&hdx_query_output_format=TSV
      response: '1

        '
      timeout: 3s
    log_debug: false
    port: 9444
    server:
      idle_timeout: 8m
      read_timeout: 2m
      write_timeout: 4m
    users:
      max_execution_time: 2m
    version: v0.6.1
Examples
spec:
  http_proxy:
    allow_ping: false
    cache:
      dir: /tmp/http-proxy/cache
      expire: 1m
      max_size: 150M
    enabled: true
    heartbeat:
      interval: 5s
      request: /query?query=SELECT%201&hdx_query_output_format=TSV
      response: '1

        '
      timeout: 3s
    log_debug: false
    port: 9444
    server:
      idle_timeout: 8m
      read_timeout: 2m
      write_timeout: 4m
    users:
      max_execution_time: 2m
spec:
  http_proxy:
    allow_ping: false
    cache:
      addresses:
      - valkey-primary:6379
      insecure_skip_verify: true
      mode: redis
      use_tls: true
    enabled: true
    heartbeat:
      interval: 5s
      request: /query?query=SELECT%201&hdx_query_output_format=TSV
      response: '1

        '
      timeout: 3s
    log_debug: false
    port: 9444
    server:
      idle_timeout: 8m
      read_timeout: 2m
      write_timeout: 4m
    users:
      max_execution_time: 2m

http_read_timeout_ms⚓︎

Type: integer | Added v4.20

Maximum time to wait between a socket read and cloud storage having data ready to be read.

Default: 1000


http_response_timeout_ms⚓︎

Type: integer | Added v4.20

Maximum time to wait for receiving HTTP headers to complete while reading from cloud storage.

Default: 1000


http_ssl_connect_timeout_ms⚓︎

Type: integer | Added v4.20

Maximum time to wait for SSL handshake during connection to cloud storage

Default: 1000


http_write_timeout_ms⚓︎

Type: integer | Added v4.20

Maximum time to wait before uploading partition to cloud is complete

Default: 10000


https_port⚓︎

Type: integer | Added v4.20

The port to serve Hydrolix HTTPS on.


hydrolix_name⚓︎

Type: string | Added v4.20

The name you would like to assign your Hydrolix cluster. Will be the same as the namespace name if not specified.


hydrolix_url⚓︎

Type: string | Added v4.20

The URL you would like to use to access your Hydrolix cluster.

Examples
spec:
  hydrolix_url: https://my-host.hydrolix.live
spec:
  hydrolix_url: https://my-host.mydomain.com
spec:
  hydrolix_url: http://my-host.local

image_pull_secret_names⚓︎

Type: array | Added v5.10

List of kubernetes secret name that contains the repository credentials


initial_exp_backoff_ms⚓︎

Type: integer | Added v4.20

Sleep time starts from this value and exponentially grows with retry count.

Default: 0


intake_head_accept_data_timeout⚓︎

Type: string | Added v4.20

Configures the maximum duration that intake-head will wait for a request to be accepted into the partition creation pipeline. If the timeout is reached, the request will be rejected with a 429 status code response. If not configured or set to 0, intake-head pods will not timeout.

Default: 0s


intake_head_catalog_spill_config⚓︎

Type: dict | Added v4.20

Configuration for intake spill functionality to object storage.

Default value
1
2
3
4
5
spec:
  intake_head_catalog_spill_config:
    enabled: false
    max_attempts_spill: '5'
    max_concurrent_spill: '20'
Example
1
2
3
4
5
spec:
  intake_head_catalog_spill_config:
    enabled: true
    max_attempts_spill: '5'
    max_concurrent_spill: '20'

intake_head_http_read_header_timeout⚓︎

Type: string | Added v5.7.4

Maximum duration intake-head will devote to reading completely all headers of an HTTP request.

Default: 10s


intake_head_http_read_timeout⚓︎

Type: string | Added v5.7.4

Maximum duration intake-head will devote to reading a complete HTTP request, including both headers and body.

Default: 3m


intake_head_max_outstanding_requests⚓︎

Type: integer | Added v4.20

Configures the maximum number of requests that an intake-head pod will allow to be outstanding and in process before rejecting new requests with a 429 status code response. If not configured or set to 0, intake-head pods will never reject new requests.

Default: 0


intake_head_raw_data_spill_config⚓︎

Type: dict | Added v4.20

Configuration for intake spill functionality to object storage.

Default value
1
2
3
4
5
spec:
  intake_head_raw_data_spill_config:
    enabled: false
    max_attempts_spill: '5'
    max_concurrent_spill: '20'
Example
1
2
3
4
5
spec:
  intake_head_raw_data_spill_config:
    enabled: true
    max_attempts_spill: '5'
    max_concurrent_spill: '20'

io_perf_mappings⚓︎

Type: string | Added v4.20

Internally used presets for io_perf_mode. Parsed as JSON Array(Array(Int)).

Default value
spec:
  io_perf_mappings: '[[2097152, 256, 256], [6291456, 128, 128], [12582912, 64, 64]]'

ip_allowlist⚓︎

Type: array | Added v4.20

A list of CIDR ranges that should be allowed to connect to the Hydrolix cluster load balancer.

Default: ["127.0.0.1/32"]


issue_wildcard_cert⚓︎

Type: boolean | Added v5.3

Whether to issue wildcard TLS certificate. NOTE: DNS Challenge will be used. Route53 credentals need to be provided in ROUTE53_AWS_ACCESS_KEY_ID and ROUTE53_AWS_SECRET_ACCESS_KEY via curated secret.

Default: False


job_purge_age⚓︎

Type: string | Added v4.20

How old a terminal job must be before it's deleted expressed as a duration string

Default: 48h


job_purge_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the Job Purge Cron Job should run.

Default: True


job_purge_schedule⚓︎

Type: string | Added v4.20

CRON schedule for Job Purge Cron Job

Default: 0 2 * * *


kafka_careful_mode⚓︎

Type: boolean | Added v4.20

Default: False


kafka_tls_ca⚓︎

Type: string | Added v4.20

A CA certificate used by the kafka_peer to authenticate Kafka servers it connects to.


kafka_tls_cert⚓︎

Type: string | Added v4.20

The PEM format certificate the kafka_peer will use to authenticate itself to a Kafka server.


kafka_tls_secret_name⚓︎

Type: string | Added v5.8.6

Name of Kubernetes secret that contains the following keys:

  • KAFKA_TLS_CERT - PEM format certificate

  • KAFKA_TLS_CA - CA certificate

  • KAFKA_TLS_KEY - PEM format key

kafka_peer will use the keys from above to authenticate itself to a Kafka server.


kibana_gateway_config⚓︎

Type: dict | Added v5.9.3

Kibana Gateway config for Hydrolix data source parameters.

Default value
spec:
  kibana_gateway_config:
    additional_tables: []
    elasticsearch_url: http://elasticsearch-es-http:9200
    elasticsearch_username: hydrolix
    enable_public_access: false
    indexes: {}
    project: hydro
    projects: []
    table: logs
    version: v1.1.23

kibana_security_enabled⚓︎

Type: boolean | Added v5.4

Enable Kibana auth and RBAC via Elasticsearch static credentials. User credentials can be found in hdx-elastic-user Kubernetes secret.

Default: False


kinesis_coordinate_period⚓︎

Type: string | Added v4.20

For Kinesis sources, how often the coordination process runs which checks for the available shards and peers and distributes consuming amongst available peers.

Default: 10s


kinesis_coordinate_strategy⚓︎

Type: string | Added v4.20

The strategy to use for coordinating Kinesis peers for a Kinesis source. Possible values are EXTERNAL_COORDINATOR or ZOOKEEPER.

Default: EXTERNAL_COORDINATOR


kubernetes_cloud⚓︎

Type: string | Added v4.20

DEPRECATED: Use kubernetes_profile.

Examples
spec:
  kubernetes_cloud: aws
spec:
  kubernetes_cloud: gcp

kubernetes_premium_storage_class⚓︎

Type: string | Added v4.20

The storage class to use with persistent volumes created in Kubernetes for parts of a Hydrolix cluster where throughput is most critical.

Examples
spec:
  kubernetes_premium_storage_class: gke
spec:
  kubernetes_premium_storage_class: eks
spec:
  kubernetes_premium_storage_class: lke
spec:
  kubernetes_premium_storage_class: aks

kubernetes_profile⚓︎

Type: string | Added v4.20

Use default settings appropriate to this type of Kubernetes deployment.

Default: generic

Examples
spec:
  kubernetes_profile: gke
spec:
  kubernetes_profile: eks
spec:
  kubernetes_profile: lke
spec:
  kubernetes_profile: aks

kubernetes_storage_class⚓︎

Type: string | Added v4.20

The storage class to use with persistent volumes created in Kubernetes as part of a Hydrolix cluster.


limit_cpu⚓︎

Type: boolean | Added v4.20

When set to false, removes all CPU container limits. By default, containers are set with the same request and limit value. Note that removing either a memory or CPU limit or request from any container on a pod removes the Guaranteed quality of service class from that pod.

Default: True


log_level⚓︎

Type: object | Added v4.20

A dictionary to specify logging verbosity. Keys are service names with the special value of * controlling the default.


logs_http_remote_table⚓︎

Type: string | Added v4.20

An existing Hydrolix where the data should land in remote cluster.

Default: hydro.logs


logs_http_remote_transform⚓︎

Type: string | Added v4.20

A transform schema for ingest in remote cluster.

Default: megaTransform


logs_http_table⚓︎

Type: string | Added v4.20

An existing Hydrolix where the data should land.

Default: hydro.logs


logs_http_transform⚓︎

Type: string | Added v4.20

A transform schema for ingest.

Default: megaTransform


logs_kafka_bootstrap_servers⚓︎

Type: string | Added v4.20

A comma separated list of Kafka bootstrap servers to send logs to.

Default: redpanda


logs_kafka_topic⚓︎

Type: string | Added v4.20

A Kafka topic to send logs to.

Default: logs


logs_sink_local_url⚓︎

Type: string | Added v4.20

The full URI to make local HTTP request to.

Default: http://hydrologs-intake-head:8089/ingest/event


logs_sink_remote_auth_enabled⚓︎

Type: boolean | Added v4.20

Specify if remote auth is enabled. If enabled and bearer token is used for authentication please specify the same via logs_sink_remote_auth_type tunable

Default: False


logs_sink_remote_auth_type⚓︎

Type: string | Added v5.5.0

Specify if auth type is basic auth or token. Allowed values are basic, token

Default: basic

Examples
spec:
  logs_sink_remote_auth_type: basic
spec:
  logs_sink_remote_auth_type: token

logs_sink_remote_url⚓︎

Type: string | Added v4.20

The full URI to make remote HTTP request to.


logs_sink_type⚓︎

Type: string | Added v4.20

Type of logs sink.

Default: http


logs_topic_partition_count⚓︎

Type: integer | Added v4.20

The number of partitions to assign to the logs topic for stream processing.

Default: 81


max_concurrent_queries⚓︎

Type: integer | Added v4.20

Max limit on total number of concurrently executed queries. Zero means unlimited.

Default: 0


max_exp_backoff_seconds⚓︎

Type: integer | Added v4.20

Cap for exponentially back off sleep time.

Default: 20


max_http_retries⚓︎

Type: integer | Added v4.20

Maximum times to retry any query-related HTTP requests that fail.

Default: 3


max_server_memory_usage_perc⚓︎

Type: integer | Added v4.20

Max % of total system memory that server can use and allocate for its operation.

Default: 0


mcp_hydrolix⚓︎

Type: dict | Added v5.9.3 | Reference

Hydrolix MCP server configuration.

NOTE: enabled by default.

  • enabled: Set to True to enable the MCP server.

  • version: Use to run an alternative version.

  • secret_name: Use to provide Hydrolix server connection credentials.

  • mcp_server: MCP server bind and transport configuration.

  • hydrolix_connection: Configuration for connection to Hydrolix cluster.

Default value
spec:
  mcp_hydrolix:
    enabled: true
    hydrolix_connection:
      connect_timeout: 30
      default_database: ''
      host: query-head
      pool_size: 100
      port: 8088
      proxy_path: ''
      query_timeout_sec: 30
      secure: false
      send_receive_timeout: 60
      verify_ssl: false
    mcp_server:
      bind_host: 0.0.0.0
      bind_port: 8000
      transport: http
      worker:
        enabled: true
        keepalive: 10
        max_requests: 10000
        max_requests_jitter: 1000
        request_timeout: 70
        worker_connections: 100
        workers: 2
    secret_name: mcp-hydrolix
    version: v0.2.1
Example
1
2
3
4
5
6
7
spec:
  mcp_hydrolix:
    enabled: true
    hydrolix_connection:
      pool_size: 100
    secret_name: mcp-hydrolix
    version: v0.1.6

memory_tracker_rss_sync_period_seconds⚓︎

Type: integer | Added v5.10

Duration in seconds specifying how often the memory tracker is synchronized with the RSS value.

Default: 0


merge_candidate_concurrency⚓︎

Type: integer | Added v4.20

Number of concurrent Merge Candidate construction queries to run.

Default: 6


merge_cleanup_batch_size⚓︎

Type: integer | Added v4.20

Number of entries to fetch for each request to the catalog.

Default: 5000


merge_cleanup_delay⚓︎

Type: string | Added v4.20

How long before a merged partition should be deleted expressed as a duration string.

Default: 15m


merge_cleanup_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the Merge Clean-up Cron Job should run.

Default: True


merge_cleanup_schedule⚓︎

Type: string | Added v4.20

CRON schedule for Merge Clean-up Cron Job

Default: */5 * * * *


merge_controller_enabled⚓︎

Type: boolean | Added v5.0

Whether or not the next generation merge controller is enabled.

Default: True


merge_dispatch_frequency⚓︎

Type: string | Added v4.20

How often a slot should be checked for exceeding max_idle. Expressed as duration string. For example, 5s.

Default: 5s


merge_download_partitions_enabled⚓︎

Type: boolean | Added v5.3

Whether or not merge-peer should download partitions locally for processing.

Default: False


merge_first_era_frequency⚓︎

Type: string | Added v4.20

How often merge candidates should be constructed for the first era.

Default: 10s


merge_head_batch_size⚓︎

Type: integer | Added v4.20

Number of records to pull from the catalog per request by the merge head.

Default: 10000


merge_interval⚓︎

Type: string | Added v4.20

The time the merge process waits between checking for mergeable partitions.

Default: 15s


merge_lock_bad_partitions_enabled⚓︎

Type: boolean | Added v5.3

Whether or not merge-peer should lock partitions which cannot be read by turbine.

Default: False


merge_max_candidates⚓︎

Type: integer | Added v4.20

Number of candidates to produce per merge target each cycle.

Default: 100


merge_max_partitions⚓︎

Type: integer | Added v5.4

Number of partitions to be buffered awaiting candidate construction.

Default: 10000


merge_max_partitions_per_candidate⚓︎

Type: integer | Added v4.20

The maximum number of partitions per merge candidate.

Default: 100


merge_min_mb⚓︎

Type: integer | Added v4.20

Size in megabytes of the smallest merge tier. All other merge tiers are multiples of this value.

Default: 1024


merge_primary_window_width⚓︎

Type: string | Added v4.20

Specifies the interval used to further filter partition selection queries. Smaller values limit the number of records the database needs to produce, but can increase query count.

Default: 1080h


merge_second_era_frequency⚓︎

Type: string | Added v4.20

How often merge candidates should be constructed for the second era.

Default: 60s


merge_streaming_selector⚓︎

Type: boolean | Added v4.20

Whether or not to use the Streaming Candidate Selector

Default: True


merge_third_era_frequency⚓︎

Type: string | Added v4.20

How often merge candidates should be constructed for the third era.

Default: 60m


metadata⚓︎

Type: object | Added v5.2 | Reference

Custom kubernetes labels and annotations to propagate to hydrolix workloads. Changing this value will trigger restarts for all services

Example
1
2
3
4
5
6
spec:
  metadata:
    annotations:
      example.com/owner: hdx
    labels:
      env: dev

monitor_ingest⚓︎

Type: boolean | Added v4.20

If enabled, deploy a service to ingest a timestamp into the hydro.monitor table every second.

Default: False


monitor_ingest_pool_exemptions⚓︎

Type: array | Added v5.7.4

List of pool names to be ignored from ingestion heartbeat validation. The names must match the pool names provided in the pools spec

Examples
spec:
  monitor_ingest_pool_exemptions: kinesis-peer-pool
spec:
  monitor_ingest_pool_exemptions: merge-pool

monitor_ingest_request_timeout⚓︎

Type: number | Added v4.21

The number in seconds for HTTP timeout in HTTP POST from monitor_ingest.

Default: 1


monitor_ingest_retry_timeout⚓︎

Type: number | Added v4.21

The deadline for one submission by monitor ingest including all retries.

Default: 1


monitor_ingest_timeout⚓︎

Type: number | Added v4.20

Deprecated. Use monitor_ingest_request_timeout.


mysql_port⚓︎

Type: integer | Added v5.0

The port to serve the Click House My SQL interface on if applicable.

Default: 9004


mysql_port_disable_tls⚓︎

Type: boolean | Added v5.0

When True, Traefik will not use TLS configuration on My SQL TCP route.

Default: False


native_port⚓︎

Type: integer | Added v4.20

The port to serve the Click House plaintext native protocol on if applicable.

Default: 9000


native_tls_port⚓︎

Type: integer | Added v4.20

The port to serve the Click House TLS native protocol on if applicable.

Default: 9440


network_policies_enabled⚓︎

Type: boolean | Added v5.10

Install network policies

Default: False


oom_detection⚓︎

Type: dict | Added v4.21

Configuration options for detecting indexing OOM scenarios and retry with smaller data sizes if possible for services that perform ingest.

Supported services:

  • intake-head

  • intake-peer

  • kafka-peer

  • kinesis-peer

  • akamai-siem-peer

Available keys under each service:

  • k8s_oom_kill_detection_enabled - Enable detection of Kubernetes OOM kills

  • k8s_oom_kill_detection_max_attempts - Maximum retry attempts after OOM kill

  • circuit_break_oom_detection_enabled - Enable circuit breaker for OOM detection

  • preemptive_splitting_enabled - Enable preemptive data splitting to avoid OOM"

Default value
1
2
3
4
5
6
7
spec:
  oom_detection:
    akamai_siem_peer: null
    intake_head: null
    intake_peer: null
    kafka_peer: null
    kinesis_peer: null
Example
1
2
3
4
5
6
7
spec:
  oom_detection:
    akamai_siem_peer: null
    intake_head: null
    intake_peer: null
    kafka_peer: null
    kinesis_peer: null

otel_endpoint⚓︎

Type: string | Added v4.20

Send OTLP data to the HTTP server at this URL.


overcommit⚓︎

Type: object | Added v4.20

When true, removes all requests and limits from Kubernetes containers. Useful when running on a single node Kubernetes cluster with constrained resources. When set to requests, only turns off requests. Similarly, limits removes just the limits. Not being set is the same as false. Note that removing either a memory or CPU limit or request from any container on a pod removes the Guaranteed quality of service class from that pod.


overrides⚓︎

Type: object | Added v5.4

Applies temporary, in-memory patches to the HDX spec during scheduled periods.

Override key should be a unique name. Each override must contain:

  • timezone (string, required): IANA TZ to evaluate the schedule; defaults to UTC. Offsets (Z, -05: 00, etc.) are ignored to avoid ambiguity.

  • Exactly one schedule block:

  • weekly: { days: [Sun.. Sat], start: "HH: MM", end: "HH: MM" }

  • window: { start: ISO-8601 datetime, end: ISO-8601 datetime }

  • cron: { expression: 5-field cron, duration: "4h" | "30m" }

  • patch (dict, required): partial HDX spec to apply while active.

Multiple overrides can be active at the same time; later entries win on key conflicts.

The HDX custom resource itself is never mutated. Overrides live only in operator memory.

The operator writes currently active override names to .status.active Overrides.

Example
spec:
  overrides:
    nightly-shrink:
      cron:
        duration: 4h
        expression: 0 2 * * 1-5
      patch:
        pools:
          hydrologs-intake-head:
            replicas: '0'
    postgres-sunday-gameday:
      patch:
        scale:
          postgres:
            replicas: 999
      timezone: America/New_York
      weekly:
        days:
        - Sun
        end: '22:00'
        start: '16:00'
    winter-freeze:
      patch:
        celebrate: true
        scale_off: 'true'
      window:
        end: '2000-01-02T09:00:00Z'
        start: '1999-12-24T18:00:00Z'

owner⚓︎

Type: string | Added v4.20

DEPRECATED: this was previously used internally by Hydrolix.


partition_cleaner_dry_run⚓︎

Type: boolean | Added v4.21

If true, Partition Cleaner will only log it's intentions and take no action

Default: True


partition_cleaner_grace_period⚓︎

Type: string | Added v4.22

Minimum age of a partition before it is considered for deactivation or deletion expressed as a duration string.

Default: 24h


partition_cleaner_schedule⚓︎

Type: string | Added v4.21

Crontab style schedule for when partition cleaner should run.

Default: 0 0 * * 1


password_expiration_policy⚓︎

Type: integer | Added v4.20

Number of days to expire password


patch_date⚓︎

Type: string | Added v5.5

Optional ISO-8601 cut-off date (YYYY-MM-DD) that pins every container image in the cluster to the newest build published on or before that date. If unset the operator freezes the cluster on the oldest patch ever released for its Hydrolix version.


pg_ssl_mode⚓︎

Type: string | Added v4.20

Determines whether and with what priority an SSL connection is negotiated when connecting to a Postgre SQL server. See https://www.postgresql.org/docs/current/libpq-ssl.html.

Default: disable

Examples
spec:
  pg_ssl_mode: disable
spec:
  pg_ssl_mode: require
spec:
  pg_ssl_mode: verify-ca
spec:
  pg_ssl_mode: verify-full

pgbouncer_client_ca_secret⚓︎

Type: string | Added v5.4

The secret that holds the ca certificates

Default: catalog-ca


pgbouncer_client_tls_secret⚓︎

Type: string | Added v5.4

The secret that holds the client tls certificates

Default: catalog-server


pgbouncer_enabled⚓︎

Type: boolean | Added v5.4

The tunable installs pgbouncer.

Default: False


pgbouncer_max_client_conn⚓︎

Type: integer | Added v5.4

Maximum number of client connections allowed.

Default: 1000


pgbouncer_metrics_port⚓︎

Type: integer | Added v5.4

The port on which pgbouncer metrics can be scraped.

Default: 9127


pgbouncer_pool_mode⚓︎

Type: string | Added v5.4

The pooling method to use for connecting to backend postgres

Default: session

Examples
spec:
  pgbouncer_pool_mode: session
spec:
  pgbouncer_pool_mode: statement

pgbouncer_pool_size⚓︎

Type: integer | Added v5.4

Number of server connections to allow per user/database pair.

Default: 20


pgbouncer_port⚓︎

Type: integer | Added v5.4

The port on which pgbouncer starts.

Default: 6432


pools⚓︎

Type: object | Added v4.20

Collection of pool configurations.

Pools can be specified as either:

  1. A dictionary mapping pool names to Pool Config objects
  2. A list of Pool Config objects (converted to dict internally)

Example (dict format):

1
2
3
4
5
pools:
my-pool:
service: intake-head
replicas: 2
cpu: "4"

Example (list format):

1
2
3
4
5
6
pools:

- name: my-pool
service: intake-head
replicas: 2
cpu: "4"

priority_classes⚓︎

Type: object | Added v5.9.3

This tunable allows specifying priority classes per workload. Allowed values are system-node-critical, hdx-critical, hdx-highest, hdx-high, hdx-medium, hdx-low, hdx-lowest.

Example
spec:
  priority_classes:
    daemonset:
      vector: hdx-low
    deployments:
      intake-head: hdx-critical
      merge-peer: hdx-low
    statefulset:
      rabbitmq: hdx-critical
      redpanda: hdx-low

prometheus_curated_configmap⚓︎

Type: string | Added v5.0

Custom curated Prometheus Config Map that will be mounted onto the Prometheus pod.


prometheus_enabled⚓︎

Type: boolean | Added v5.3

This tunable controls prometheus installation

Default: True


prometheus_ignored_apps⚓︎

Type: string | Added v5.2

A comma delimited list of app labels to ignore when determining scrape targets for prometheus

Examples
spec:
  prometheus_ignored_apps: batch-head
spec:
  prometheus_ignored_apps: stream-peer,vector

prometheus_label_value_length_limit⚓︎

Type: integer | Added v4.20

If a label value is larger than the value configured, Prometheus discards the entire scrape.

Default: 512


prometheus_namespace⚓︎

Type: string | Added v5.3

The namespace where kube-prometheus service is running which is used when prometheus operator is used instead of the defacto prometheus installed with hydrolix.


prometheus_operator_installed⚓︎

Type: boolean | Added v5.3

This tunable indicates if prometheus-operator is installed. It does not perform installation if the flag is set to False

Default: False


prometheus_remote_write_url⚓︎

Type: string | Added v4.20

A URL you wish to use to configure Prometheus's remote-write functionality.


prometheus_remote_write_username⚓︎

Type: string | Added v4.20

The username for Prometheus to use with basic auth to connect to a remote-write endpoint. Ignored if prometheus_remote_write_url is not set.

Default: hdx


prometheus_retention_ratio⚓︎

Type: string | Added v4.20

The amount of the volume to reserve for Prometheus data. Example: 0.7

Default: 0.7


prometheus_retention_size⚓︎

Type: string | Added v4.20

The maximum number of bytes of Prometheus data to retain. Overrides prometheus_retention_ratio. Units supported: B, KB, MB, GB, TB, PB, EB


prometheus_retention_time⚓︎

Type: string | Added v4.20

When to remove old Prometheus data. Example: 15d


prometheus_scrape_interval⚓︎

Type: string | Added v4.20

How frequently to scrape targets by default.

Default: 15s


prometheus_service_name⚓︎

Type: string | Added v5.3

The kube-prometheus service name which is used when prometheus operator is used instead of the defacto prometheus installed with hydrolix.


prometheus_service_port⚓︎

Type: integer | Added v5.3

The kube-prometheus service port which is used when prometheus operator is used instead of the defacto prometheus installed with hydrolix.

Default: 9090


prometheus_servicemonitor_selector⚓︎

Type: object | Added v5.3

Prometheus custom resource(CR) uses this tunable to identify the servicemonitors to be scraped for metrics

Example
1
2
3
spec:
  prometheus_servicemonitor_selector:
    hydrolix: 'true'

prune_locks_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the Prune Locks Cron Job should run.

Default: True


prune_locks_grace_period⚓︎

Type: string | Added v4.20

Minimum age of a lock before it is considered for removal expressed as a duration string.

Default: 24h


prune_locks_schedule⚓︎

Type: string | Added v4.20

CRON schedule for Prune Locks Cron Job

Default: 30 0 * * *


publish_deleted_age⚓︎

Type: string | Added v5.7.4

How long to publish a deleted table in the config, expressed as a duration string.

Default: 14d


query_peer_liveness_check_path⚓︎

Type: string | Added v4.20

The HTTP path used to configure a Kubernetes liveness check for query-peers. Set to none to disable.

Default value
spec:
  query_peer_liveness_check_path: ?query=select%20count%28id%29%20from%20hdx.liveliness%20SETTINGS%20hdx_log_query=false%2Chdx_internal_query=1

query_peer_liveness_failure_threshold⚓︎

Type: integer | Added v4.20

How many times query liveness check can fail.

Default: 5


query_peer_liveness_initial_delay⚓︎

Type: integer | Added v4.23

Time in seconds to wait before starting query liveness checks.

Default: 300


query_peer_liveness_period_seconds⚓︎

Type: integer | Added v4.20

How often should query liveness check run, in seconds.

Default: 60


query_peer_liveness_probe_timeout⚓︎

Type: integer | Added v4.23

Number of seconds after which the liveness probe times out

Default: 10


query_readiness_initial_delay⚓︎

Type: integer | Added v4.20

Time in seconds to delay startup probes for turbine containers.

Default: 0


refresh_job_statuses_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the Refresh Job Statuses Cron Job should run.

Default: True


refresh_job_statuses_schedule⚓︎

Type: string | Added v4.20

CRON schedule for Refresh Job Statuses Cron Job

Default: * * * * *


registry⚓︎

Type: string | Added v4.20

A docker registry to pull Hydrolix containers from.

Default: us-docker.pkg.dev/hdx-art/t


rollout_strategy_max_surge⚓︎

Type: integer | Added v4.23 | Reference

Configures the number of pods (represented as percentage) that can be created above the desired amount of pods during deployment rollout update.

Default: 25


rollout_strategy_max_unavailable⚓︎

Type: integer | Added v4.23 | Reference

Ensures the number of pods (represented as integer) that can be unavailable during deployment rollout update.

Default: 0


sample_data_url⚓︎

Type: string | Added v4.20

The storage bucket URL to use to load sample data.


scale⚓︎

Type: dict | Added v4.20

Top-level scale configuration for Hydrolix services.

This is a dictionary where:

  • Keys can be scalable service names (e.g., 'postgres', 'intake-head', 'query-peer') mapped to Service Scale Config objects

  • The special key 'profile' maps to named scale profiles, where each profile contains its own dictionary of service names to Service Scale Config objects

Example combining direct service configs and profiles:

scale:
intake-head:
replicas: 2
cpu: "4"
profile:
prod:
query-peer:
replicas: 10
cpu: "32"
dev:
query-peer:
replicas: 1
cpu: "4"

The 'profile' key is reserved and cannot be used as a service name. Within a profile, only service configurations are allowed (no nested profiles).

Valid service names are dynamically loaded from hkt.scale.get_raw_table_entries().


scale_min⚓︎

Type: boolean | Added v5.3

When true, similar to scale_off but keeps API, UI and their dependencies running.

Default: False


scale_off⚓︎

Type: boolean | Added v4.20

When true, override all deployment and Stateful Set replica counts with a value of 0 and disable vector.

Default: False


scale_profile⚓︎

Type: string | Added v4.20

Selects from a set of predefined defaults for scale

Default: eval


sdk_timeout_sec⚓︎

Type: integer | Added v4.20

How many seconds the Merge SDK should be given to run before it is killed.

Default: 300


siem_backoff_duration⚓︎

Type: string | Added v4.20

Backoff duration when SIEM limit not hit, for politeness.

Default: 1s


silence_linode_alerts⚓︎

Type: boolean | Added v4.20

If true will run a Daemon Set that turns off Linode alerts for LKE nodes.

Default: False


skip_init_turbine_api⚓︎

Type: boolean | Added v4.20

Skips running database migrations in the init-turbine-api job. Set to true when running multiple clusters with a shared database

Default: False


sql_transform_max_ast_elements⚓︎

Type: integer | Added v4.20

The number of AST elements an SQL transform can contain. This limits the maximum complexity of a SQL transform.

Examples
spec:
  sql_transform_max_ast_elements: 100000
spec:
  sql_transform_max_ast_elements: 150000

sql_transform_max_expanded_ast_elements⚓︎

Type: integer | Added v4.20

The number of expanded AST elements an SQL transform can contain. This limits the maximum complexity of a SQL transform.

Examples
spec:
  sql_transform_max_expanded_ast_elements: 100000
spec:
  sql_transform_max_expanded_ast_elements: 150000

stale_job_monitor_batch_size⚓︎

Type: integer | Added v4.20

How many jobs to probe in a single request.

Default: 300


stale_job_monitor_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the Stale Job Monitor Cron Job should run.

Default: True


stale_job_monitor_limit⚓︎

Type: integer | Added v4.20

How many jobs in total Stale Job will process per cycle.

Default: 3000


stale_job_monitor_schedule⚓︎

Type: string | Added v4.20

CRON schedule for Stale Job Monitor

Default: */5 * * * *


str_dict_enabled⚓︎

Type: boolean | Added v4.20

Enable/disable multi-threaded string dictionary decoding.

Default: True


str_dict_min_dict_size⚓︎

Type: integer | Added v4.20

Controls the number of entries in each string dictionary block.

Default: 32768


str_dict_nr_threads⚓︎

Type: integer | Added v4.20

Sets the maximum number of concurrent v CPU used for decoding.

Default: 8


stream_concurrency_limit⚓︎

Type: integer | Added v4.20

The number of concurrent stream requests per cpu allocated across all pods beyond which Traefik will return 429 busy error responses. If not set or set to null no limit is enforced.


stream_load_balancer_algorithm⚓︎

Type: string | Added v4.20

The load balancer algorithm to use with stream-head and intake-head services. Available load balancing algorithms: Round Robin - rr, Least Connection Power of Two Choices - p2c

Default: rr


stream_partition_block⚓︎

Type: integer | Added v4.20

The number of partitions to use on a non-default Redpanda stream topic per TB/day of usage.

Default: 6


stream_partition_count⚓︎

Type: integer | Added v4.20

The number of partitions to use on the default Redpanda topic for stream service.

Default: 50


stream_replication_factor⚓︎

Type: integer | Added v4.20

The replication factor for the internal Redpanda topic used by the stream service it must always be less than the number of Redpanda replicas. If it is not, the configuration will not change.

Default: 3


targeting⚓︎

Type: object | Added v4.20

A dictionary to pass targeting related Kubernetes settings to resources according to what Hydrolix service they are part of.

Keys can be:

  • "*": Default configuration for all services

  • Service names: e.g., "postgres", "query-peer", "intake-head"

  • Pool-specific: e.g., "pool: my-pool"

Target.from_spec() merges configurations with priority (later selectors override earlier). For example, with selectors ["*", "query-peer", "pool: my-pool"]:

  1. Applies "*" (global defaults)
  2. Overrides with "query-peer" (service-specific)
  3. Overrides with "pool: my-pool" (pool-specific)

task_monitor_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the Task Monitor Cron Job should run.

Default: True


task_monitor_heartbeat_timeout⚓︎

Type: integer | Added v4.20

How old a tasks heartbeat should be (in seconds) before it is timed out.

Default: 600


task_monitor_schedule⚓︎

Type: string | Added v4.20

CRON schedule for Task Monitor.

Default: */2 * * * *


task_monitor_start_timeout⚓︎

Type: integer | Added v4.20

How old a ready task should be (in seconds) before it is considered lost and timed out.

Default: 21600


terminate_tls_at_lb⚓︎

Type: boolean | Added v5.4

This flag controls if traefik accepts http or https traffic from load balancer. This is required if TLS termination needs to be done at the load balancer and plain traffic is sent to traefik backend

Default: False


thanos_enabled⚓︎

Type: boolean | Added v5.6.2

If true, configures prometheus statefulset with thanos sidecar.

Default: False


thanos_tls_secret⚓︎

Type: string | Added v5.6.2

The secret that holds the tls certificates for performing MTLS with thanos query server

Default: thanos-tls


traefik_external_ips⚓︎

Type: array | Added v4.20

Traffic that ingresses into the cluster with one of these IPs gets directed to the Traefik service. Useful in particular when deploying all on one node.

Examples
1
2
3
4
spec:
  traefik_external_ips:
  - 192.168.1.5
  - 192.16.1.4
1
2
3
spec:
  traefik_external_ips:
  - 172.16.0.8

traefik_hsts_expire_time⚓︎

Type: integer | Added v4.20

Expiration time for HSTS caching in seconds.

Default: 315360000


traefik_keep_alive_max_time⚓︎

Type: integer | Added v4.20

The number of seconds a client HTTP connection can be reused before receiving a Connection: close response from the server. Zero means no limit.

Default: 26


traefik_load_balancer_class⚓︎

Type: string | Added v5.8.6

Sets the load Balancer Class field for the traefik service. Useful for finer control of cloud specific service controllers

Example
spec:
  traefik_load_balancer_class: service.k8s.aws/nlb

traefik_pre_stop_seconds⚓︎

Type: integer | Added v5.9.3

Number of seconds to sleep when traefik pods are terminating

Default: 15


traefik_service_allowed_headers⚓︎

Type: array | Added v5.4

Header keys used in intake-head route configuration for pools. The entries should match the keys being passed in the pool annotations. The traefik daemon checks if pool annotations are in this list and if so crafts the traefik router rule accordingly. If left empty, operator checks if x-hdx-table, x-hdx-transform annotations are set for the pool and if so, uses them.

Examples
spec:
  traefik_service_allowed_headers: x-hdx-table
spec:
  traefik_service_allowed_headers: x-hdx-transform

traefik_service_allowed_query_params⚓︎

Type: array | Added v5.4

Query params used in intake-head route configuration for pools. The entries should match the keys being passed in the pool query_params. The traefik daemon checks if pool query_params are in this list and if so crafts the traefik router rule accordingly. If left empty, operator checks if table, transform params are set for the pool and if so, uses them.

Examples
spec:
  traefik_service_allowed_query_params: table
spec:
  traefik_service_allowed_query_params: transform

traefik_service_annotations⚓︎

Type: object | Added v4.22

Additional annotations for Traefik service.


traefik_service_cors_headers⚓︎

Type: object | Added v5.2 | Reference

Optional key values pairs of CORS headers


traefik_service_custom_response_headers⚓︎

Type: object | Added v5.2 | Reference

Optional key value pairs of custom headers that will be applied to the response


traefik_service_type⚓︎

Type: string | Added v4.20

The type of service to use for Traefik, the entry point to the cluster.

Default: public_lb

Examples
spec:
  traefik_service_type: public_lb
spec:
  traefik_service_type: private_lb
spec:
  traefik_service_type: node_port
spec:
  traefik_service_type: cluster_ip

traefik_use_local_policy⚓︎

Type: boolean | Added v5.9.3

Sets external Traffic Policy to Local for the traefik service

Default: False


tulugaq⚓︎

Type: dict | Added v5.9.3

Tulugaq metrics streaming configuration.

Default value
spec:
  tulugaq:
    compress: false
    config:
      filters:
        metrics:
          mode: include
          raw: "\n                    http_source_request_count{'component_ip': '10.2.2.11'}\
            \ 1\n                    http_source_request_count 0\n               \
            \     "
    enabled: false
    max_request_rows: 2000
    scrape_interval: 15
    stream_table: hydro.metrics
    stream_to_url: http://stream-head:8089/ingest/event
    stream_transform: hydro_wide_metrics_transform

turbine_api_database_connection_max_age⚓︎

Type: integer | Added v5.10

This tunable allows specifying the default connection max age for the database at the turbine_api application layer. When 0, closes the connection at the end of each request, the default behavior.

Default: 0


turbine_api_database_connection_timeout⚓︎

Type: integer | Added v5.10

This tunable allows specifying the default connection timeout for the database at the turbine_api application layer. Measured in seconds.

Default: 10


turbine_api_init_pools⚓︎

Type: boolean | Added v4.20

If enabled, the turbine-api component initializes some pools.

Default: False


turbine_api_metrics_enabled⚓︎

Type: boolean | Added v5.10

If true, exports metrics for turbine-api service.

Default: True


turbine_api_require_table_default_storage⚓︎

Type: boolean | Added v5.0

If enabled, turbine-api will require tables to have their storage_map be populated with a default_storage_id. Useful when use of the cluster's default bucket should be discouraged.

Default: False


turbine_api_worker_count⚓︎

Type: integer | Added v5.7.4

Determines the number of workers that turbine-api will start within one container. Used to fine-tune the API's ability to handle large numbers of requests. Larger worker counts may require memory to be scaled up as well.

Default: 8


unified_auth⚓︎

Type: boolean | Added v4.20

Use the same auth used with the API for all services.

Default: True


usagemeter_enabled⚓︎

Type: boolean | Added v4.20

Whether or not the usage meter cron job should run.

Default: True


usagemeter_preserve⚓︎

Type: string | Added v4.23

Duration to hang onto old, already-reported usage meter data on local clusters.

Default: 1440h


usagemeter_query_timeout⚓︎

Type: string | Added v4.20

Maximum time to wait for query against catalog to complete.

Default: 4m


usagemeter_reporting_table⚓︎

Type: string | Added v4.20

Hydrolix table to send usage to, in project.table format.

Default: metering_project.metering_table


usagemeter_reporting_transform⚓︎

Type: string | Added v4.20

Hydrolix transform name or UUID for usage reporting.

Default: metering_transform


usagemeter_reporting_url⚓︎

Type: string | Added v4.20

URL to send usage data to.

Default: https://prometheus-us.trafficpeak.live/ingest


usagemeter_request_timeout⚓︎

Type: string | Added v4.20

Maximum time to wait for reporting HTTP request to complete.

Default: 1m


usagemeter_schedule⚓︎

Type: string | Added v4.20

CRON schedule for usage meter cron job. Defaults to every 10 minutes.

Default: */10 * * * *


use_auth_traefik_plugins⚓︎

Type: boolean | Added v5.8.6

Setting this parameter to true will enable hdx-auth traefik plugin instead of hdx-traefik-auth sidecar.

Default: False


use_https_with_s3⚓︎

Type: string | Added v4.20

DEPRECATED: Use db_bucket_url or db_bucket_http_enabled.


use_hydrolix_dns_resolver⚓︎

Type: boolean | Added v4.20

If true, use Hydrolix DNSResolver. If false, use system resolver.

Default: True


use_tls⚓︎

Type: boolean | Added v4.20

DEPRECATED: inferred from hydrolix_url.


user_acl_refresh_interval_secs⚓︎

Type: integer | Added v4.20

Frequency at which user ACL permissions are refreshed (in secs)

Default: 30


user_token_expiration_secs⚓︎

Type: integer | Added v4.20

user token expiration period (in secs)

Default: 1800


user_token_refresh_interval_secs⚓︎

Type: integer | Added v4.20

Frequency at which user tokens are refreshed (in secs)

Default: 240


vector_bucket⚓︎

Type: string | Added v4.20

Bucket where Vector should save JSON format pod logs.


vector_bucket_path⚓︎

Type: string | Added v4.20

Prefix under which vector will save pod logs.

Default: logs


vector_custom_fields⚓︎

Type: object | Added v5.7.4

Custom key/value pairs to add as fields to all vector logs. Useful for distinguishing logs when using a remote sink

Example
1
2
3
4
spec:
  vector_custom_fields:
    env: dev
    source: lke

vector_extra_namespaces⚓︎

Type: array | Added v5.4

List of additional namespaces that vector should scrape pod logs from

Example
1
2
3
4
spec:
  vector_extra_namespaces:
  - kube-system
  - default