Skip to content

Pod Priority Classes

Overview⚓︎

Hydrolix uses Kubernetes priority and preemption for pod scheduling and eviction. If Kubernetes can't schedule a pod, it preempts lower-priority pods in favor of higher-priority pending pods requesting scheduling.

Priority classes are installed when the Hydrolix operator is deployed. No additional action is needed to create them.

This feature was introduced in Hydrolix version 5.9.

Terminology⚓︎

Term Description
PriorityClass A non-namespaced Kubernetes object that maps a priority class name to an integer priority value.
Priority Indicates the importance of a pod relative to other pods. Higher values mean higher priority.
Preemption Policy Determines whether lower-priority pods are evicted to accommodate higher-priority pods. Allowed values are Never and PreemptLowerPriority, default for all custom Hydrolix priority classes.

Priority classes⚓︎

Hydrolix uses these priority classes:

Name (priorityClassName) Value Preemption policy Source
system-node-critical 2_000_001_000 PreemptLowerPriority Kubernetes built-in
system-cluster-critical 2_000_000_000 PreemptLowerPriority Kubernetes built-in
hdx-critical 10_000 PreemptLowerPriority Custom Hydrolix priority class
vector 10_000 PreemptLowerPriority Custom Hydrolix priority class
hdx-highest 9_900 PreemptLowerPriority Custom Hydrolix priority class
hdx-high 9_800 PreemptLowerPriority Custom Hydrolix priority class
hdx-medium 9_700 PreemptLowerPriority Custom Hydrolix priority class
hdx-low 9_600 PreemptLowerPriority Custom Hydrolix priority class
hdx-lowest 9_500 PreemptLowerPriority Custom Hydrolix priority class

All custom Hydrolix priority classes use PreemptLowerPriority, meaning higher-priority pods can evict lower-priority pods when resources are constrained.

Default service assignments⚓︎

Every Hydrolix workload is assigned a default priority class. The operator applies these assignments during deployment.

system-node-critical⚓︎

Node-level DaemonSets that must remain on their assigned node:

  • cadvisor
  • node-exporter
  • silence-linode
  • hdx-node

hdx-critical⚓︎

Core infrastructure services. Losing these affects the entire cluster:

  • catalog-proxy, operator, grafana, hdx-gate, hdx-scaler
  • keycloak, pgbouncer, pushgateway, spill-controller
  • superset, tooling, traefik, turbine-api
  • quesma
  • postgres, prometheus, rabbitmq, zookeeper
  • elasticsearch, kibana

hdx-highest⚓︎

Ingest-path services. These handle incoming data and must stay running to prevent data loss:

  • http-head, intake-head, intake-peer, intake-router
  • kafka-peer, merge-controller, monitor-ingest
  • hdx-pg-monitor, stream-head, stream-peer

hdx-high⚓︎

Query and processing services:

  • query-head, query-peer, alter-head, batch-head, batch-controller
  • merge-head, merge-peer, autoingest
  • hydrologs-intake-head, http-proxy, intake-api
  • turbine-api-worker, anomaly-rca, ariadne-core
  • ariadne-janus, tulugaq-autobot, mcp-hydrolix

hdx-medium⚓︎

Supporting services:

  • janus-proxy, pgbouncer-exporter, merge-peer-i
  • merge-peer-summary-a, kibana-gateway, reloader

hdx-low⚓︎

Lower-priority services that can tolerate brief interruptions:

  • alter-peer, batch-peer, merge-peer-ii
  • merge-peer-summary-b, partition-cleaner, periodic-service
  • presidio-analyzer, presidio-anonymizer, ui
  • redpanda

hdx-lowest⚓︎

Background services:

  • merge-peer-iii, merge-peer-summary-c
  • reaper, validator, version

Vector⚓︎

The vector DaemonSet uses a dedicated priority class with the same value as hdx-critical: 10_000.

Non-Hydrolix workloads in the same namespace⚓︎

Hydrolix pods preempt unprotected workloads

If you run additional Kubernetes workloads in the same namespace as Hydrolix, all Hydrolix pods have priority classes that allow them to preempt pods that lack a priority class or have a lower-priority value.

During resource constraints, such as traffic spikes while new nodes are provisioning, the Kubernetes scheduler can evict your non-Hydrolix pods to make room for Hydrolix services.

To protect non-Hydrolix workloads sharing the namespace, assign priority classes to those workloads. You can use any Hydrolix priority class such as hdx-high, or create your own. See the Kubernetes documentation on PriorityClass for instructions.

For instructions on overriding default Hydrolix priority class assignments, see Override pod priority classes.