Scale Clusters
Overview⚓︎
Hydrolix supports both manual and automatic scaling. You can scale individual components, apply cluster-wide profiles, or enable the HDX Autoscaler to react to Prometheus metrics. This page introduces each option and links to detailed guides.
Profiles and Overrides⚓︎
- Scale profiles: Use pre-made profiles designed for different ingest workloads.
- Custom profiles: Create named overrides for specific services or pools.
- Scale profile overrides: Override profile settings for individual components.
- Scheduled overrides: Apply scaling changes on a schedule.
Component-Level Scaling⚓︎
- Scale by component: Set resources or replica counts in the
scalesection ofhydrolixcluster.yaml. - Stateful and stateless scaling: Learn how scaling differs for storage-backed and stateless services.
Autoscaling⚓︎
- HDX Autoscaler with Prometheus: Scale dynamically based on custom metrics.
- Cool-up and cool-down: Set minimum wait times between scale-up or scale-down events to prevent thrash.
- Scheduled overrides: Apply scaling changes on a schedule.
- Scale to zero: Reduce all replicas to zero except the operator.
- Scale to minimal: Keep the API and UI available while reducing most components.
- Tolerance dead-zone: Define a dead-zone around the target metric where no scaling occurs, to avoid constant up/down adjustments.
Resource tuning⚓︎
- Overcommit: Ignore Kubernetes CPU and memory requests and limits when running in resource-constrained environments.