Autoscaling and Spot Instances

Autoscaling

Hydrolix supports the ability to enable Autoscaling for Query-Peers. It can be set for other components, however this is not recommended.

The argument to scaling count arguments takes not just a single value but also allows specifying a range. You can either specify a single number, a min, and max or a min, desired and max separated by dashes. If you specify a range, the control plane will create Cloudwatch alerts on CPU that will trigger scaling events. The following works as before, no autoscaling alarms are created:

$ hdxctl scale --query-peer-count 1

The following sets the min to one, the desired count to one and the max to five and creates alerts for scaling up and down based on CPU.

$ hdxctl scale --stream-peer-count 1-5

The following does the same as the above but sets the desired count to 3.

$ hdxctl scale --stream-peer-count 1-3-5

Alternatively Autoscaling can be set-up through the user of the scaling TOML.

Spot instances

Hydrolix supports the ability to deploy spot instances in replacement of on-demand. If this setting is enabled for a pool, the autoscale group will only request one reserved node (or 5% of the count if the count is above 30) and will attempt to request spot instances for the remaining requested capacity.

There are two methods to enable spot instances:

  • There is a hidden scaling option for each instance type that allows you to turn on the use of spot instances. For example, hdxctl scale --query-peer-spot true CLIENT_ID CLUSTER_ID, will turn on the use of spot instances for query peers

  • The setting is edited through the TOML settings using spot = true. An example is below:

[pool.query-peer0.instance]
service = "peer"
count = "1-3-5"
family = "c5n"
size = "9xlarge"
spot = true
disk = 30
cache_disk = 24

Did this page help you?