AutoScaling on EKS

We need to install the metric server to enable horizontal pod autoscaling

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Create a new policy to enable autoscaling nodegroups

read -r -d '' AUTOSCALER_POLICY_DOC << EOF
{   
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/k8s.io/cluster-autoscaler/ci": "owned"
                }
            }
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:DescribeAutoScalingGroups",
                "ec2:DescribeLaunchTemplateVersions",
                "autoscaling:DescribeTags",
                "autoscaling:DescribeLaunchConfigurations"
            ],
            "Resource": "*"
        }
    ]
}
EOF

aws iam create-policy \
    --policy-name eks-${KUBERNETES_CLUSTER}-autoscaler \
    --policy-document "$AUTOSCALER_POLICY_DOC"

Trust the service account autoscaler into our K8s cluster:

read -r -d '' AUTOSCALER_TRUST_DOC << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/$OIDC_PROVIDER"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "$OIDC_PROVIDER:aud": "sts.amazonaws.com",
          "$OIDC_PROVIDER:sub": "system:serviceaccount:kube-system:cluster-autoscaler"
        }
      }
    }
  ]
}
EOF

Create a new role using the previously created policy:

aws iam create-role --role-name "eks-${KUBERNETES_CLUSTER}-autoscaler" \
    --assume-role-policy-document "$AUTOSCALER_TRUST_DOC" \
    --description "eks-${KUBERNETES_CLUSTER}-autoscaler"

Attach the new role into our K8s cluster:

aws iam attach-role-policy --role-name "eks-${KUBERNETES_CLUSTER}-autoscaler" \
--policy-arn="arn:aws:iam::$AWS_ACCOUNT_ID:policy/eks-${KUBERNETES_CLUSTER}-autoscaler"

Finally we need to download and modify the cluster-autoscaler-autodiscover.yaml:

curl -o cluster-autoscaler-autodiscover.yaml https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

Modify the line containing <YOUR CLUSTER NAME> (currently line 163) and replace it with your cluster name $KUBERNETES_CLUSTER
Apply this configuration to your K8s cluster:

kubectl apply -f cluster-autoscaler-autodiscover.yaml

Finally annotate your deployment to enable cluster autoscaler:

kubectl annotate serviceaccount cluster-autoscaler -n kube-system \
eks.amazonaws.com/role-arn=arn:aws:iam::${AWS_ACCOUNT_ID}:role/eks-${KUBERNETES_CLUSTER}-autoscaler

You are now ready to deploy Hydrolix to your cluster !