EKS - How to annotate some nodes in USERDATA? - kubernetes

Question

To prevent Cluster Auto Scaler from terminating some of nodes, I would need to annotate them with:
cluster-autoscaler.kubernetes.io/scale-down-disabled=true;
Is there a way to do so in USERDATA script?
For labeling the nodes, there is no issue, and it is possible to do so via:
--kubelet-extra-args \
"--node-labels=
Thanks

Answer 1

No, it not possible.
The list of supported parameters for the bootstrap script:
--use-max-pods Sets --max-pods for the kubelet when true. (default: true)
--b64-cluster-ca The base64 encoded cluster CA content. Only valid when used with --apiserver-endpoint. Bypasses calling \"aws eks describe-cluster\"
--apiserver-endpoint The EKS cluster API Server endpoint. Only valid when used with --b64-cluster-ca. Bypasses calling \"aws eks describe-cluster\"
--kubelet-extra-args Extra arguments to add to the kubelet. Useful for adding labels or taints.
--enable-docker-bridge Restores the docker default bridge network. (default: false)
--aws-api-retry-attempts Number of retry attempts for AWS API call (DescribeCluster) (default: 3)
--docker-config-json The contents of the /etc/docker/daemon.json file. Useful if you want a custom config differing from the default one in the AMI

Answer 2

You can add node labels, taints, etc by using the --kubelet-extra-args option on the bootstrap.sh invokation as you guessed. For an example, see the AWS Blog post: Improvements for Amazon EKS Worker Node Provisioning
Use a USERDATA script similar to the following:
UserData: !Base64
"Fn::Sub": |
#!/bin/bash
set -o xtrace
/etc/eks/bootstrap.sh ${ClusterName} ${BootstrapArguments}
/opt/aws/bin/cfn-signal --exit-code $? \
--stack ${AWS::StackName} \
--resource NodeGroup \
--region ${AWS::Region}
The above is a fragment from the CloudFormation template. Of course you can make your script more complex, with security hardening, etc. if you so desire.
For a complete CloudFormation template, download the sample from AWS:
curl -O https://amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2019-11-15/amazon-eks-nodegroup.yaml

Answer 3

It is absolutely possible. Here is part of my example userdata, specifically useful if you want to run both OnDemand and Spot instance. In my example I am adding lifecycle node label which changes based on the type. See below:
--use-max-pods 'true' \
--kubelet-extra-args ' --node-labels=lifecycle=OnDemand \
--system-reserved cpu=250m,memory=0.2Gi,ephemeral-storage=1Gi \
--kube-reserved cpu=250m,memory=1Gi,ephemeral-storage=1Gi \
--eviction-hard memory.available<0.2Gi,nodefs.available<10% \
--event-qps 0'
I hope that gives you a nice example.