An EKS-optimized AMI with a custom kernel can be valuable when you require the AWS-tuned EKS base image for Kubernetes while also needing kernel-level optimizations or features not included in the default Amazon Linux kernel.

Note: Building an AMI this way is not ideal. Whenever possible, use automated image creation tools like Packer instead of manual modifications.

Creating an AMI with a Custom Kernel

First, identify the standard EKS-optimized AMI name for your target Kubernetes version. For example, Kubernetes v1.28 corresponds to amazon/amazon-eks-node-1.28-v20231106, which resolves in the AWS Console to AMI ID ami-06b72a2948fb82288. This AMI ships with Linux kernel 5.10, which can be upgraded to kernel 5.15.

Steps:

1. Launch an EC2 instance from ami-06b72a2948fb82288 and connect via SSH.

2. Verify that eksctl scripts are present.

   ls -al /etc/eks/bootstrap.sh
   ls -al /var/lib/cloud/scripts/eksctl/bootstrap.helper.sh
   
   sudo mkdir -p /var/lib/cloud/scripts/eksctl/
   sudo curl https://github.com/eksctl-io/eksctl/blob/main/pkg/nodebootstrap/assets/scripts/bootstrap.helper.sh -o /var/lib/cloud/scripts/eksctl/bootstrap.helper.sh
   sudo chmod +x /var/lib/cloud/scripts/eksctl/bootstrap.helper.sh

3. Upgrade kernel and cleanup leftovers

   # Check current kernel
   uname -r
   # Output: 5.10.198-187.748.amzn2.x86_64
   
   # Get available kernels
   sudo amazon-linux-extras | grep kernel
   # Enable newer kernel from Amazon RPM repo
   sudo amazon-linux-extras enable kernel-5.15
   # Disable old
   sudo amazon-linux-extras disable kernel-5.4
   sudo amazon-linux-extras disable kernel-5.10
   
   # See if kernel package locked
   sudo yum versionlock list | grep kernel
   # Output: 0:kernel-5.10.198-187.748.amzn2.*
   # Output: 0:kernel-headers-5.10.198-187.748.amzn2.*
   # Output: 0:kernel-devel-5.10.198-187.748.amzn2.*
   
   # Remove package locks
   sudo yum versionlock delete 0:kernel-5.10.198-187.748.amzn2.*
   sudo yum versionlock delete 0:kernel-headers-5.10.198-187.748.amzn2.*
   sudo yum versionlock delete 0:kernel-devel-5.10.198-187.748.amzn2.*
   
   # Upgrade kernel versions (this example: 5.10->5.15)
   sudo yum upgrade kernel kernel-devel kernel-headers
   
   # See installed kernels
   rpm -qa | grep kernel
   
   sudo reboot
   
   # Check new kernel
   uname -r
   
   # Cleanup leftovers
   sudo package-cleanup --leaves
   sudo package-cleanup --oldkernels
   
   # Cleanup old kernels
   sudo yum remove kernel-5.10* kernel-devel-5.10*
   
   sudo yum clean metadata
   sudo yum clean all
   
   # Clean history
   history -c
   history -w

4. Stop the instance and create AMI image

Use custom AMI in unmanaged nodepool

Below is the example of eksctl that uses unmanaged nodepool

...<line break>...

  - name: custom_image
    amiFamily: AmazonLinux2
    ami: ami-12345 # custom ami with kernel-5.15 build on ami-06b72a2948fb82288
    overrideBootstrapCommand: |
      #!/bin/bash
      echo "eksctl relies on a specific set of labels to be on the node, so it can find them."

      source /var/lib/cloud/scripts/eksctl/bootstrap.helper.sh

      # Note "--node-labels=${NODE_LABELS}" needs the above helper sourced to work, otherwise will have to be defined manually.
      /etc/eks/bootstrap.sh ${CLUSTER_NAME} --container-runtime containerd --kubelet-extra-args "--node-labels=${NODE_LABELS}"

    desiredCapacity: 1
    minSize: 0
    maxSize: 40
    instancesDistribution:
      instanceTypes
        - c5.4xlarge    # 32.0 GiB 16 vCPUs
        - c6a.4xlarge   # 32.0 GiB 16 vCPUs
        - c6i.4xlarge   # 32.0 GiB 16 vCPUs
        - m5a.4xlarge   # 64.0 GiB 16 vCPUs
      onDemandBaseCapacity: 0
      onDemandPercentageAboveBaseCapacity: 0
      spotAllocationStrategy: "price-capacity-optimized"
    tags:
      'k8s.io/cluster-autoscaler/node-template/label/role': 'role_for_nodeselector'
      'tag_name': 'tag_value'
    labels:
      role: role
    volumeSize: 100
    privateNetworking: true
    securityGroups:
      attachIDs:
        - sg-12345
    iam:
      withAddonPolicies:
        externalDNS: true
        certManager: true
        awsLoadBalancerController: true
        imageBuilder: true
      attachPolicy:
        Statement:
          - Effect: Allow
            Action:
              - "s3:ListBucket"
            Resource:
              - "arn:aws:s3:::prefix-*"
          - Effect: Allow
            Action:
              - "s3:*Object"
            Resource:
              - "arn:aws:s3:::prefix-*/*"
    ssh:
      allow: true
      publicKeyPath: ~/.ssh/key_rsa.pub

...<line break>...

References

• https://serverfault.com/questions/1048470/how-to-get-the-latest-kernel-package-on-aws-ec2
• https://repost.aws/knowledge-center/amazon-linux-2-kernel-upgrade
• https://docs.aws.amazon.com/imagebuilder/latest/userguide/security-best-practices.html