Bottlerocket: Operating System to Run Containers

Containers are now widely used to package and scale applications. Using general-purpose operating systems as host operating systems brings some difficulties with security, overhead, and issues during updates. AWS designed a free, open-source, and Linux-based operating system called Bottlerocket to solve these problems. This blog post will cover the benefits of Bottlerocket, what problems it solves, how it is used with EKS and the update process.

Bottlerocket is a stripped-down Linux distribution with only essential software to run containers. OS updates are applied all at once and can be rolled back. Bottlerocket improves performance as it is lighter, increases security as it reduces the attack surface, improves uptime with update strategy and reduces management overhead and operational costs.

AWS-Provided builds come with three years of support and AWS support plans cover these builds. There is also community support on the Bottlerocket Github page.

Security

Bottlerocket improves security by reducing the attack surface with a minimal package set. In addition to this, it uses a read-only file system and it is checked at boot time with dm-verity.  Bottlerocket image has no SSH server and a shell to improve security, but there are options to use it as a typical Linux system. Bottlerocket has a control container and an administrative container that run outside of Bottlerocket’s container orchestrator in a separate instance of containerd.

Control container enabled by default and runs AWS SSM agent to run commands or start shell sessions on Bottlerocket instances on EC2.

Admin container is disabled by default. It has an SSH server to log in as ec2-user using the EC2-registered SSH key. It is useful when shell access to the underlying host is needed for development or troubleshooting scenarios because Bottlerocket doesn’t have a shell built into the OS for security reasons.

Updates

In general-purpose operating systems, update failures are common because of unrecoverable during package-by-package updates. Unlike general-purpose Linux distributions that include a package manager allowing you to update and install individual pieces of software, Bottlerocket downloads a full filesystem image and reboots into it. It can automatically rollback if boot failures occur, and workload failures can trigger manual rollbacks. This update method simplifies the update processes and makes it easier, faster, and safer to perform updates through automation.

Using a Bottlerocket AMI with Amazon EKS

In this part, I will set up an Amazon EKS cluster with Bottlerocket and update the OS.

This procedure requires eksctl version 0.74.0 or later. The version can be checked with the following command:

$ eksctl version
  

First, I will create a key pair.

$ aws ec2 create-key-pair --key-name bottlerocket --query 'bottlerocket' --output text > bottlerocket.pem
  

I will create a file named bottlerocket.yaml with the following content.

---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: bottlerocket
  region: eu-central-1
  version: '1.21'

iam:
  withOIDC: true

nodeGroups:
  - name: ng-bottlerocket
    instanceType: m5.large
    desiredCapacity: 3
    amiFamily: Bottlerocket
    ami: auto-ssm
    iam:
       attachPolicyARNs:
          - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
          - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
          - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
    ssh:
        allow: true
        publicKeyName: bottlerocket
    bottlerocket:
      settings:
        motd: "Hello from eksctl!"
      enableAdminContainer: true

  

I set the value for the amiFamily field to Bottlerocket and the ami field to auto-ssm so that eksctl automatically searches for the correct Bottlerocket AMI for the different regions.

Next, I will deploy the cluster with the following command.

$ eksctl create cluster --config-file=bottlerocket.yaml

It takes about 15 minutes to create the stacks in Cloudformation. You can check the progress of the stacks in the Cloudformation console:

https://console.aws.amazon.com/cloudformation/home 

When the above command is completed, I can check the cluster with the following command.

$ eksctl get cluster

I will use the following command to check the OS image.

$ kubectl get nodes -o=custom-columns=Name:.metadata.name,OS:.status.nodeInfo.osImage

Updating Bottlerocket 

First, I will need to connect to the admin container of my Bottlerocket node via SSH by using the following command:

$ ssh -i <YOUR_EC2_KEYPAIR_FILE>.pem ec2-user@<INSTANCE IP>
  

Replace “<YOUR_EC2_KEYPAIR_FILE>” with the name of your Keypair and the “<INSTANCE IP>” with the IP of your Bottlerocket instance.

I will use sheltie to drop into a root shell on your bottlerocket node.

[ec2-user@ip-192-168-33-55 ~]$ sudo sheltie

Now, I will check for updates.

bash-5.0# updog check-update
 

An update is available, I can initiate the update.

bash-5.0# updog update

This will download the new update image and update the boot flags so that when you reboot it will attempt to boot to the new version.

When that’s complete, I need to reboot.

bash-5.0# reboot
  

And that’s it. The node is now running the latest version of Bottlerocket OS.

Using the following commands I will check the version. As you can see we updated one of our nodes.

bash-5.0# cat /etc/os-release
  

$ kubectl get nodes -o=custom-columns=Name:.metadata.name,OS:.status.nodeInfo.osImage
  

Deleting Resources

Run the following command to delete resources.

$ eksctl delete cluster bottlerocket
  

Conclusion

In this blog post, you've seen what Bottlerocket is and its advantages over other Linux distributions. 

Bottlerocket is stripped down to decrease operational cost, reduce management complexity and improve security. Updates are applied as a single unit and can be rolled back as needed. So you don't have the risk of "botched" updates which can make the system unusable. Also, AWS gives support for AWS-provided builds.