Kubernetes in the cloud: AKS vs EKS

AWS EKS and Azure AKS are two major managed Kubernetes Services. As organizations continue their cloud transformation journeys, these platforms serve as the foundation for countless enterprise workloads worldwide. While both cloud providers offer wide service portfolios, their managed Kubernetes offerings have become critical for modern applications.

Whether you are an Azure or AWS customer, or simply evaluating your options for managed Kubernetes, this blogpost aims to show how those providers approach aspects like version management, networking, and other key differences between the two cloud platforms.

To help you easily compare these two managed Kubernetes services, I have included a comparison table at the end of this post.

Let’s dive in!

Kubernetes Version Management

Kubernetes releases currently happen approximately three times per year and Kubernetes project maintains release branches for the most recent three minor releases. Each Kubernetes minor version receives approximately one year of patch support.

Both AKS and EKS, as downstream distributions, have their own timelines for version adoption and support policies.

Version Release Timeline

AKS

• After a new minor version is released from upstream, it generally becomes available on Azure Preview within two months and then achieves General Availability (GA) approximately one month later after preview release. Basically it takes 3 months for a new upstream version to become generally available.

EKS

• After a new minor version is released from upstream, it generally becomes available on EKS within one month. There is no preview release in EKS.
  
  Here is the version release timeline for the latest 5 releases for EKS and AKS. Those are the dates of GA version availability.

Version Support Policy

AKS

• AKS supports the latest and two previous minor versions. For the last 5 Kubernetes versions it’s 12 months of support.(Each supported minor version also supports a maximum of two stable patches.)
• LTS version is also an option which supports one version at 2 years support. (Premium tier only)

EKS

• EKS offers standard support for at least four production-ready versions of Kubernetes at any given time.
• A minor version is under standard support in Amazon EKS for the first 14 months after it's released.
• Once a version is past the end of standard support date, it automatically enters extended support for the next 12 months. (incurs additional cost per cluster hour.)

Node Management

Node Auto Repair

Both services have node level health checks. Auto Repair checks the status of Kubernetes related processes. (i.e kubelet)

AKS

• AKS continuously monitors the health state of worker nodes and performs automatic node repair if they become unhealthy. This feature is available by default.
• If the node reports NotReady status for 10 minutes of timeframe or node doesn’t report status within 10 minutes, AKS first attempt to restart node, if restart is unsuccessful it reimages the node. If reimage is unsuccessful it redeploys the node.

EKS

• Node Health monitoring and Node Auto repair for EKS is a built in feature on AWS from December 2024

• Continuously monitors the health of the nodes within an EKS cluster, automatically detecting health issues and replacing nodes with issues when they arise.
• It can be activated via installing EKS node monitoring agent add-on.

Node Limit

If you have a huge cluster, it might be at risk of hitting the limits, here are the limits for maximum number of nodes per node group and per cluster.

AKS

• 100 managed node groups per cluster (soft limit)
• 1000 nodes per node group (soft limit)
• Total cluster limit of 5000 nodes (hard limit)
• Soft limits adjustable via support request

EKS

• 30 managed node groups per cluster (soft limit)
• 450 nodes per node group (soft limit)
• Limits adjustable via support request
• Support for Fargate pods (serverless)

Stop/Start Clusters

Stopping and starting clusters helps manage costs and resources, especially when 24/7 operation isn't necessary.

AKS

• On AKS, clusters can be stopped and started using the Azure portal, Azure CLI, or SDKs. Stopping the cluster preserves all state, ensuring that it resumes as before when restarted up to 12 months. Charges for resources like storage and static IPs continue while the cluster is stopped.

EKS

• EKS doesn't have a built-in feature to stop and start the entire cluster. You can scale down worker nodes to zero, but control plane charges continue. To avoid charges, the cluster must be deleted and recreated.

Storage Options

AKS

• Each AKS cluster includes the following pre-created storage classes by default. Blob, File, Disk storage types can be used in AKS.

EKS

• In EKS, CSI drivers aren't installed by default, you need to create IAM permissions for CSI drivers and then install drivers via add-on. CSI drivers exist for EBS, EFS, S3. After installing drivers, you can take advantage of persistent volumes.

For both services Local (Ephemeral) disks can be mounted to containers with local disk provisioners.

Pricing

In this section, we'll compare the pricing models for EKS and AKS, including the costs for cluster management and a sample calculation for a 3-node cluster over the course of a month. Disk, network and monitoring costs are not included to keep it simple.

EKS

• AWS charges $0.10 per hour for cluster management. For the Kubernetes version that is covered by extended support(14 months later after version released), it costs $0.60 per hour.
• For a cluster with 3 nodes(m7i.xlarge - 4 vCPU 16GiB RAM) under standard support, it costs $147 for nodes, $72 for cluster management, totaling $219 per month.

AKS

• AKS has a free tier with no management costs, while the standard tier charges $0.10 per hour for cluster management. The premium tier, which includes enhanced SLAs and long-term support, costs $0.60 per hour.
• For a cluster with 3 nodes (Standard_D4_v5 - 4 vCPUs, 16 GiB RAM), the cost          breakdown for AKS is as follows:
  • Free Tier: No management costs; only the cost of the nodes, which is approximately $140.16 per month for the 3 nodes.
  • Standard Tier: The cost of the nodes is still $140.16 per month, with an additional $72 per month for cluster management (charged at $0.10 per hour), totaling $212.16 per month.
  • Premium Tier: The cost of the nodes is $140.16 per month, and the premium cluster management costs $432 per month (charged at $0.60 per hour), bringing the total to $572.16 per month.

Service Level Agreements (SLA)

AKS

EKS

Identity and Access Management

Identity and Access Management plays a key role in controlling who can access your resources and what they can do. In both AKS and EKS, Kubernetes native Role-Based Access Control (RBAC) is in place, ensuring that access rules fit well with Kubernetes itself.

AKS 

• Single sign-on integration with Active Directory allows seamless cluster access for users.
• Azure RBAC support using Microsoft Entra ID enables precise permission management across the cluster.
• Managed identities for secure cluster access authorize resources without sharing credentials.
• Administrators can create built-in and custom roles through the Azure Portal for flexible access control.
• Pod-managed identities facilitate direct access to Azure resources using Microsoft Entra ID.
• The platform provides a unified access control approach across Azure resources and Kubernetes infrastructure.
  
  

EKS 

• IAM roles for service accounts (IRSA) create a robust authentication mechanism for Kubernetes clusters.
• EKS enables mapping between AWS IAM users and Kubernetes RBAC roles for comprehensive access management.
• Fine-grained access control is achieved through detailed AWS IAM policies.
• Pod identity management allows services to securely authenticate and access resources.
• The service supports secure service account authentication across the Kubernetes environment.
• EKS provides granular permission management that ensures precise access rights for different user roles.

Cluster Networking

AKS

There are two networking models available in AKS, kubenet and Azure CNI.There are additional evolutions of Azure CNI, including Azure CNI Overlay, Azure CNI for dynamic IP allocation and Azure CNI Powered by Cilium. 

You can also choose from other CNI providers like  Flannel, but those are not natively supported by AKS.

EKS

The Amazon VPC CNI plugin for Kubernetes is the only CNI plugin supported by Amazon EKS. Amazon EKS runs upstream Kubernetes, so you can install alternate compatible CNI plugins to Amazon EC2 nodes in your cluster, same with AKS.

Monitoring and Logging

AKS

• Native integration with Azure Monitor helps track resource and performance details.
• Container Insights is enabled by default, offering container health monitoring.
• Log Analytics workspace helps look into log data.
• Azure Application Insights shows how applications are working.
• Azure Managed Prometheus helps collect and check metrics.
• Azure Managed Grafana visualizes Kubernetes data.

EKS

• Amazon Managed Service for Prometheus & Grafana supports effective monitoring of containerized applications.
• Can integrate with CloudWatch Logs, for worker nodes Cloudwatch Agent needs to be installed, for control plane there is no need for installation of Cloudwatch Agent.
• Fluentbit and Fluentd is also supported for sending container logs to CloudWatch Logs.

Cluster Provisioning Time

To measure how much time it takes to have an operational cluster from scratch, clusters were provisioned via the user interfaces (UIs) of both platforms. 

AKS

• The cluster was provisioned with a node pool, and it took 4 minutes and 46 seconds for the cluster to become fully operational in the US East region. As a node type Standard_D4_v5 has been selected. (4 vCPU, 16 GB RAM)

EKS

• In EKS you need to create a node pool separately, it took 9 minutes for EKS to be available, then 4 minutes 7 seconds for creating a node group and adding to the cluster. In total, it took approximately 13 minutes for the EKS cluster to be fully operational in the US-east-1 region. As a node type m7i.xlarge has been selected.(4 vCPU, 16 GB RAM)

Backup 

AKS

• AKS has a native backup mechanism that users can take scheduled or on-demand backups for cluster state and application data. In order to use the backup feature, backup extension needs to be installed inside the AKS Cluster.

EKS

• EKS does not have a native backup solution, but external solutions like Velero can be implemented. With using Velero it’s possible to take volume snapshots and store cluster state in s3 buckets.

Auto Mode

Recently both providers have announced auto-mode for Kubernetes clusters (Azure calls this feature “automatic mode” while AWS calls it “auto mode”). Which simplifies Kubernetes cluster management.There are more features covered by auto mode, some of the key features will be focused on. 

AKS

• Automatic mode is currently in preview.
• In automatic mode, node provisioning based on workload requests is preconfigured.
• Clusters are automatically upgraded to the new versions. If any deprecated Kubernetes API usage exists, cluster upgrade stops.
• As a CNI, Azure CNI Overlay powered by Cillium is preconfigured.
• Deployment safeguards are in place preconfigured via Azure Policy to ensure best practices are applied such as container images can not use the latest image tag or cluster containers have readiness and liveness probes.

EKS

• In auto mode, node scaling is preconfigured with leveraging Karpenter.
• Security patching, operating system and cluster component upgrades are automatic. If any deprecated Kubernetes API usage exists, cluster upgrade stops.
• Also it enforces 21 days of maximum lifetime of a node to ensure up-to-date software and API’s
• Direct ssh or ssm access to nodes is prevented.
• Some features enabled built-in such as EBS CSI driver, GPU plugins, that would otherwise have to be managed as add-ons.

Conclusion

Modern cloud-native apps can be built on top of either of the two successful feature-rich managed Kubernetes services, AKS or EKS. Organizations need to carefully evaluate their specific requirements, cloud strategy, team expertise, and other aspects when deciding which service to use.

Support for Kubernetes versions and upgrade schedules, networking capabilities, integrations for monitoring and logging, backup and disaster recovery features, and pricing/SLA models are a few important factors to take into account. You can make an informed choice that positions your teams and applications for long-term success by being aware of the distinct capabilities in each of these areas.

In the end, both AKS and EKS are extremely powerful Kubernetes platforms that can help businesses move more quickly toward cloud nativeness. Making the best decision will require matching the capabilities of the services to the unique requirements and limitations of your company.

Comparison Table

References:

• Supported Kubernetes versions in Azure Kubernetes Service (AKS). - Azure Kubernetes Service | Microsoft Learn
• Understand the Kubernetes version lifecycle on EKS - Amazon EKS
• https://www.cncf.io/blog/2023/09/28/kubernetes-security-best-practices-for-kubernetes-secrets-management
• Use AWS Secrets Manager secrets in Amazon Elastic Kubernetes Service
• GitHub - Azure/karpenter-provider-azure
• Extending the EKS API: Managed Node Groups | Containers
• Access Amazon S3 objects with Mountpoint for Amazon S3 CSI driver - Amazon EKS
• Create and use a volume with Azure Blob storage in Azure Kubernetes Service (AKS)
• Azure Kubernetes Service (AKS) node auto-repair
• Data Plane - EKS Best Practices Guides
• Storage options for a Kubernetes cluster - Azure Architecture Center | Microsoft Learn
• View and manage Amazon EKS and Fargate service quotas
• Releases | Kubernetes
• What is Azure Kubernetes Service (AKS) backup?
• Logging and monitoring on Amazon EKS - AWS Prescriptive Guidance
• Use virtual nodes - Azure Kubernetes Service | Microsoft Learn
• Simplify compute management with AWS Fargate - Amazon EKS