Update 2023-02-10: All clusters have been upgraded to v1.24.

We have started rolling out AKS and EKS 1.24. This brings our supported AKS platforms to v1.24.6 and EKS to v1.24.8.

Upon writing we have already upgraded all non-production clusters. Production clusters will follow next week after extra validation. This changelog will be updated once all upgrades have finished rolling out.

Important changes between K8s 1.23 and 1.24

For more detailed info on what’s new and changed, please make sure to check the Kubernetes 1.24 release announcement and the full Kubernetes 1.24.x changelog.

Here’s a small list of some major themes:

• Dockershim removal from kubelet
  • This was deprecated since v1.20 and has now been completely removed in v1.24. All clusters are using containerd as engine. This is mostly a transparent change and you shouldn’t notice any changes with your workloads. However we’ll get in touch if we detected you mount the docker socket in any workload (see actions to take).
• Beta APIs are no longer enabled by default
  • K8s will no longer enable new beta features by default. Existing APIs will continue working however.
• Volume expansion is now stable
  • This allows resizing existing persistent volumes through its CSI driver

AKS specific changes

Azure AKS 1.24 GA release notes: https://github.com/Azure/AKS/releases/tag/2022-08-14

• Default to kubelogin for AAD clusters. The azure cli has been ouputting a warning regarding this for the past releases, so normally you should already have kubelogin installed by now:

  The default format of clusterUser credential for AAD enabled clusters will be ‘exec’, which requires kubelogin binary in the execution PATH. If you are using Azure CLI, it will prompt users to download kubelogin. There will be no behavior change for non-AAD clusters, or AAD clusters whose version is older than 1.24. Existing downloaded kubeconfig will still work. We provide an optional query parameter ‘format’ when getting clusterUser credential to overwrite the default behavior change, you can explicitly specify format to ‘azure’ to get old format kubeconfig.

• Behavioral change for LoadBalancer Services:

  For Kubernetes 1.24+ the services of type LoadBalancer with appProtocol HTTP/HTTPS will switch to use HTTP/HTTPS as health probe protocol (while before v1.24.0 it uses TCP). And / will be used as the default health probe request path. If your service doesn’t respond 200 for /, please ensure you’re setting the service annotation service.beta.kubernetes.io/port_{port}_health-probe_request-path or service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path (applies to all ports) with the correct request path to avoid service breakage.

EKS specific changes

In the process of upgrading EKS the following components have also been upgraded:

• KubeProxy to v1.24.9
• Cluster Autoscaler to v1.24.0
• AWS VPC CNI to v1.12.1

Actions to take

It is recommended to migrate your HorizontalPodAutoscaler resources to the stable autoscaling/v2 API.

Kubernetes stopped supporting dockershim in v1.20 and it has now been removed in v1.24. AWS EKS AMIs have containerd as the only available runtime (this was already the case for AKS). There’s a kubectl plugin available to detect whether you have workloads mounting the Docker socket volume. For more info, check this AWS page. We also performed this scan, and if mounting of the docker socket was detected, each customer has been notified through GitHub issues.

PodSecurityPolicies (PSP) have been deprecated for a while and are scheduled for removal in Kubernetes 1.25. PSPs are replaced with Pod Security Admission (PSA). You are encouraged to move to using PSAs if you were relying on these PSPs before.