We have started rolling out AKS and EKS 1.21. This brings both our supported AKS and EKS platforms on Kubernetes v1.21.2.

Upon writing we have already upgraded all non-production clusters. Production clusters will follow in the next days after some extra validation.

Important changes between K8s 1.20 and 1.21

For more detailed info on what’s new and changed, please make sure to check the Kubernetes 1.21 release announcement and the full Kubernetes 1.21.x changelog.

Here’s a small list of some major themes:

• CronJobs graduate to stable!

  If you’re using CronJobs, you should migrate them to the apiVersion: batch/v1 spec.

• Immutable Secrets and ConfigMaps

  These resources introduce a new immutable field. When true this will reject any updates to these resources and comes with the following advantages:

  • Protection from accidental or unwanted updates that could cause applications outages
  • Reduces load on the K8s API server significantly, by closing watches for ConfigMaps/Secrets marked as immutable

• Graceful Node shutdown has graduated to beta.

  With this update, the kubelet will gracefully terminate Pods during a node shutdown. Before this update workload problems were possible during a node shutdown because Pods didn’t follow the normal, expected termination lifecycle.

• [PodSecurityPolicies] are now deprecated

  Currently there’s no replacement ready yet and current PSPs will still be functional for several more releases. To learn more, read PodSecurityPolicy Deprecation: Past, Present, and Future.

AKS specific changes

For AKS clusters, this update enables the CSI drivers for Azure Disk and Azure Files. The default Storage Class now uses standard SSD LRS storage through the CSI driver.

You can now use the following available Storage Classes:

• managed-csi or default: Uses Azure Standard SSD locally redundant storage (LRS) to create a managed disk.
• managed-csi-premium: Uses Azure Premium LRS to create a managed disk.
• azurefile-csi: Uses Azure Standard Storage to create an Azure Files share.
• azurefile-csi-premium: Uses Azure Premium Storage to create an Azure Files share.

Please refer to the Azure Disk CSI drivers and Azure Files CSI drivers documentation for more information.

EKS specific changes

In the process of upgrading EKS the following components have also been upgraded:

• KubeProxy from v1.20.4 to v1.21.2
• CoreDNS from v1.8.3 to v1.8.4
• Cluster Autoscaler from v1.20.0 to v1.21.0

Actions to take

This is the last version before support for the many deprecated API versions is dropped in Kubernetes 1.22! Make sure to check out the deprecated API migration guide for more details.

Below you will find more details on some of the most common deprecated resources used by our customers.

Deprecated API removal for Ingress

If you haven’t already, you need to migrate your Ingress resources to the new GA apiVersion of networking.k8s.io/v1!

Notable changes between networking.k8s.io/v1beta1 and networking.k8s.io/v1:

• spec.backend is renamed to spec.defaultBackend
• The backend serviceName field is renamed to service.name
• Numeric backend servicePort fields are renamed to service.port.number
• String backend servicePort fields are renamed to service.port.name
• pathType is now required for each specified path. Options are Prefix, Exact, and ImplementationSpecific. To match the undefined v1beta1 behavior, use ImplementationSpecific

The networking.k8s.io/v1 spec also comes with an ingressClassName field as replacement for the deprecated kubernetes.io/ingress.class annotation. By default, on our platform you can now set ingressClassName: nginx or ingressClassName: nginx-internal respectively.

For example, a simple Ingress resource would look like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/tls-acme: "true"
  name: marketing-website
  namespace: production
spec:
  ingressClassName: nginx
  rules:
    - host: www.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  name: http
  tls:
    - secretName: www-example-com-tls
      hosts:
        - www.example.com

Check out the upstream Ingress page for more examples.

Deprecated API removal for RBAC

Similar to Ingress, Kubernetes 1.22 will drop support for the rbac.authorization.k8s.io/v1beta1 API version. This impacts the resources: ClusterRole, ClusterRoleBinding, Role, and RoleBinding.

Please make sure to update the apiVersion of these resources to rbac.authorization.k8s.io/v1.