A note on Let's Encrypt chain issues due to DST Root CA X3 expiry

Let’s Encrypt certificates are (usually) cross-signed with the DST Root CA X3 root certificate, however this root certificate expired on September 30th 2021.

From the upstream Let’s Encrypt documentation on the DST Root CA X3 Expiration:

Let’s Encrypt has a “root certificate” called ISRG Root X1. Modern browsers and devices trust the Let’s Encrypt certificate installed on your website because they include ISRG Root X1 in their list of root certificates. To make sure the certificates we issue are trusted on older devices, we also have a “cross-signature” from an older root certificate: DST Root CA X3.

Generally speaking the expiry of this root certificate shouldn’t cause any troubles, however there’s some cases reported with our customers with (IoT) clients using OpenSSL 1.0.x which can’t be updated.

In OpenSSL 1.0.x, a quirk in certificate verification means that even clients that trust ISRG Root X1 will fail when presented with the Android-compatible certificate chain we are recommending by default.

By default our letsencrypt-prod ClusterIssuer we configure on our managed Kubernetes clusters will prefer this chain with extended Android device compatibility.

If you’re using some clients with OpenSSL 1.0.x but with the ISRG Root X1 certificate in their trust store, you can do the following to offer the alternate chain. Create a new cert-manager (Cluster)Issuer with the preferredChain option. For example:

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-isrgx1
spec:
  acme:
    email: an@email.com
    server: https://acme-v02.api.letsencrypt.org/directory
    preferredChain: "ISRG Root X1"
    solvers:
    - http01:
        ingress:
          class: nginx
      selector:
        matchLabels:
          use-http-solver: "true"
    - dns01:
        route53:
          region: eu-west-1
          secretAccessKeySecretRef:
            name: ""

Then add the cert-manager.io/issuer: letsencrypt-isrgx1 annotation to your Ingress for which you want to update the preferred chain.