Grafana security patch following High Severity CVE-2021-39226
On the 5th of October a notice for CVE-2021-39226 with a severity of high went out, impacting the Grafana deployments.
More …RDS snapshots cross-account replication available
In some cases, a disaster recovery plan might require RDS snapshots to be replicated / copied over to a different AWS account and region. We can now set up this replication process for the managed RDS instances of our customers. Note that this will work in conjunction of the normal automated daily RDS snapshots that AWS already performs.
More …Upgraded cluster add-ons
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. We’ve already rolled these out to all clusters.
More …Guaranteed QoS for all critical system and infrastructure Pods
We’ve seen in multiple occasions that, due to resource starvation in a cluster, the kubelet starts evicting critical infrastructure Pods. This can lead to important downtimes and disruptions in multiple occasions.
More …Improved EC2 instance interruption notifications
We’ve improved the looks and the content of the EC2 instance interruption notifications that we receive in Slack.
More …Upgrade AKS and EKS clusters to 1.21. Actions to take!
We have started rolling out AKS and EKS 1.21. This brings both our supported AKS and EKS platforms on Kubernetes v1.21.2.
Istio upgraded to version 1.11.2
We have upgraded Istio on all clusters that use it. The version was upgraded from 1.10.0 to 1.11.2. The new version comes with some features meant for operators and no breaking changes that you should be concerned of.
More …Mute critical KubeAPIErrorBudgetBurn alerts
We have muted the critical KubeAPIErrorBudgetBurn alerts.
More …VPA enabled by default
During the last year we have tested out the Vertical Pod Autoscaler on several of our workloads and customers. These results were positive and therefore we decided to roll out the VPA on all our clusters.
More …Downgraded Grafana to v7.5
Last month we rolled out a major Grafana update, going from 7.5 to 8.1. While initially everything looked in order, some customers experienced issues with their custom dashboards which were working perfectly in the previous release. Mainly data coming from SQL data sources, visualized through the “old” graph panel, is sorted differently or got visualized completely wrong.
More …Cert-manager upgraded to 1.4.4
We’ve upgraded Cert-manager to the version 1.4.4 on all our Kubernetes clusters. This patch upgrade contains a bug-fix for a renewal time issue that affected some of our clusters.
More …Vault upgraded to 1.8.2
We are in the process of upgrading our Kubernetes based Vault setups to the latest version 1.8.2.
Vault upgraded to 1.8.1
We are in the process of upgrading our Kubernetes based Vault setups to the latest version 1.8.1.
Upgraded cluster components & increased Pod density. Actions to take!
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. We’ve already rolled these out to all clusters.
More …Reduced memory usage of Cluster Autoscaler
In an effort to optmize as much as possible the resources being used by the infrastrucutre components running on our Reference Solution Kubernetes platforms, we’ve considerably reduced the memory used by the Cluster Autoscaler by optimizing its configuration. This means that the autoscaler will now run more reliably, and that there’ll be a bit more memory available for other workloads running on the K8s clusters.
More …Upgraded Teleport to version 6.2.8
We’ve upgraded all Teleport clusters to version 6.2.8. Coming from version 4.x, this is a (double) major release, coming with many new features:
More …Concourse upgraded to v7.3.2
We have upgraded our Concourse setups to the latest version 7.3.2.
More …Using encryption at rest for Prometheus and Alertmanager
We’re switching to encrypted volumes for all our Prometheus and Alertmanager set ups. These were the last of our managed infrastructure components to receive encryption at rest.
More …Pod Disruption Budget for CoreDNS
In order to improve the resilience of CoreDNS during upgrades, we have added a Pod Disruption Budget for the CoreDNS pods. The CoreDNS deployment already has a proper update strategy and anti affinity applied, however the extra PDB will prevent the posibility where there are no available CoreDNS pods running in the cluster during a rolling upgrade.
Using Telepresence in our Reference Solution
Telepresence is an open source tool that allows cluster users and operators to access cluster services and resources as if they were running in a local network. It also allows a developers to debug Kubernetes services locally and run local services as if they were on the cluster network. You can read more on how it works and what it can do in its documentation.
More …