We discovered an issue with our Elasticsearch monitoring for Prometheus that was introduced a while back in a rutinary chart upgrade. Because of this problem some Elasticsearch metrics were not being reported into Prometheus, like available storage space for example, and as a result there were some problematic situations in an Elasticsearch cluster that we didn’t pick up in time.
More …Certain AWS EC2 instances come with fast local NVMe Instance Storage and can usually be recognized with the d suffix (eg. m5d.large). Our platform will now automatically mount these volumes under the /ephemeralX paths (eg. /ephemeral0, /ephemeral1, …).
As of today we have the option to only allow access to the Kubernetes API over a private network connection.
More …We added a new docker image called concourse-ecr-login. This image has the AWS cli and docker installed.
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. We’ve already rolled these out to all clusters.
More …We’re releasing an optional feature where it’s possible to assign Elastic IPs to nodepools. This can be useful for whitelisting purposes. For example when nodes, running in public subnets, need to connect to services behind enterprise firewalls.
More …We’ve recently improved in the automation of our Kubernetes reference solution in several ways. Automation is very important for us to be able to quickly deliver new features and bug fixes to our customers, while performing seamless rollouts that cause no disruption to running workloads.
More …By default we integrated our Grafana deployments with Dex for handling authentication. We’ve now also made it possible to configure any other supported Grafana authentication backend, for example to use Azure AD directly.
By default, the AWS Elasticsearch Service already comes with regular automated snapshots. However these snapshots can not be used for recovery or migration to a new Elasticsearch cluster and furthermore can only be accessed as long as the Elasticsearch API of the cluster is available.
More …Our reference solution eks-based Velero backups on AWS S3 now supports automatic replication to an additional S3 bucket on an AWS region of choice. The feature is disabled by default, contact your lead engineer to discuss about enabling it for your cluster(s) if needed.
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. These updates are being rolled out to all clusters and will be finished by the end of the week.
More …We have upgraded our Concourse setups to the latest version 6.7.3.
More …We have updated our Kubernetes based Vault setups to the latest version 1.6.1 in the past days.
We have fixed a problem in our main Grafana dashboard. Previously we counted the resources for all pods (including those who already completed). This gave an incorrect indication on the cluster usage. Now we filter out the failed and succeeded pods so the dashboard indicates a more correct usage of the cluster.
More …We’ve upgraded Dex on all our clusters with a new version (v2.27.0) that contains a fix for a security vulnerability.
More …The Nginx Ingress Controller has configuration options for setting a default TLS certificate and a default backend. We have now exposed this settings to be configurable on a per-cluster basis.
More …We already offer a cost-effective and default logging solution based on Grafana Loki. However we realize this logging solution is not a perfect fit for everybody and thus now also allow deploying and configuring Fluent Bit through our reference solution offering, including all the benefits like regular updates.
More …We’ve updated all Teleport clusters to version 4.4.5. This is a minor release, which includes mostly bugfixes and small improvements.
More …We have started rolling out EKS 1.18. This brings EKS on Kubernetes v1.18.9.
For increased security, our Vault setups are configured to terminate TLS sessions directly at the Vault server process. To do so, we use cert-manager to provision LetsEncrypt certificates that the Vault server Pods can use. There was an issue with this setup, where the Vault servers didn’t reload the certificate when this was renewed by cert-manager, rendering Vault insecure / unavailable.
More …