Automated AWS ELasticsearch Service backups to S3

By default, the AWS Elasticsearch Service already comes with regular automated snapshots. However these snapshots can not be used for recovery or migration to a new Elasticsearch cluster and furthermore can only be accessed as long as the Elasticsearch API of the cluster is available.

To mitigate situations where Elasticsearch backups wouldn’t be available, we have extended our awselasticsearch Terraform module (open source) for creating regular snapshots to an S3 bucket. This extension will create an S3 bucket for storing the snapshots, a Lambda function and all required resources to automatically:

  • Register the S3 bucket as snapshot repository (s3-manual) in Elasticsearch
  • Delete (automated) snapshots in this repo that are older than s3_snapshots_retention
  • Create a new snapshot in this repo with name automatic-<datetime>