We have identified a problem where our Concourse workers went unavailable. It turned out that the root EBS volume was filling up with logs too fast for the logrotation to take care of the archiving.
We patched the workers to compress the old logs after rotating to free up disk space. This way we keep the 4 weeks of logs without the need to increase the disk volume.
All environments have been patched.