Since our SSO overhaul you might’ve been noticing sudden HTTP 500 errors while using the Alertmanager, Kubernetes of Prometheus dashboards when your token’s TTL expires.
Normally when your OIDC token expires, your session should automatically get refreshed with a new one. However this didn’t always work, throwing a 500 error, due to the nature of how oauth2_proxy is setup in our environments using cookies for session tracking. As stated in the documentation, since multiple requests can be made concurrently to the proxy, this implementation cannot lock the sessions and it can cause conflicts during an update or refresh, which forces users to re-authenticate.
We’re rolling out an updated oauth2_proxy using Redis as session store, effectively eliminating the issue.