We have finished deploying our announced changes to the default indexed labels and structured metadata we assign to Loki logs. Please read on to know what changes have been made and how this impacts you. Please reach out if you have any questions, need guidance or if you would like extra, custom labels to be indexed.
What changed?
We no longer add all Kubernetes labels automatically as Loki stream labels, as they cloud include values with high cardinality, slowing down Loki performance.
Duplicates we had before are also be removed:
namespacewas removed in favor of the defaultnamespace_namelabel.
By default we provide the following index labels:
list
app
app_kubernetes_io_component
app_kubernetes_io_instance
app_kubernetes_io_name
app_kubernetes_io_part_of
component
container_image
container_name
cronjob
job
k8s_app
namespace_name
release
statefullset_kubernetews_io_pod_name
stream
\
Loki also automatically adds the service_name label, which takes the value of common “app” labels in this order:
list
service_name
service
app
application
name
app_kubernetes_io_name
container
container_name
component
workload
job
\
In conclusion: prefer using service_name in your queries to select logs from the correct application, regardless whether they internally have app or the newer app_kubernetes_io_name labels.
Finally we have moved the content of all Kubernetes labels now to Loki Structured Metadata. In other words, your logs still get enriched with useful metadata but this is not indexed and thus not affecting backend performance. We also include the pod_name and host_name as structured metadata.
Example usage of this: {job="kube", service_name="my-app"} | pod_name="my-app-abc1234-56789". Where the part between {} uses indexed labels and the part after | is structured metadata.
