Exporting Kubernetes Event Objects for Better Observability
Objects in Kubernetes, such as Pod, Deployment, Ingress, Service publish events to indicate status updates or problems. Most of the time, these events are overlooked and their 1 hour lifespan might ca …
Talk Title | Exporting Kubernetes Event Objects for Better Observability |
Speakers | Mustafa Akın (SRE, Atlassian), Ahmet Şeker (SRE, Atlassian) |
Conference | KubeCon + CloudNativeCon North America |
Conf Tag | |
Location | San Diego, CA, USA |
Date | Nov 15-21, 2019 |
URL | Talk Page |
Slides | Talk Slides |
Video | |
Objects in Kubernetes, such as Pod, Deployment, Ingress, Service publish events to indicate status updates or problems. Most of the time, these events are overlooked and their 1 hour lifespan might cause missing important updates. They are also not searchable and cannot be aggregated.We are open-sourcing our internal tool for publishing the events in Kubernetes to Opsgenie, Slack, Elasticsearch, Webhooks, Kinesis, Pub/Sub. It has a configuration language for matching events based on various criteria, such as the content and the related object’s labels. It also has the capability to route the events intelligently, inspired by Prometheus Alertmanager.For instance, you can notify an owner of Pod for runtime OCI failures, you can aggregate how many times the images are pulled, how many times container sandbox changes for various resource labels.