Kubernetes Ingress Controller with Apache Traffic Server [I]

Today, the Oath Media Brands and Products container platform is serving critical application workloads like Yahoo Sports and Yahoo Finance at a large scale using Kubernetes as the orchestration framew …

Today, the Oath Media Brands and Products container platform is serving critical application workloads like Yahoo Sports and Yahoo Finance at a large scale using Kubernetes as the orchestration framework. For a platform at this scale, it is critical to have a powerful and flexible ingress routing layer (controller) that is able to handle the dynamic behavior of container based applications, such as auto-scaling, frequently changing pod IP addresses, self-serve onboarding and cluster-aware routing. This L7 routing layer must be quick to react to changes on the cluster without affecting its routing capabilities and impacting the in-flight requests. In a multi-tenant system it is even more vital that a single application deployment does not cause an impact to user traffic or hinder the release velocity of other tenants. We developed an ingress controller based on Apache Traffic Server that satisfies the requirements stated above, while remaining scalable and easy to integrate with both Kubernetes and the Oath ecosystem. In this talk/presentation, we will elaborate on the architecture of the ingress controller, the performance metrics we’ve achieved, and the key learnings from supporting such a critical infrastructure component.