Global Container Networks on Kubernetes at DigitalOcean

Building a container network that is reliable, fast and easy to operate has become increasingly important in DigitalOceans distributed systems running on Kubernetes. Todays container networking tech …

Building a container network that is reliable, fast and easy to operate has become increasingly important in DigitalOcean’s distributed systems running on Kubernetes. Today’s container networking technologies can be restrictive as Pod and Service IPs are not reachable externally which forces cluster administrators to operate load balancers. The addition of load balancers introduces new points of failure in a cluster and hinders observability since source IPs are either NAT’d or masqueraded. This talk will be a deep dive of how DigitalOcean uses BGP, Anycast and a variety of open source technologies (kube-router, CNI, etc) to achieve a fast and reliable container network where Pod and Service IPs are reachable from anywhere on DigitalOcean’s global network. Design considerations for scalability, lessons learned in production and advanced use cases will also be discussed.