Kubernetes in the Datacenter: Squarespaces Journey Towards Self-Service Infrastructure [I]

As Squarespaces engineering organization evolved, microservices became an obvious solution to quickly deliver new features and improve infrastructure reliability. We encountered significant challenge …

As Squarespace’s engineering organization evolved, microservices became an obvious solution to quickly deliver new features and improve infrastructure reliability. We encountered significant challenges in our transition to a microservice-based architecture. Each new service increased the operations burden to provision and maintain a growing fleet of servers, frequently slowing the process of adding new services and scaling existing services in our datacenters. I’ll discuss how we used Kubernetes to containerize our microservice ecosystem and solve those challenges. To effectively work with ephemeral Kubernetes pods, we replaced Graphite with Prometheus and Sensu with AlertManager to monitor service health rather than individual instances. We discovered massive performance issues containerizing our Java services and worked around JVM complexities. To ease our transition from virtualization to containerization, services running inside and outside of Kubernetes must seamlessly discover each other with Consul and communicate with each other. Thanks to Calico, BGP, and our Leaf-Spine Layer 3 network topology, we efficiently route pod network traffic with the rest of our network.