Standing on the shoulders of giants: Unleashing the power of scriptable load balancers

Once reserved for companies large enough to write a load balancer from scratch, load balancer middleware can be a powerful tool for scaling applications. Emil Stolarsky and Justin Li explain how Shopify uses scriptable load balancers to solve difficult infrastructure problems, such as sharding across data centers, handling flash sales, and responding quickly to DDoS attacks.

Every year, organizations continue adding more services. It’s unsustainable to have a dedicated team of SREs for each one, so the industry is moving to a product team SRE model. We’re now accustomed to building custom services that applications reach out to but not middleware services that operate on requests before they reach their destination. Load balancers have the potential to provide application-aware middleware without making changes to the application itself. However, traditional load balancers can’t be easily and deeply customized or redeployed quickly without significant risk. Instead, a scripting language must be embedded to fulfill these requirements. Emil Stolarsky and Justin Li explain how Shopify uses scriptable load balancers (with NGINX and LuaJIT via OpenResty) to solve difficult infrastructure problems. Its NGINX scripts deploy in 10 seconds, run through a thorough suite of automated tests, and have allowed Shopify to solve sharding across data centers, handle some of the world’s biggest flash sales, and respond quickly to layer 7 DDoS attacks. What once took a large team of engineers can now be accomplished by one of any size.