Cost-effective Compute Clusters with Spot and Pre-emptible Instances [I]

Kubernetes and Spot/Pre-emptible Instances (SPIs) are arguably a match made in heaven. Traditionally, the uncertainty of SPIs (they can be terminated at any time due to price fluctuations) have made m …

Kubernetes and Spot/Pre-emptible Instances (SPIs) are arguably a match made in heaven. Traditionally, the uncertainty of SPIs (they can be terminated at any time due to price fluctuations) have made managing them tricky, and restricted them to specific workloads and use cases. Kubernetes, in contrast, not only handles node failure very well, it has trained developers and architects to design applications to tolerate and even embrace failure. The prospect of Kubernetes abstracting the complexities of SPIs is now a reality, enabling applications to take advantage of low-cost compute across different clouds and possibly vendors. The purpose of this talk is to educate the audience on strategies for making the most out of this powerful combination. Specifically, we will discuss these topics: 1. What are spot bidding strategies, and what is their cost vs. predictability trade-off? 2. What class of Kubernetes applications would benefit the most from SPIs? 3. Available Kubernetes mechanisms (e.g taints/tolerations, affinity, availability zones) for placing applications based on their tolerance with SPIs 3. Implementation strategies (e.g. blending multiple autoscaling groups to satisfy both SPI-optimized applications vs. applications that are more mission-critical or stateful) 4. What out-of-the box solutions exist, either free or commercial? 5. How to take abstract away clouds from different regions and vendors, allowing workloads to always take advantage of the best available pricing? The talk concludes with real-world test results involving multiple use cases and configurations, giving the audience an idea of the potential cost savings and trade-offs (if any) of combining Kubernetes and SPIs.