Large-scale automated storage on Kubernetes

Managing large stateful applications is tough. Matt Schallert outlines the challenges of automating stateful systems at scale and details how embracing a declarative approach can ease operation and automation of these systems on orchestrators such as Kubernetes. He then demonstrates how to apply this methodology to different types of stateful workloads.

Managing large stateful applications is tough. Matt Schallert outlines how, using Kubernetes, Uber automated managing a challenging stateful workload—M3DB, its sharded, replicated, multizone time series database—and examines the operational challenges the company faced while scaling M3DB from a handful of clusters to over 40 clusters across multiple data centers and cloud providers, all while trying to create an environment-agnostic solution for open source users. Matt then demonstrates methods of managing stateful workloads in a declarative manner to ease operational burden. You’ll see how M3DB’s declarative approach to cluster management can be extended to other workloads using its common set of open source libraries. This approach made orchestrating M3DB easier. Along the way, Matt shares lessons learned that you can apply to a variety of stateful workloads across bare metal and cloud environments, regardless of whether it’s running under an orchestration system or managing instances directly. You’ll walk away with advice for managing stateful systems at scale and lessons to bear in mind when considering using an orchestration system for state management.