Queueing Theory, In Practice: Performance Modelling in Cloud-Native Territory [I]

Kubernetes and similar cloud-native infrastructure make it easier than ever to adjust a service's capacity based on variable demand. In practice, it's still hard to take observed metrics, and translat …

Kubernetes and similar cloud-native infrastructure make it easier than ever to adjust a service’s capacity based on variable demand. In practice, it’s still hard to take observed metrics, and translate them into quantitative predictions about what will happen to service performance as load changes. Resource limits are often chosen by guesstimation, and teams are likely to find themselves reacting to slowdowns and bottlenecks, rather than anticipating them. Queueing theory can help, by treating large-scale software systems as mathematical models. But it’s not easy to translate between real-world systems and textbook models. This talk will cover practical techniques for turning operational data into actionable predictions. We’ll show how to use results from queueing theory to develop a model of system performance. We’ll discuss what data to gather in production to better inform its predictions – for example, why it’s important to capture the shape of a latency distribution, and not just a few percentiles. We’ll also talk about some of the limitations and pitfalls of performance modelling.