Kubernetes for machine learning: Productivity over primitives

Sophie Watson and William Benton demonstrate high-level open source tools that build on Kubernetes to solve machine learning workflow pain points. They explain why Kubernetes is great for ML and present tools that effortlessly provision custom research environments, publish reproducible notebooks, operationalize models and pipelines as services, and detect data drift automatically.

Kubernetes is today’s hottest way to deploy and manage contemporary applications in the cloud, but it also offers the essential foundation for repeatable and reliable machine learning workflows. Sophie Watson and William Benton demonstrate open source tools that build on Kubernetes to facilitate solving data science workflow challenges for practitioners. They focus on high-level tools that build productive solutions on powerful primitives without forcing data scientists to care about the primitive details of their infrastructure. They’ll walk you through a typical machine learning workflow and show you how Kubernetes supports data scientists at each step. You’ll see tools that effortlessly provision custom research environments, publish reproducible notebooks, operationalize models and pipelines as services, and detect data drift automatically.