Executive Briefing: What it takes to use machine learning in fast data pipelines

Dean Wampler dives into how (and why) to integrate ML into production streaming data pipelines and to serve results quickly; how to bridge data science and production environments with different tools, techniques, and requirements; how to build reliable and scalable long-running services; and how to update ML models without downtime.

Dean Wampler helps you develop a conceptual understanding of the challenges faced by your teams as they develop and deploy machine learning (ML) and artificial intelligence (AI) services integrated with fast data (streaming) pipelines. While combining these technologies is challenging, the benefits include timely delivery of innovative services to your customers. You’ll gain a brief overview of the business justification for integrating ML and AI and streaming as well as the ML and AI scenarios that are best delivered through streaming. Dean walks you through the main challenges when using these technologies together; ways to bridge the gap between data science and production teams, their tools, methods, and sometimes conflicting goals, for example, the exploration of ideas and optimal scoring results versus production reliability and efficiency; streaming ML and AI services must run reliably and handle variable loads for a long time, requiring you to leverage best practices from the microservices world; and updating models in the streaming application before they become stale without downtime and other practical problems.