Serverless for data and AI

What is serverless, and how can it be utilized for data analysis and AI? Avner Braverman outlines the benefits and limitations of serverless with respect to data transformation (ETL), AI inference and training, and real-time streaming. This is a technical talk, so expect demos and code.

Serverless computing has emerged as the simplest way for developers to run code in the cloud. Serverless platforms let developers break down their code into functions and deploy them directly to the cloud. Functions are then invoked and autoscaled in response to different events like HTTP requests or queued messages. Some of the major use cases for serverless are data transformation in batch and ETL scenarios and data processing using MapReduce patterns. Avner Braverman explains how the elasticity and scale of serverless can revolutionize data processing and how functions can be used for cases like AI inference, model training, stream processing, and streaming pipelines. You’ll also learn how the ease of use of serverless can simplify data science and engineering by offloading the burden of managing infrastructure for data processing. Avner concludes by discussing recent advancements in serverless and demonstrating how breaking from performance and cost limitations enables large-scale real-time serverless data pipelines.