Why and how to leverage the power and simplicity of SQL on Apache Flink

Fabian Hueske discusses why SQL is a great approach to unify batch and stream processing. He gives an update on Apache Flink's SQL support and shares some interesting use cases from large-scale production deployments. Finally, Fabian presents Flink's new query service that enables users and applications to submit streaming and batch SQL queries and retrieve low-latency updated results.

Everybody working with data knows SQL. Apache Flink provides SQL support for querying and processing batch and streaming data. Flink’s SQL support powers large-scale production systems at Alibaba, Huawei, and Uber. Based on Flink SQL, these companies have built systems for their internal users as well as publicly offered services for paying customers. Fabian Hueske discusses why and how to leverage the simplicity and power of SQL on Flink. Fabian starts by exploring the use cases that Flink SQL was designed for and presents some real-world problems that it can solve. In particular, he explains why unified batch and stream processing is important and what it means to run SQL queries on streams of data. Fabian then demonstrates how to leverage Flink’s full potential. Since the end of last year, the Flink community has been working on a service that integrates a query interface, (external) table catalogs, and result serving functionality for static, appending, and updating result sets. Fabian explores the design and features of this query service and details how it enables exploratory batch and streaming queries, ETL pipelines, and live updating query results that serve applications, such as real-time dashboards.