December 5, 2019

123 words 1 min read

Stream processing for the practitioner: Blueprints for common stream processing use cases with Apache Flink

Stream processing for the practitioner: Blueprints for common stream processing use cases with Apache Flink

Aljoscha Krettek offers an overview of the modern stream processing space, details the challenges posed by stateful and event-time-aware stream processing, and shares core archetypes ("application blueprints) for stream processing drawn from real-world use cases with Apache Flink.
Talk Title Stream processing for the practitioner: Blueprints for common stream processing use cases with Apache Flink
Speakers Aljoscha Krettek (Ververica)
Conference Strata Data Conference
Conf Tag Making Data Work
Location London, United Kingdom
Date May 22-24, 2018
URL Talk Page
Slides Talk Slides
Video

Aljoscha Krettek offers an overview of the modern stream processing space, details the challenges posed by stateful and event-time-aware stream processing, and shares core archetypes (“application blueprints”) for stream processing drawn from real-world use cases with Apache Flink. Topics include:

flink apache use case
comments powered by Disqus
Building stream processing as a service at Netflix

Building stream processing as a service at Netflix

November 17, 2019

Steven Wu explains how Netflixs SPaaS platform empowers users to focus on extracting insights from data streams and build stream processing applications and shares lessons learned building and operating the largest SPaaS use case: Netflixs Keystone data pipeline, a self-serve platform for creating near-real-time event pipelines that processes three trillion events and 12 PB of data every day.
The ultimate data scientist's playground: Building a multipetabyte analytic infrastructure for cyber defense

The ultimate data scientist's playground: Building a multipetabyte analytic infrastructure for cyber defense

December 5, 2019

Lee Blum offers an overview of Verint's large-scale cyber-defense system built to serve its data scientists with versatile analytic operations on petabytes of data and trillions of records, covering the company's extremely challenging use case, decision considerations, major design challenges, tips and tricks, and the systems overall results.
Automatic 3D MRI knee damage classification with 3D CNN using BigDL on Spark

Automatic 3D MRI knee damage classification with 3D CNN using BigDL on Spark

November 29, 2019

Damage to the meniscus is a physically limiting injury that can lead to further medical complications. Automatically classifying this damage at the time of an MRI scan would allow quicker and more accurate diagnosis. Jennie Wang, Valentina Pedoia, Berk Norman, and Yulia Tell offer an overview of their classification system built with 3D convolutional neural networks using BigDL on Apache Spark.
Effectively once in Apache Pulsar, the next-generation messaging system

Effectively once in Apache Pulsar, the next-generation messaging system

November 26, 2019

Traditionally, messaging systems have offered at-least-once delivery semantics, leaving the task of implementing idempotent processing to the application developers. Matteo Merli explains how to add effectively once semantics to Apache Pulsar using a message deduplication layer that can ensure those stricter semantics with guaranteed accuracy and no performance penalty.
Lyft's analytics pipeline: From Redshift to Apache Hive and Presto

Lyft's analytics pipeline: From Redshift to Apache Hive and Presto

November 23, 2019

Lyfts business has grown over 100x in the past four years. Shenghu Yang explains how Lyfts data pipeline has evolved over the years to serve its ever-growing analytics use cases, migrating from the world's largest AWS Redshift clusters to Apache Hive and Presto for solving scalability and concurrency hard limits.
Streaming SQL to unify batch and stream processing: Theory and practice with Apache Flink at Uber

Streaming SQL to unify batch and stream processing: Theory and practice with Apache Flink at Uber

November 20, 2019

Fabian Hueske and Shuyi Chen explore SQL's role in the world of streaming data and its implementation in Apache Flink and cover fundamental concepts, such as streaming semantics, event time, and incremental results. They also share their experience using Flink SQL in production at Uber, explaining how Uber leverages Flink SQL to solve its unique business challenges.