November 22, 2019

151 words 1 min read

Deploying SQL Stream Processing in Kubernetes with Ease

Deploying SQL Stream Processing in Kubernetes with Ease

Real-time processing allows you to act faster and SQL allows you to construct flows quicker and reuse existing skills. Apache Kafka is a key component but how do you peek into the data, the topologies …
Talk Title Deploying SQL Stream Processing in Kubernetes with Ease
Speakers Antonios Chalkiopoulos (CEO, Landoop), Andrew Stevenson (CTO, Landoop)
Conference KubeCon + CloudNativeCon Europe
Conf Tag
Location Copenhagen, Denmark
Date Apr 30-May 4, 2018
URL Talk Page
Slides Talk Slides
Video

Real-time processing allows you to act faster and SQL allows you to construct flows quicker and reuse existing skills. Apache Kafka is a key component but how do you peek into the data, the topologies and create repeatable deployments that can be promoted to production? Andrew and Antonios will show you how to easily create end to end data pipelines on Apache Kafka and deploy them to Kubernetes with no code required, only configuration.

kafka code apache sql pipeline kubernetes
comments powered by Disqus
Moving the needle of the pin: Streaming hundreds of terabytes of pins from MySQL to S3/Hadoop continuously

Moving the needle of the pin: Streaming hundreds of terabytes of pins from MySQL to S3/Hadoop continuously

November 22, 2019

With the rise of large-scale real-time computation, there is a growing need to link legacy MySQL systems with real-time platforms. Henry Cai and Yi Yin offer an overview of WaterMill, Pinterest's continuous DB ingestion system for streaming SQL data into near-real-time computation pipelines to support dynamic personalized recommendations and search indices.
Stream processing with Kafka

Stream processing with Kafka

November 20, 2019

Tim Berglund leads a basic architectural introduction to Kafka and walks you through using Kafka Streams and KSQL to process streaming data.
Streaming applications as microservices using Kafka, Akka Streams, and Kafka Streams

Streaming applications as microservices using Kafka, Akka Streams, and Kafka Streams

November 20, 2019

Join Dean Wampler and Boris Lublinsky to learn how to build two microservice streaming applications based on Kafka using Akka Streams and Kafka Streams for data processing. You'll explore the strengths and weaknesses of each tool for particular design needs and contrast them with Spark Streaming and Flink, so you'll know when to choose them instead.
Playing well together: Big data beyond the JVM with Spark and friends

Playing well together: Big data beyond the JVM with Spark and friends

November 22, 2019

Holden Karau and Rachel Warren explore the state of the current big data ecosystem and explain how to best work with it in non-JVM languages. While much of the focus will be on Python + Spark, the talk will also include interesting anecdotes about how these lessons apply to other systems (including Kafka).
Using kubectl to Run your End-to-End Tests

Using kubectl to Run your End-to-End Tests

November 22, 2019

End-to-end (e2e) tests are tasked to verify that a system with all its moving components (without mocks) can work together and satisfy the end user goals. Just as Kubernetes helps us with managing mic …
Monitoring: Follow the Data from the Darkest Corners of Kubernetes

Monitoring: Follow the Data from the Darkest Corners of Kubernetes

November 21, 2019

Have you ever thought what are the sources of useful signals in Kubernetes? How to collect those data? The data that are crucial for understanding the system. How to ingest them to a human friendly to …