Why stream? The advantages of working with streaming data

Life doesnt happen in batches. Being able to work with data from continuous events as data streams is a better fit to the way life happens, but doing so presents some challenges. Ellen Friedman examines the advantages and issues involved in working with streaming data, takes a look at emerging technologies for streaming, and describes best practices for this style of work.

Our best understanding comes when conclusions fit evidence and when evidence and analysis is a good fit to the way life happens. That is in part why people are increasingly looking to work with data streams. Telecommunications companies handle large volume streaming data and need to gain insights from anomaly detection and predictive modeling to understand their networks and their users. Web-based retail companies, IoT-based industries, and healthcare companies all have uses for streaming data as well. Ellen Friedman demonstrates the advantages of a stream-based approach, exploring real-world situations in which companies in a variety of sectors are using stream processing, including in production, as she dives deeper into streaming issues such as low latency, windowing, and maintaining state—in essence different aspects of correctness. Examples will focus on best practices for streaming architecture, the importance of stream transport capabilities of tools like Apache Kafka, and how the new stream processing engine Apache Flink provides real-time or batch-based processing.