Building a distributed real-time stream processing system
Amy Boyle walks you through building, scaling, and monitoring a stream processing pipeline.
| Talk Title | Building a distributed real-time stream processing system |
| Speakers | A boyle (New Relic) |
| Conference | O’Reilly Velocity Conference |
| Conf Tag | Build systems that drive business |
| Location | London, United Kingdom |
| Date | October 31-November 2, 2018 |
| URL | Talk Page |
| Slides | Talk Slides |
| Video | |
The future of software is distributed. If you run a backend service of consequence, you’re probably dealing with some sort of distributed system. Stream processing applications form the backbone of New Relic’s data pipeline processing billions of data points a minute. As a result, the company has learned a few useful things about building scalable distributed stream processing systems. While there are many great tools such as Kafka and Docker orchestration upon which to build feature-rich systems, you still need to understand how these building blocks work and how to apply them effectively and reliably at scale. Amy Boyle walks you through building, scaling, and monitoring a stream processing pipeline, drawing on examples from New Relic’s data pipeline. Topics include:
