ETL 2.0: Its not just for data engineers anymore

Robin Moffatt explores the concepts of events, their relevance to software and data engineers, and their ability to unify architectures in a powerful way. Join in to learn why analytics, data integration, and ETL fit naturally into a streaming world. Along the way, Robin leads a hands-on demonstration of these concepts in practice and commentary on the design choices made.

Developers have long employed message queues to decouple subsystems and provide an approximation of asynchronous processing. But these legacy queuing systems don’t adequately deliver on the promise of event-driven architectures and often lead to contrived integration patterns. Their limited scalability and durability limit you in using events to their full potential. Events carry both notification and state and form a powerful primitive upon which to build systems for developers and data engineers alike. Developers benefit from the asynchronous communication that events enable between services, and data engineers benefit from the integration capabilities. Everyone gains from using the standards-based, scalable, and resilient streaming platform. Robin Moffatt explores the concepts of events, their relevance to software and data engineers, and their powers for unifying architectures in a powerful way. You’ll see how stream processing makes sense in both a microservices and ETL environment, and how ETL is actually more common than you may think. Robin provides a hands-on demonstration of these concepts in practice using Apache Kafka and commentary on the design choices made.