March 25, 2020

214 words 2 mins read

Data lakes and distributed systems: The truths and myths

Data lakes and distributed systems: The truths and myths

In his time designing and deploying large-scale data lakes and distributed systems, Jesus Jackson has learned many hard truths and discovered many myths. Join in to hear some of these myths, lessons learned, and war stories.

Talk Title Data lakes and distributed systems: The truths and myths
Speakers Jesus Jackson (eGlobalTech)
Conference O’Reilly Software Architecture Conference
Conf Tag Engineering the Future of Software
Location New York, New York
Date February 24-26, 2020
URL Talk Page
Slides Talk Slides
Video

In his time designing and deploying large-scale data lakes and distributed systems for clients including Fortune 100 companies and federal agencies, Jesus Jackson has learned many hard truths (and techniques that worked well across all of his projects) as well as some solutions that rarely ever work (the myths). Join in to explore these great techniques and approaches for designing a data lake, such as how to integrate data across multiple indices in real time to provide a performant user experience when searching across datasets, securely manage highly sensitive data using cell-level security and encryption, implement real data governance and provenance to really understand your data and how it’s being disseminated across the lake, and more. Jesus also dives into the myths—explaining why they rarely work and providing alternatives—along with a few (humorous) war stories.

comments powered by Disqus