Data lakes and distributed systems: The truths and myths
In his time designing and deploying large-scale data lakes and distributed systems, Jesus Jackson has learned many hard truths and discovered many myths. Join in to hear some of these myths, lessons learned, and war stories.
|Talk Title||Data lakes and distributed systems: The truths and myths|
|Speakers||Jesus Jackson (eGlobalTech)|
|Conference||O’Reilly Software Architecture Conference|
|Conf Tag||Engineering the Future of Software|
|Location||New York, New York|
|Date||February 24-26, 2020|
In his time designing and deploying large-scale data lakes and distributed systems for clients including Fortune 100 companies and federal agencies, Jesus Jackson has learned many hard truths (and techniques that worked well across all of his projects) as well as some solutions that rarely ever work (the myths). Join in to explore these great techniques and approaches for designing a data lake, such as how to integrate data across multiple indices in real time to provide a performant user experience when searching across datasets, securely manage highly sensitive data using cell-level security and encryption, implement real data governance and provenance to really understand your data and how it’s being disseminated across the lake, and more. Jesus also dives into the myths—explaining why they rarely work and providing alternatives—along with a few (humorous) war stories.