Building a data lake is a hard task. You have to centralize all the data of the company in one place, it must be easily accessible, and governance has to be done right. And, last but not least, the price has to stay reasonable. All those aspects come up as quite a challenge. But never fear. Viacheslav Inozemtsev outlines the experience of building Zalando's data lake.

Launched in 2018, several blueprints are work in progress such as KNI(Kubernetes Native Infrastructure for Edge) project for 5G edge computing, in Akraino Edge Stack.Service Providers started to explo …

For T-Mobile, the purpose of public cloud is to accelerate time-to-value for customers, unlocking developer agility while protecting the business from critical security and compliance issues. This is …

Jungwook Seo walks you through a data analytics platform in the cloud by the name of AccuInsight+ with eight data analytic services in the CloudZ (one of the biggest cloud service providers in Korea), which SK Holdings announced in January 2019.

According to Gartner, over 80% of data lake projects were deemed inefficient. Data lakes come and go. Swamps happen. Data agility is fleeting. Chuck Yarbrough walks you through how data ops practices and a modern data architecture bring greater visibility and allow faster data access with proper governance.

Andrew Brust provides a primer on data catalogs and a review of the major vendors and platforms in the market. He examines the use of data catalogs with classic and newer data repositories, including data warehouses, data lakes, cloud object storage, and even software and applications. You'll learn about AI's role in the data catalog world and get an analysis of data catalog futures.

Damon Cortesi demonstrates how to use AWS Glue and Amazon Athena to implement an end-to-end pipeline.

Nikki Rouda shares key trends in data lakes and analytics and explains how they shape the services offered by AWS. Specific topics include the rise of machine-generated data and semistructured and unstructured data as dominant sources of new data, the move toward serverless, SPI-centric computing, and the growing need for local access to data from users around the world.

Francesco Mucio shares the basic tools he and his team had to learn (or relearn) moving from the coziness of their database to the big world of Spark, cloud, distributed systems, and continuous applications. It was an unexpected journey that ended exactly where it started: with an SQL query.

Max Schultze details Zalondo's end-to-end data integration platform to serve analytical use cases and machine learning throughout the company, covering raw data collection, standardized data preparation (binary conversion, partitioning, etc.), user-driven analytics, and machine learning.

Travel platform Expedia Group likes to give its data teams flexibility and autonomy to work with different technologies. However, this approach generates challenges that cannot be solved by existing tools. Pradeep Bhadani and Elliot West explain how the company built a unified virtual data lake on top of its many heterogeneous and distributed data platforms.

NMC (Nielsen Marketing Cloud) provides customers (both marketers and publishers) with real-time analytics tools to profile their target audiences. To achieve that, the company needs to ingest billions of events per day into its big data stores in a scalable, cost-efficient way. Itai Yaffe explains how NMC continuously transforms its data infrastructure to support these goals.

AI will enable more intelligent automation, able to constantly adapt to field reality and specificities.Through this talk, a prospective vision of the management of an Al model learning, deployment an …

Recent headline-grabbing data breaches demonstrate that protecting data is essential for every enterprise. The best-of-breed approach for big data is HDFS configured with Transparent Data Encryption (TDE). But TDE is difficult to configure and manageparticularly when run in Docker containers. Thomas Phelan discusses these challenges and explains how to overcome them.

Have you ever launched a large ETL job to check a fix for a corner case in a derived calculation or normalization? Marc Siegel shares lessons learned applying the event sourcing pattern within an ETL pipeline. Key takeaway in regex form: E{1}TL – that is, "Extract once, transform and load infinite times."

In todays world, data is at the core of every business decision made. As data grows, companies have started implementing their own Data Lakes to store and run analytics on the data. Ceph is widely us …