October 28, 2019

209 words 1 min read

Amazon for information: Building a modern data catalog

Amazon for information: Building a modern data catalog

A data catalog provides context to help data analysts, data scientists, and other data consumers (including those with little technical background) find a relevant dataset, determine if it can be trusted, understand what it means, and utilize it to make better products and better decisions. Aaron Kalb explores how enterprises build interfaces that make sourcing data as easy as shopping on Amazon.
Talk Title Amazon for information: Building a modern data catalog
Speakers Aaron Kalb (Alation)
Conference Strata + Hadoop World
Conf Tag Big Data Expo
Location San Jose, California
Date March 29-31, 2016
URL Talk Page
Slides Talk Slides
Video

A data catalog provides context to help data analysts, data scientists, and other data consumers (including those with little technical background) find a relevant dataset, determine if it can be trusted, understand what it means, and utilize it to make better products and better decisions. Aaron Kalb explores how enterprises build interfaces that make sourcing data as easy as shopping on Amazon. Aaron gives an overview of data catalogs and explains how they relate to concepts like data dictionaries or data inventories. He also covers some of the fastest and most effective ways to build a data catalog, discussing the roles crowds, experts, and machines play. Topics include:

dataset
comments powered by Disqus
Atom smashing using machine learning at CERN

Atom smashing using machine learning at CERN

October 28, 2019

Siddha Ganju explains how CERN uses machine-learning models to predict which datasets will become popular over time. This helps to replicate the datasets that are most heavily accessed, which improves the efficiency of physics analysis in CMS. Analyzing this data leads to useful information about the physical processes.
Fast data made easy with Apache Kafka and Apache Kudu (incubating)

Fast data made easy with Apache Kafka and Apache Kudu (incubating)

October 25, 2019

Ted Malaska and Jeff Holoman explain how to go from zero to full-on time series and mutable-profile systems in 40 minutes. Ted and Jeff cover code examples of ingestion from Kafka and Spark Streaming and access through SQL, Spark, and Spark SQL to explore the underlying theories and design patterns that will be common for most solutions with Kudu.
How to make analytic operations look more like DevOps: Lessons learned moving machine-learning algorithms to production environments

How to make analytic operations look more like DevOps: Lessons learned moving machine-learning algorithms to production environments

October 24, 2019

There is a big difference between running a machine-learning algorithm manually from time to time and building a production system that runs thousands of machine-learning algorithms each day on petabytes of data, while also dealing with all the edge cases that arise. Robert Grossman discusses some of the lessons learned when building such a system and explores the tools that made the job easier.
IoT in the enterprise: A look at Intel (IoT) Inside

IoT in the enterprise: A look at Intel (IoT) Inside

October 23, 2019

Moty Fania shares Intels IT experience implementing an on-premises big data IoT platform for internal use cases. This unique platform was built on top of several open source technologies and enables highly scalable stream analytics with a stack of algorithms such as multisensor change detection, anomaly detection, and more.
TensorFlow: Large-scale analytics and distributed machine learning with TensorFlow, BigQuery, and Dataflow (Apache Beam)

TensorFlow: Large-scale analytics and distributed machine learning with TensorFlow, BigQuery, and Dataflow (Apache Beam)

October 20, 2019

Kazunori Sato and Amy Unruh explore how you can use TensorFlow to drive large-scale distributed machine learning against your analytic data sitting in Google BigQuery, with data preprocessing driven by Dataflow (now Apache Beam). Kazunori and Amy dive into practical examples of how these technologies can work together to enable a powerful workflow for distributed machine learning.