Strata + Hadoop World

Docker on YARN

Docker on YARN

November 6, 2019

Docker makes it easy to bundle an application with its dependencies and provide full isolation, and YARN now supports Docker as an execution engine for submitted applications. Daniel Templeton explains how YARN's Docker support works, why you'd want to use it, and when you shouldn't.
Driving enterprise open source adoption, from data lake to AI (sponsored by Teradata)

Driving enterprise open source adoption, from data lake to AI (sponsored by Teradata)

November 6, 2019

It is no surprise that reducing operational IT expenditures and increasing software capabilities is a top priority for large enterprises. Given its advantages, open source software has proliferated across the globe. Ron Bodkin explains how Teradata drives open source adoption inside enterprises using open source data management and AI techniques leveraged across the analytical ecosystem.
Executive Briefing: IoT and unconventional data

Executive Briefing: IoT and unconventional data

November 6, 2019

The IoT is driven by outcomes delivered by applications, but to gain operational efficiency, many organizations are looking toward a horizontal platform for delivering and supporting a number of applications. Teresa Tung explores how to choose and implement a platformand deal with the fact that the platform is horizontal and application outcomes are vertical.
From rivulets to rivers: Elastic stream processing in Heron

From rivulets to rivers: Elastic stream processing in Heron

November 6, 2019

Twitter processes billions of events per day the instant the data is generated using Heron, an open source streaming engine tailored for large-scale environments. Bill Graham, Avrilia Floratau, and Ashvin Agrawal explore the techniques Heron uses to elastically scale resources in order to handle highly varying loads without sacrificing real-time performance or user experience.
Hoodie: Incremental processing on Hadoop at Uber

Hoodie: Incremental processing on Hadoop at Uber

November 6, 2019

Uber relies on making data-driven decisions at every level, and most of these decisions can benefit from faster data processing. Vinoth Chandar and Prasanna Rajaperumal introduce Hoodie, a newly open sourced system at Uber that adds new incremental processing primitives to existing Hadoop technologies to provide near-real-time data at 10x reduced cost.
How Lufthansa German Airlines is using data analytics to create the next level of customer experience

How Lufthansa German Airlines is using data analytics to create the next level of customer experience

November 5, 2019

The aviation industry is facing a huge pressure in costs as well as a profound disruption in marketing and service. With ticket revenues dropping, increasing customer loyalty is key. Andreas Ribbrock explains how Lufthansa German Airlines uses data science and data-driven decision making to create the next level of digital customer experience along the full customer journey.
How Microsoft predicts churn of cloud customers using deep learning and explains those predictions in an interpretable way

How Microsoft predicts churn of cloud customers using deep learning and explains those predictions in an interpretable way

November 5, 2019

Although deep learning has proved to be very powerful, few results are reported on its application to business-focused problems. Feng Zhu and Val Fontama explore how Microsoft built a deep learning-based churn predictive model and demonstrate how to explain the predictions using LIMEa novel algorithm published in KDD 2016to make the black box models more transparent and accessible.
How Pinterest scaled to build the worlds catalog of 75+ billion ideas

How Pinterest scaled to build the worlds catalog of 75+ billion ideas

November 5, 2019

Over the course of just six years, Pinterest has helped over 100 million pinners discover and collect over 75+ billion ideas to plan their everyday lives. Romit Jadhwani walks you through the different phases of this hypergrowth journey and explores the focuses, thought processes, and decisions of Pinterests data team as they scaled and enabled this growth.
How Spark can fail or be confusing and what you can do about it

How Spark can fail or be confusing and what you can do about it

November 5, 2019

Just like any six-year-old, Apache Spark does not always do its job and can be hard to understand. Yin Huai looks at the top causes of job failures customers encountered in production and examines ways to mitigate such problems by modifying Spark. He also shares a methodology for improving resilience: a combination of monitoring and debugging techniques for users.
How to leverage your private cloud infrastructure to deploy Hadoop

How to leverage your private cloud infrastructure to deploy Hadoop

November 5, 2019

Dwai Lahiri explains how to leverage private cloud infrastructure to successfully build Hadoop clusters and outlines dos, don'ts, and gotchas for running Hadoop on private clouds.
How Vnomics built and deployed a digital twin in commercial trucking that led to $160M (and counting) in verified operational fuel savings

How Vnomics built and deployed a digital twin in commercial trucking that led to $160M (and counting) in verified operational fuel savings

November 5, 2019

Lloyd Palum explores the importance of identifying the target business value in an IIoT applicationa prerequisite to justifying a return on technology investmentand explains how to deliver that value using the concept of a digital twin.
Improving public policy with behavioral science

Improving public policy with behavioral science

November 5, 2019

Maya Shankar discusses the motivation for and impact of the White House Social and Behavioral Sciences Team and shares lessons learned building a startup within the federal government.
Kyligence Inc.

Kyligence Inc.

November 5, 2019

.
Launching Pokmon GO

Launching Pokmon GO

November 5, 2019

Pokmon GO was one of the fastest-growing games of all time, becoming a worldwide phenomenon in a matter of days. In conversation with Beau Cronin, Phil Keslin, CTO of Niantic, explains how the engineering team prepared forand just barely survivedthe experience.
Learning from incomplete, imperfect data with probabilistic programming

Learning from incomplete, imperfect data with probabilistic programming

November 5, 2019

Real-world data is incomplete and imperfect. The right way to handle it is with Bayesian inference. Michael Williams demonstrates how probabilistic programming languages hide the gory details of this elegant but potentially tricky approach, making a powerful statistical method easy and enabling rapid iteration and new kinds of data-driven products.
Leveraging deep learning to predict breast cancer proliferation scores with Apache Spark and Apache SystemML

Leveraging deep learning to predict breast cancer proliferation scores with Apache Spark and Apache SystemML

November 4, 2019

Estimating the growth rate of tumors is a very important but very expensive and time-consuming part of diagnosing and treating breast cancer. Michael Dusenberry and Frederick Reiss describe how to use deep learning with Apache Spark and Apache SystemML to automate this critical image classification task.
Machine learning at Google (sponsored by Google)

Machine learning at Google (sponsored by Google)

November 4, 2019

Rob Craft shares some of the ways machine learning is being used inside of Google, explores cloud-based neural networks, and discusses some customer use cases.
Machine learning is about your data and deployment, not just model development (sponsored by IBM)

Machine learning is about your data and deployment, not just model development (sponsored by IBM)

November 4, 2019

Which is more important: the model or the data? Dinesh Nirmal explains how your data can help you build the right cognitive systems to learn about, reason with, and engage with your customers.
Machine-learning opportunities within the airline industry

Machine-learning opportunities within the airline industry

November 4, 2019

Rodrigo Fontecilla explains how many of the largest airlines use different classes of machine-learning algorithms to create robust and reusable predictive models to provide a holistic view of operations and provide business value.
Machines and the magic of fast learning (sponsored by MemSQL)

Machines and the magic of fast learning (sponsored by MemSQL)

November 4, 2019

Eric Frenkiel explains how to use real-time data as a vehicle for operationalizing machine-learning models by leveraging MemSQL, exploring advanced tools, including TensorFlow, Apache Spark, and Apache Kafka, and compelling use cases demonstrating the power of machine learning to effect positive change.
Making architecture choices for small and big data problems

Making architecture choices for small and big data problems

November 4, 2019

Not all data science problems are big data problems. Lots of small and medium product companies want to start their journey to become data driven. Nischal HP and Raghotham Sripadraj share their experience building data science platforms for various enterprises, with an emphasis on making the right architecture choices and using distributed and fault-tolerant tools.
Making good robots

Making good robots

November 4, 2019

Lets stop talking about bad robots and start talking about what makes a robot good. A good or ethical robot must be carefully designed. Andra Keay outlines five principles of good robot design and discusses the implications of implicit bias in our robots.
Mistakes were made, but not by us: Lessons from a year of supporting Apache Kafka

Mistakes were made, but not by us: Lessons from a year of supporting Apache Kafka

November 4, 2019

Dustin Cote and Ryan Pridgeon share their experience troubleshooting Apache Kafka in production environments and discuss how to avoid pitfalls like message loss or performance degradation in your environment.
Paint the landscape and secure your data center with Apache Spot

Paint the landscape and secure your data center with Apache Spot

November 4, 2019

Cesar Berho and Alan Ross offer an overview of open source project Apache Spot (incubating), which delivers next-generation cybersecurity analytics architecture through unsupervised learning using machine-learning techniques at cloud scale for anomaly detection.
Pluggable security in Hadoop

Pluggable security in Hadoop

November 3, 2019

Security will always be very important in the world of big data, but the choices today mostly start with Kerberos. Does that mean setting up security is always going to be painful? What if your company standardizes on other security alternatives? What if you want to have the freedom to decide what security type to support? Yuliya Feldman and Bill ODonnell discuss your options.
Predicting customer lifetime value for a subscription-based business

Predicting customer lifetime value for a subscription-based business

November 3, 2019

Chao Zhong offers an overview of a new predictive model for customer lifetime value (LTV) in a cloud-computing business. This model is also the first known application of the Fader RFM approach to a cloud businessa Bayesian approach that predicts a customer's LTV with a symmetric absolute percentage error (SAPE) of only 3% on an out-of-time testing dataset.
Presto: Distributed SQL on anything (sponsored by Teradata)

Presto: Distributed SQL on anything (sponsored by Teradata)

November 3, 2019

Teradata joined the Presto community in 2015 and is now a leading contributor to this open source SQL engine, originally created by Facebook. Join Kamil Bajda-Pawlikowski to learn about Presto, Teradata's recent enhancements in query performance, security integrations, and ANSI SQL coverage, and its roadmap for 2017 and beyond.
PyTorch: A flexible and intuitive framework for deep learning

PyTorch: A flexible and intuitive framework for deep learning

November 3, 2019

James Bradbury offers an overview of PyTorch, a brand-new deep learning framework from developers at Facebook AI Research that's intended to be faster, easier, and more flexible than alternatives like TensorFlow. James makes the case for PyTorch, focusing on the library's advantages for natural language processing and reinforcement learning.