Strata + Hadoop World

Embeddable data transformation for real-time streams

Embeddable data transformation for real-time streams

October 25, 2019

Real-time analysis starts with transforming raw data into structured records. Typically this is done with bespoke business logic custom written for each use case. Joey Echeverria presents a configuration-based, reusable library for data transformation that can be embedded in real-time stream-processing systems and demonstrates its real-world use cases with Apache Kafka and Apache Hadoop.
Empowering business users to lead with data

Empowering business users to lead with data

October 25, 2019

The most valuable people in your organization combine business acumen with data savviness. But these data heroes are rare. Denise McInerney describes how she has empowered business users at Intuit to make better decisions with data and explains how you can do the same thing in your organization.
Fast data made easy with Apache Kafka and Apache Kudu (incubating)

Fast data made easy with Apache Kafka and Apache Kudu (incubating)

October 25, 2019

Ted Malaska and Jeff Holoman explain how to go from zero to full-on time series and mutable-profile systems in 40 minutes. Ted and Jeff cover code examples of ingestion from Kafka and Spark Streaming and access through SQL, Spark, and Spark SQL to explore the underlying theories and design patterns that will be common for most solutions with Kudu.
Faster conclusions using in-memory columnar SQL and machine learning

Faster conclusions using in-memory columnar SQL and machine learning

October 25, 2019

Hadoops traditional batch technologies are quickly being supplanted by in-memory columnar execution to drive faster data-to-value. Wes McKinney and Jacques Nadeau provide an overview of in-memory columnar execution, survey key related technologies, including Kudu, Ibis, Impala, and Drill, and cover a sample use case using Ibis in conjunction with Apache Drill to deliver real-time conclusions.
Filling the data lake

Filling the data lake

October 25, 2019

A major challenge in todays world of big data is getting data into the data lake in a simple, automated way. Coding scripts for disparate sources is time consuming and difficult to manage. Developers need a process that supports disparate sources by detecting and passing metadata automatically. Chuck Yarbrough and Mark Burnette explain how to simplify and automate your data ingestion process.
Hadoop in the cloud: Good fit or round peg in a square hole?

Hadoop in the cloud: Good fit or round peg in a square hole?

October 25, 2019

Thomas Phelan and Joel Baxter investigate the advantages and disadvantages of running specific Hadoop workloads in different infrastructure environments. Thomas and Joel then provide a set of rules to help users evaluate big data runtime environments and deployment options to determine which is best suited for a given application.
High-performance clickstream analytics with Apache Phoenix and HBase

High-performance clickstream analytics with Apache Phoenix and HBase

October 25, 2019

Traditional data-warehousing techniques are sometimes limited by the scalability of the implementation tools themselves. Arun Thangamani explains how the advanced architectural approaches by tools like Apache Phoenix and HBase allow new, highly scalable live-analytics solutions using the same traditional techniques and showcases a successful implementation at CDK.
How the oil and gas industry is igniting a spark with information fusion and metadata analytics

How the oil and gas industry is igniting a spark with information fusion and metadata analytics

October 24, 2019

Oil and gas organizations are at the forefront of big data, adopting technologies such as Hadoop and Spark to develop next-generation fusion systems. Brian Clark and Marco Ippolito introduce a case study from CGG, a builder of common data models to drive analytics of sensor data and associated metadata from fast-changing big data streams, to show how to derive richer value from big data assets.
How to build a successful data lake

How to build a successful data lake

October 24, 2019

It is fashionable today to declare doom and gloom for the data lake. Alex Gorelik discusses best practices for Hadoop data lake success and provides real-world examples of successful data lake implementations in a non-vendor-specific talk.
How to make analytic operations look more like DevOps: Lessons learned moving machine-learning algorithms to production environments

How to make analytic operations look more like DevOps: Lessons learned moving machine-learning algorithms to production environments

October 24, 2019

There is a big difference between running a machine-learning algorithm manually from time to time and building a production system that runs thousands of machine-learning algorithms each day on petabytes of data, while also dealing with all the edge cases that arise. Robert Grossman discusses some of the lessons learned when building such a system and explores the tools that made the job easier.
How to turn your house into a robot: An adaptive-learning algorithm for the Internet of Things

How to turn your house into a robot: An adaptive-learning algorithm for the Internet of Things

October 24, 2019

Modern houses and robots have a lot in common. Both have a lot of sensors and have to make a lot of decisions. However, unlike houses, robots adapt and perform helpful tasks. Brandon Rohrer details an algorithm specifically designed to help houses, buildings, roads, and stores learn to actively help the people that use them.
How to use your data science team: Becoming a data-driven organization

How to use your data science team: Becoming a data-driven organization

October 24, 2019

Youve decided you need data scientists. You know who to hire. Now, what do you do with them? Yael Garten offers examples of how companies like LinkedIn use data to make business and product decisions. Yael reviews the spectrum of data science, and discusses the culture, process and tools needed to transform companies into data-driven organizations.
In search of database nirvana: The challenges of delivering HTAP

In search of database nirvana: The challenges of delivering HTAP

October 24, 2019

Companies are looking for a single database engine that can address all their varied needsfrom transactional to analytical workloads, against structured, semistructured, and unstructured data, leveraging graph databases, document stores, text search engines, column stores, key value stores, and wide column stores. Rohit Jain discusses the challenges one faces on the path to this nirvana.
Inside Cigna's big data journey

Inside Cigna's big data journey

October 24, 2019

How do you implement Apache Hadoop in a large healthcare company with a mature data-analysis infrastructure? Jeffrey Shmain and Mohammad Quraishi describe Cigna's journey toward big data and Hadoop, including an overview of new Hadoop capabilities like heterogeneous data integration and large-scale machine learning.
IoT in the enterprise: A look at Intel (IoT) Inside

IoT in the enterprise: A look at Intel (IoT) Inside

October 23, 2019

Moty Fania shares Intels IT experience implementing an on-premises big data IoT platform for internal use cases. This unique platform was built on top of several open source technologies and enables highly scalable stream analytics with a stack of algorithms such as multisensor change detection, anomaly detection, and more.
Large-scale product classification via text and image-based signals using a fusion of discriminative and deep learning-based classifiers

Large-scale product classification via text and image-based signals using a fusion of discriminative and deep learning-based classifiers

October 23, 2019

Typically, 810% of product URLs in ecommerce sites are misclassified. Sreeni Iyer and Anurag Bhardwaj discuss a machine-learning-based solution that relies on an innovative fusion of classifiers that are both text- and image-based, along with human touch to handle edge cases, to automatically classify product URLs according to a canonical taxonomic organization with a high F-score.
Lessons learned building a scalable self-serve, real-time, multitenant monitoring service at Yahoo

Lessons learned building a scalable self-serve, real-time, multitenant monitoring service at Yahoo

October 23, 2019

Building a real-time monitoring service that handles millions of custom events per second while satisfying complex rules, varied throughput requirements, and numerous dimensions simultaneously is a complex endeavor. Sumeet Singh and Mridul Jain explain how Yahoo approached these challenges with Apache Storm Trident, Kafka, HBase, and OpenTSDB and discuss the lessons learned along the way.
Let's get real: Acting on data in real time

Let's get real: Acting on data in real time

October 23, 2019

Big data is not limited to reporting and analysis; increasingly, companies are differentiating themselves by acting on data in real time. But what does "real time" really mean? Jack Norris discusses the challenges of coordinating data flows, analysis, and integration at scale to truly impact business as it happens.
Machine learning for human rights advocacy: Big benefits, serious consequences

Machine learning for human rights advocacy: Big benefits, serious consequences

October 23, 2019

Megan Price demonstrates how machine-learning methods help us determine what we know, and what we don't, about the ongoing conflict in Syria. Megan then explains why these methods can be crucial to better understand patterns of violence, enabling better policy decisions, resource allocation, and ultimately, accountability and justice.
Netflix's big leap from Oracle to Cassandra

Netflix's big leap from Oracle to Cassandra

October 23, 2019

Roopa Tangirala details Netflix's migration from Oracle to Cassandra, covering the problems encountered, what worked and what didn't, and lessons learned along the way.
NoLambda: A new architecture combining streaming, ad hoc, machine-learning, and batch analytics

NoLambda: A new architecture combining streaming, ad hoc, machine-learning, and batch analytics

October 22, 2019

Developers who want both streaming analytics and ad hoc, OLAP-like analysis have often had to develop complex architectures such as Lambda. Helena Edelson and Evan Chan highlight a much simpler approach using the NoLambda stack (Apache Spark/Scala, Mesos, Akka, Cassandra, Kafka) plus FiloDB, a new entrant to the distributed-database world, which combines streaming and ad hoc analytics.
Nonsense science

Nonsense science

October 22, 2019

Paula Poundstone isnt just any comedian. After years of justly criticizing and questioning the purpose of the many studies used for questions on NPRs #1 show, Wait Wait. . .Dont Tell Me, on which she's a popular panelist, shes here to explore what we can learn about asking the right questions from a unique critique of published behavioral research.
Old industries, sexy data: How machine learning is reshaping the world's backbone industries

Old industries, sexy data: How machine learning is reshaping the world's backbone industries

October 22, 2019

Over the past decade, machine learning has become intertwined with newer, Internet-born businesses. This despite the fact that the vast majority of global GDP turns on larger, less visible industries like energy and construction. David Beyer explores the ways these backbone industries are adopting machine-intelligent applications and the trends underlying this shift.
Open by design, open for data

Open by design, open for data

October 22, 2019

As the volume and variety of data continue to grow, organizations have the opportunity to transform their industries and professions, but companies are grappling with how to deliver innovation. Adam Kocoloski shares his experience around this market shift and challenges attendees to join his mission of contributing to the community and investing in the power of open source and the cloud.
Practical machine learning (Full Day)

Practical machine learning (Full Day)

October 22, 2019

This hands-on tutorial provides a quick start to building intelligent business applications using machine learning. Learn about machine-learning basics, feature engineering, anomaly detection, recommender systems, and deep learning as you are guided through all the steps of prototyping and production: data cleaning, feature engineering, model building and evaluation, and deployment.
Putting Kafka into overdrive

Putting Kafka into overdrive

October 22, 2019

Apache Kafka lies at the heart of the largest data pipelines, handling trillions of messages and petabytes of data every day. Learn the right approach for getting the most out of Kafka from the experts at LinkedIn and Confluent. Todd Palino and Gwen Shapira explore how to monitor, optimize, and troubleshoot performance of your data pipelinesfrom producer to consumer, development to production.
Python scalability: A convenient truth

Python scalability: A convenient truth

October 21, 2019

Despite Python's popularity throughout the data-engineering and data science workflow, the principles behind its performance and scaling behavior are less understood. Travis Oliphant explains best practices and modern tools to scale Python to larger-than-memory and distributed workloads without sacrificing its ease of use or being forced to adopt heavyweight frameworks.
Radical candor: Be a better boss

Radical candor: Be a better boss

October 21, 2019

Bad bosses make people miserable. They kill innovation, stifle growth, increase costs, and create instability. Well-meaning people become bad bosses without even realizing it. Great bosses have relationships with employees and are sources of growth and stability for both individuals and companies. Kim Scott outlines three principles for approaching the relationship between a boss and their team.