big data

The business bottom line of data lakes: Real-life experiences

The business bottom line of data lakes: Real-life experiences

November 16, 2019

Data is no longer a by-product of business transactions; now, data is the business. Franz Aman explains how data lakes can put the power of big data into the hands of every business person, sharing the inside scoop on how he turned marketing into a new kind of revenue-generation machine and interviewing an Informatica customer about how data lakes have innovated and transformed their business.
The Internet of Things: Its the (sensor) data, stupid

The Internet of Things: Its the (sensor) data, stupid

November 16, 2019

Martin Willcox shares the lessons he's learned from successful Teradata IoT projects about about how to manage and leverage sensor data and explains why data management, data integration, and multigenre analytics are foundational to driving business value from IoT initiatives.
Triggers in Apache Beam (incubating): User-controlled balance of completeness, latency, and cost in streaming big data pipelines

Triggers in Apache Beam (incubating): User-controlled balance of completeness, latency, and cost in streaming big data pipelines

November 16, 2019

Drawing on important real-world use cases, Kenneth Knowles delves into the details of the language- and runner-independent semantics developed for triggers in Apache Beam, demonstrating how the semantics support the use cases as well as all of the above variability in streaming systems. Kenneth then describes some of the particular implementations of those semantics in Google Cloud Dataflow.
Transforming big data into experiences that drive behavior changes

Transforming big data into experiences that drive behavior changes

November 3, 2019

Data alone lacks the social connection necessary to drive behavior change. However, combining the huge amount of available data with interactive experiences allows us to tell powerful stories that can motivate people to make lasting behavior changes. Erica Stanley and Erich Lee dive into the psychology of behavior change and explain how to convey stories in the data to support desired changes.
Reverse evaluating Netflix's architecture

Reverse evaluating Netflix's architecture

October 30, 2019

Netflix is built on modern, efficient, and robust architectural concepts. Should you follow Netflix's lead and refactor your systems into microservices, split up big databases, and use polyglot approaches? Stefan Toth discusses an inverse architecture evaluation that embarc Software Consulting GmbH conducted to find the answers.
A few things engineers can learn from designers

A few things engineers can learn from designers

October 29, 2019

Big data is great for feeding ML algorithms, but you quickly face a bandwidth issue when interfacing with humans. The brain is a fantastic information-processing machine and has an unparalleled, innate ability to detect patterns. Sbastien Pierre explains what designers can teach engineers about creating new ways to make large volumes of data understandable at the human level.
Advanced analytics and the mystery of the missing jeans

Advanced analytics and the mystery of the missing jeans

October 28, 2019

Bob Rogers, Intel's chief data scientist for big data solutions, demonstrates the power of the question in analytics. Learn how different types of data, from cubes of structured data to live video streams from mobile systems, combine with analytical technology to inform the questions that can be answered.
Apache Hadoop operations for production systems (Full Day)

Apache Hadoop operations for production systems (Full Day)

October 28, 2019

In this full-day tutorial, participants will get an overview of all aspects of successfully managing Hadoop clustersfrom installation to configuration management, service monitoring, troubleshooting, and support integrationwith an emphasis on production systems.
Architecting HBase in the field

Architecting HBase in the field

October 28, 2019

Most already know HBase, but many don't know that it can be coupled with other tools from the ecosystem to increase efficiency. Jean-Marc Spaggiari and Kevin O'Dell walk attendees through some real-life HBase use cases and demonstrate how they have been efficiently implemented.
Automating decision making with big data: How to make it work

Automating decision making with big data: How to make it work

October 28, 2019

While many companies struggle to adopt big data, a number of industry leaders are leapfrogging big data adoption by going straight to automating core business processes. Andreas Schmidt presents examples from leading European companies that have overcome cultural, technical, and scientific challenges and unlocked the potential of big data in an entirely different way.
Big data-fueled feedback loops leveraging streaming data in SDN/NFV

Big data-fueled feedback loops leveraging streaming data in SDN/NFV

October 27, 2019

Software-defined networking (SDN) and network functions virtualization (NFV) hold tremendous potential to enable efficiency and flexibility in service delivery, but SDN/NFV environments are also highly complex and multilayered. Matt Olson explains why effective support for SDN/NFV services requires leveraging the tremendous amount of service and data streaming from the platform.
Building a scalable architecture for processing streaming data on AWS

Building a scalable architecture for processing streaming data on AWS

October 27, 2019

Analyzing real-time streams of data is becoming increasingly important to remain competitive. Siva Raghupathy and Manjeet Chayel guide attendees through some of the proven architectures for processing streaming data using a combination of cloud and open source tools such as Apache Spark. Watch a live demo and learn how you can easily scale your applications with Amazon Web Services.
Can you afford to drop ACID? Understanding real-world SQL requirements in the big data era

Can you afford to drop ACID? Understanding real-world SQL requirements in the big data era

October 26, 2019

Hadoop can bring great value to businesses but also big headaches. Some solutions that provide SQL access to Hadoop data mean changing your business processes to overcome limitations in the technologies. Emma McGrattan explains how users can unlock tremendous business value through SQL-driven Hadoop solutions. Emma outlines what should be on your checklist and the pitfalls to avoid.
Developing a big data business strategy

Developing a big data business strategy

October 25, 2019

Organizations do not need a big data strategy. They need a business strategy that incorporates big data. Most organizations lack a roadmap for using big data to uncover new business opportunities. Bill Schmarzo explains how to explore, justify, and plan big data projects with business management.
Faster conclusions using in-memory columnar SQL and machine learning

Faster conclusions using in-memory columnar SQL and machine learning

October 25, 2019

Hadoops traditional batch technologies are quickly being supplanted by in-memory columnar execution to drive faster data-to-value. Wes McKinney and Jacques Nadeau provide an overview of in-memory columnar execution, survey key related technologies, including Kudu, Ibis, Impala, and Drill, and cover a sample use case using Ibis in conjunction with Apache Drill to deliver real-time conclusions.
Filling the data lake

Filling the data lake

October 25, 2019

A major challenge in todays world of big data is getting data into the data lake in a simple, automated way. Coding scripts for disparate sources is time consuming and difficult to manage. Developers need a process that supports disparate sources by detecting and passing metadata automatically. Chuck Yarbrough and Mark Burnette explain how to simplify and automate your data ingestion process.
Hadoop in the cloud: Good fit or round peg in a square hole?

Hadoop in the cloud: Good fit or round peg in a square hole?

October 25, 2019

Thomas Phelan and Joel Baxter investigate the advantages and disadvantages of running specific Hadoop workloads in different infrastructure environments. Thomas and Joel then provide a set of rules to help users evaluate big data runtime environments and deployment options to determine which is best suited for a given application.
How the oil and gas industry is igniting a spark with information fusion and metadata analytics

How the oil and gas industry is igniting a spark with information fusion and metadata analytics

October 24, 2019

Oil and gas organizations are at the forefront of big data, adopting technologies such as Hadoop and Spark to develop next-generation fusion systems. Brian Clark and Marco Ippolito introduce a case study from CGG, a builder of common data models to drive analytics of sensor data and associated metadata from fast-changing big data streams, to show how to derive richer value from big data assets.
How to build a successful data lake

How to build a successful data lake

October 24, 2019

It is fashionable today to declare doom and gloom for the data lake. Alex Gorelik discusses best practices for Hadoop data lake success and provides real-world examples of successful data lake implementations in a non-vendor-specific talk.
Inside Cigna's big data journey

Inside Cigna's big data journey

October 24, 2019

How do you implement Apache Hadoop in a large healthcare company with a mature data-analysis infrastructure? Jeffrey Shmain and Mohammad Quraishi describe Cigna's journey toward big data and Hadoop, including an overview of new Hadoop capabilities like heterogeneous data integration and large-scale machine learning.
IoT in the enterprise: A look at Intel (IoT) Inside

IoT in the enterprise: A look at Intel (IoT) Inside

October 23, 2019

Moty Fania shares Intels IT experience implementing an on-premises big data IoT platform for internal use cases. This unique platform was built on top of several open source technologies and enables highly scalable stream analytics with a stack of algorithms such as multisensor change detection, anomaly detection, and more.
Lessons learned building a scalable self-serve, real-time, multitenant monitoring service at Yahoo

Lessons learned building a scalable self-serve, real-time, multitenant monitoring service at Yahoo

October 23, 2019

Building a real-time monitoring service that handles millions of custom events per second while satisfying complex rules, varied throughput requirements, and numerous dimensions simultaneously is a complex endeavor. Sumeet Singh and Mridul Jain explain how Yahoo approached these challenges with Apache Storm Trident, Kafka, HBase, and OpenTSDB and discuss the lessons learned along the way.
Let's get real: Acting on data in real time

Let's get real: Acting on data in real time

October 23, 2019

Big data is not limited to reporting and analysis; increasingly, companies are differentiating themselves by acting on data in real time. But what does "real time" really mean? Jack Norris discusses the challenges of coordinating data flows, analysis, and integration at scale to truly impact business as it happens.
Python scalability: A convenient truth

Python scalability: A convenient truth

October 21, 2019

Despite Python's popularity throughout the data-engineering and data science workflow, the principles behind its performance and scaling behavior are less understood. Travis Oliphant explains best practices and modern tools to scale Python to larger-than-memory and distributed workloads without sacrificing its ease of use or being forced to adopt heavyweight frameworks.
Scala and the JVM as a big data platform: Lessons from Apache Spark

Scala and the JVM as a big data platform: Lessons from Apache Spark

October 21, 2019

The success of Apache Spark is bringing developers to Scala. For big data, the JVM uses memory inefficiently, causing significant GC challenges. Spark's Project Tungsten fixes these problems with custom data layouts and code generation. Dean Wampler gives an overview of Spark, explaining ongoing improvements and what we should do to improve Scala and the JVM for big data.
Self-service, interactive analytics at multipetabyte scale in capital markets regulation on the cloud

Self-service, interactive analytics at multipetabyte scale in capital markets regulation on the cloud

October 20, 2019

Scott Donaldson and Matt Cardillo detail the security measures and system architecture needed to bring alive a multipetabyte data warehouse via interactive analytics and directed graphs from several trillions of market events, using HBase, EMR, Hive, Redshift, and S3 technologies in a cost-efficient manner.
The Internet of Things: How to do it. Seriously!

The Internet of Things: How to do it. Seriously!

October 20, 2019

The Internet of Things (IoT) continues to provide value and hold promise for both the consumer and enterprise alike. To succeed, an IoT project must concern itself with how to ingest data, build actionable models, and react in real time. Chris Rawles describes approaches to addressing these concerns through a deep dive into an interactive demo centered around classification of human activities.
Three principles for a data-centric security architecture on Hadoop to simplify your life

Three principles for a data-centric security architecture on Hadoop to simplify your life

October 19, 2019

Pratik Verma and Paulo Pereira share three security architecture principles for Hadoop to protect sensitive data without disrupting users: modifying requests to filter content makes security transparent to users; centralizing data-access decisions and distributing enforcement makes security scalable; and using metadata instead of files or tables ensures systematic protection of sensitive data.