streaming

Building a sales AI platform: Key principles and lessons learned

Building a sales AI platform: Key principles and lessons learned

January 14, 2020

Moty Fania shares his experience implementing a sales AI platform that handles processing of millions of website pages and sifts through millions of tweets per day. The platform is based on unique open source technologies and was designed for real-time data extraction and actuation.
Executive Briefing: What it takes to use machine learning in fast data pipelines

Executive Briefing: What it takes to use machine learning in fast data pipelines

January 11, 2020

Your team is building machine learning capabilities. Dean Wampler demonstrates how to integrate these capabilities in streaming data pipelines so you can leverage the results quickly and update them as needed and covers challenges such as how to build long-running services that are very reliable and scalable and how to combine a spectrum of very different tools, from data science to operations.
From BI to big data; Or, There and back again

From BI to big data; Or, There and back again

January 10, 2020

Francesco Mucio shares the basic tools he and his team had to learn (or relearn) moving from the coziness of their database to the big world of Spark, cloud, distributed systems, and continuous applications. It was an unexpected journey that ended exactly where it started: with an SQL query.
Real-time SQL stream processing at scale with Apache Kafka and KSQL

Real-time SQL stream processing at scale with Apache Kafka and KSQL

January 7, 2020

Robin Moffatt walks you through the architectural reasoning for Apache Kafka and the benefits of real-time integration. You'll then build a streaming data pipeline using nothing but your bare hands, Kafka Connect, and KSQL.
Serverless for data and AI

Serverless for data and AI

January 6, 2020

What is serverless, and how can it be utilized for data analysis and AI? Avner Braverman outlines the benefits and limitations of serverless with respect to data transformation (ETL), AI inference and training, and real-time streaming. This is a technical talk, so expect demos and code.
Stream, stream, stream: Different streaming methods with Spark and Kafka

Stream, stream, stream: Different streaming methods with Spark and Kafka

January 6, 2020

NMC (Nielsen Marketing Cloud) provides customers (both marketers and publishers) with real-time analytics tools to profile their target audiences. To achieve that, the company needs to ingest billions of events per day into its big data stores in a scalable, cost-efficient way. Itai Yaffe explains how NMC continuously transforms its data infrastructure to support these goals.
The changing face of ETL: Event-driven architectures for data engineers

The changing face of ETL: Event-driven architectures for data engineers

January 6, 2020

Robin Moffatt discusses the concepts of events, their relevance to software and data engineers, and their ability to unify architectures in a powerful way. Join in to learn why analytics, data integration, and ETL fit naturally into a streaming world. Along the way, Robin will lead a hands-on demonstration of these concepts in practice and commentary on the design choices made.
Veo5G Project: A 5G Use Case for Network Data Analytics

Veo5G Project: A 5G Use Case for Network Data Analytics

January 6, 2020

This presentation will describe Gradiant's role in the VEO5G R&D project: provide data analytics for networking data in a 5G scenario.Gradiant researchers will share their experience working with the …
Turn devices into data scientistsat the edge

Turn devices into data scientistsat the edge

December 28, 2019

Todays approach to processing streaming data is based on legacy big-data centric architectures, the cloud, and the assumption that organizations have access to data scientists to make sense of it allleaving organizations increasingly overwhelmed. Simon Crosby shares a new architecture for edge intelligence that turns this thinking on its head.
Analytics Zoo: Distributed TensorFlow in production on Apache Spark

Analytics Zoo: Distributed TensorFlow in production on Apache Spark

December 27, 2019

Yuhao Yang and Jennie Wang demonstrate how to run distributed TensorFlow on Apache Spark with the open source software package Analytics Zoo. Compared to other solutions, Analytics Zoo is built for production environments and encourages more industry users to run deep learning applications with the big data ecosystems.
Apache Spark 2.4 and beyond

Apache Spark 2.4 and beyond

December 27, 2019

Xiao Li and Wenchen Fan offer an overview of the major features and enhancements in Apache Spark 2.4 and give insight into upcoming releases. Then you'll get the chance to ask all your burning Spark questions.
Executive Briefing: What it takes to use machine learning in fast data pipelines

Executive Briefing: What it takes to use machine learning in fast data pipelines

December 25, 2019

Your team is building machine learning capabilities. Dean Wampler demonstrates how to integrate these capabilities in streaming data pipelines so you can leverage the results quickly and update them as needed and covers challenges such as how to build long-running services that are very reliable and scalable and how to combine a spectrum of very different tools, from data science to operations.
Flink SQL in action

Flink SQL in action

December 24, 2019

Processing streaming data with SQL is becoming increasingly popular. Fabian Hueske explains why SQL queries on streams should have the same semantics as SQL queries on static data. He then shares a selection of common use cases and demonstrates how easily they can be addressed with Flink SQL.
From flat files to deconstructed databases: The evolution and future of the big data ecosystem

From flat files to deconstructed databases: The evolution and future of the big data ecosystem

December 24, 2019

Big data infrastructure has evolved from flat files in a distributed filesystem to an efficient ecosystem to a fully deconstructed and open source database with reusable components. Julien Le Dem discusses the key open source components of the big data ecosystem and explains how they relate to each other and how they make the ecosystem more of a database and less of a filesystem.
Serverless for data and AI

Serverless for data and AI

December 21, 2019

What is serverless, and how can it be utilized for data analysis and AI? Avner Braverman outlines the benefits and limitations of serverless with respect to data transformation (ETL), AI inference and training, and real-time streaming. This is a technical talk, so expect demos and code.
Spark-PMoF: Accelerating big data analytics with Persistent Memory over Fabric

Spark-PMoF: Accelerating big data analytics with Persistent Memory over Fabric

December 21, 2019

Yuan Zhou, Haodong Tang, and Jian Zhang offer an overview of Spark-PMOF and explain how it improves Spark analytics performance.
The magic behind your Lyft ride prices: A case study on machine learning and streaming

The magic behind your Lyft ride prices: A case study on machine learning and streaming

December 20, 2019

Rakesh Kumar and Thomas Weise explore how Lyft dynamically prices its rides with a combination of various data sources, ML models, and streaming infrastructure for low latency, reliability, and scalabilityallowing the pricing system to be more adaptable to real-world changes.
Kubernetes the New Research Platform

Kubernetes the New Research Platform

November 13, 2019

Academic research institutions are at a precipice. They have historically been constrained to supporting classic job style workloads. With the growth of new workflow practices such as streaming data …
Edgility - 5G Orchestration in Serverless-Edge Cloud-Native Environments

Edgility - 5G Orchestration in Serverless-Edge Cloud-Native Environments

October 30, 2019

Edge computing is becoming the major component in 5G networks. While edge computing provides an u-low-latency solution, it brings with it other challenges. On one hand, edge clouds are small, and thei …
No More Chaos: Audit and Inspect Kubernetes at Scale

No More Chaos: Audit and Inspect Kubernetes at Scale

September 28, 2019

Accuracy in fault detection and efficiency of issue analysis are important for availability and stability of Kubernetes clusters.While there are huge number of resources, events, and metrics in Kubern …
Flowchain Ecosystem: The Solutions for AI Model Training and Distributed Network Storage of Streaming Data

Flowchain Ecosystem: The Solutions for AI Model Training and Distributed Network Storage of Streaming Data

September 19, 2019

Flowchain is a hybrid blockchain containing one public blockchain and multiple private blockchains. To run the hybrid blockchain in real-time without compromising the security, PPKI (Pseudonymous Publ …