January 21, 2020

220 words 2 mins read

IoT edge processing with Apache NiFi, Apache MiniFi, and multiple deep learning libraries

IoT edge processing with Apache NiFi, Apache MiniFi, and multiple deep learning libraries

Timothy Spann leads a hands-on deep dive into using Apache MiniFi with Apache MXNet and other deep learning libraries on edge devices.
Talk Title IoT edge processing with Apache NiFi, Apache MiniFi, and multiple deep learning libraries
Speakers TIMOTHY SPANN (Cloudera)
Conference Strata Data Conference
Conf Tag Make Data Work
Location New York, New York
Date September 11-13, 2018
URL Talk Page
Slides Talk Slides
Video

Timothy Spann leads a hands-on deep dive into using Apache MiniFi with Apache MXNet and other deep learning libraries on edge devices, such as Raspberry Pis with Movidius and the NVIDIA Jetson TX1. You’ll learn how to run deep learning models on edge devices and send images, GPS data, sensor data, and deep learning results if values exceed norms. Using S2S, data is sent to NiFi for further processing, additional TensorFlow processing, and data augmentation with weather and geolocation. A stream of data is landed as ORC files in HDFS with Hive tables on top. Processed data in the Avro format with a schema stored in Schema Registry is sent to Streaming Analytics Manager via Kafka, and additional processing and rules are processed to Druid endpoints. Visualization is shown in Zeppelin and Superset. Potential use cases for this solution include security camera monitoring, utility asset anomaly detection, and temperature and humdity filtering for devices.

kafka streaming apache raspberry security tensorflow mxnet analytics visualization hdfs anomaly detection deep learning nvidia monitoring use case jetson
comments powered by Disqus
Deep learning with TensorFlow and Spark using GPUs and Docker containers

Deep learning with TensorFlow and Spark using GPUs and Docker containers

December 10, 2019

In the past, you needed a high-end proprietary stack for advanced machine learning, but today, you can use open source machine learning and deep learning algorithms available with distributed computing technologies like Apache Spark and GPUs. Nanda Vijaydev and Thomas Phelan demonstrate how to deploy a TensorFlow and Spark with NVIDIA CUDA stack on Docker containers in a multitenant environment.
How BT delivers better broadband and TV using Spark and Kafka

How BT delivers better broadband and TV using Spark and Kafka

December 9, 2019

In the past year, British Telecom has added a streaming network analytics use case to its multitenant data platform. Phillip Radley demonstrates how the solution works and explains how it delivers better broadband and TV services, using Kafka and Spark on YARN and HDFS encryption.
Distributed deep learning with containers on heterogeneous GPU clusters

Distributed deep learning with containers on heterogeneous GPU clusters

November 26, 2019

Deep learning model performance relies on underlying data. Dong Meng offers an overview of a converged data platform that serves as a data infrastructure, providing a distributed filesystem, key-value storage and streams, and Kubernetes as orchestration layer to manage containers to train and deploy deep learning models using GPU clusters.
Real-time analytics and BI with data lakes and data warehouses using Kudu, HBase, Spark, and Kafka: Lessons learned

Real-time analytics and BI with data lakes and data warehouses using Kudu, HBase, Spark, and Kafka: Lessons learned

January 19, 2020

Mauricio Aristizabal shares lessons learned from migrating Impact's traditional ETL platform to a real-time platform on Hadoop (leveraging the full Cloudera EDH stack). Mauricio also discusses the company's data lake in HBase, Spark Streaming jobs (with Spark SQL), using Kudu for "fast data" BI queries, and using Kafka's data bus for loose coupling between components.
Using machine learning to drive intelligence at the edge

Using machine learning to drive intelligence at the edge

January 16, 2020

The focus on the IoT is turning increasingly to the edge, and the way to make the edge more intelligent is by building machine learning models in the cloud and pushing them back out to the edge. Dave Shuman and Bryan Dean explain how Cloudera and Red Hat executed this architecture at one of Europe's leading manufacturers, along with a demo highlighting this architecture.
Reproducible quantum chemistry in Jupyter

Reproducible quantum chemistry in Jupyter

January 6, 2020

In silico prediction of chemical properties has seen vast improvements in both veracity and volume of data but is currently hamstrung by a lack of transparent, reproducible workflows coupled with environments for visualization and analysis. Chris Harris offers an overview of a platform that uses Jupyter notebooks to enable an end-to-end workflow from simulation setup to visualizing the results.