January 7, 2020

195 words 1 min read

Scaling AI Inference Workloads with GPUs and Kubernetes

Scaling AI Inference Workloads with GPUs and Kubernetes

Deep Learning (DL) is a computational intense form of machine learning that has revolutionize many fields including computer vision, automated speech recognition, natural language processing and artif …
Talk Title Scaling AI Inference Workloads with GPUs and Kubernetes
Speakers Renaud Gaubert (Software Engineer, NVIDIA), Ryan Olson (Solutions Architect, NVIDIA)
Conference KubeCon + CloudNativeCon North America
Conf Tag
Location Seattle, WA, USA
Date Dec 9-14, 2018
URL Talk Page
Slides Talk Slides
Video

Deep Learning (DL) is a computational intense form of machine learning that has revolutionize many fields including computer vision, automated speech recognition, natural language processing and artificial intelligence (AI). DL impacts every vertical market from automotive to healthcare to cloud, as a result, the training and deployment of Deep Neural Networks (DNNs) has shifted datacenter workloads from traditional CPUs to AI-specific accelerators like NVIDIA GPUs. Leveraging several popular CNCF projects such as Prometheus, Envoy, and gRPC, we will demonstrate an implementation of NVIDIA’s reference scale-out inference architecture, capable of delivering petaops per second of performance. This is a new and challenging problem in the datacenter and we will discuss these challenges and ways to optimize for service delivery metrics (latency/throughput), cost, and redundancy.

healthcare grpc intel automotive gpu envoy nvidia automated speech network computer vision intelligence metrics dl ai performance health dnn prometheus deep learning machine learning cloud datacenter kubernetes neural network artificial intelligence
comments powered by Disqus
Learning how to design automatically updating AI with Apache Kafka and Deeplearning4j

Learning how to design automatically updating AI with Apache Kafka and Deeplearning4j

December 7, 2019

Jason Bell offers an overview of a self-learning knowledge system that uses Apache Kafka and Deeplearning4j to accept data, apply training to a neural network, and output predictions. Jason covers the system design and the rationale behind it and the implications of using a streaming data with deep learning and artificial intelligence.
Operationalize deep learning models for fraud detection with Azure Machine Learning Workbench

Operationalize deep learning models for fraud detection with Azure Machine Learning Workbench

December 6, 2019

Advancements in computing technologies and ecommerce platforms have amplified the risk of online fraud, which results in billions of dollars of loss for the financial industry. This trend has urged companies to consider AI techniques, including deep learning, for fraud detection. Francesca Lazzeri and Jaya Mathew explain how to operationalize deep learning models with Azure ML to prevent fraud.
A high-performance system for deep learning inference and visual inspection

A high-performance system for deep learning inference and visual inspection

December 12, 2019

Moty Fania explains how Intel implemented an AI inference platform to enable internal visual inspection use cases and shares lessons learned along the way. The platform is based on open source technologies and was designed for real-time streaming and online actuation.
Smart diagnosis in healthcare with deep learning

Smart diagnosis in healthcare with deep learning

December 3, 2019

Deep learning with ConvNet in particular has emerged as a promising tool in medical research labs and diagnostic centers to help analyze images and scans, and systems are now surpassing human capability for manual inspection. Nishant Sahay explains how to apply deep learning to analyze high-end microscope images and X-ray scans to provide accurate diagnosis.
Sharded and Federated Prometheus Servers to Monitor Distributed Databases

Sharded and Federated Prometheus Servers to Monitor Distributed Databases

December 12, 2019

At eBay we have developed a geo-distributed transactional document store called NuData. It is deployed on Kubernetes. The current deployment has thousands of pods across three datacenters, and is moni …
Deep learning for recommender systems

Deep learning for recommender systems

December 11, 2019

In the last few years, deep learning has achieved significant success in a wide range of domains, including computer vision, artificial intelligence, speech, NLP, and reinforcement learning. However, deep learning in recommender systems has, until recently, received relatively little attention. Nick Pentreath explores recent advances in this area in both research and practice.