January 13, 2020

271 words 2 mins read

Efficient neural network training on Intel Xeon-based supercomputers

Efficient neural network training on Intel Xeon-based supercomputers

Vikram Saletore and Luke Wilson discuss a collaboration between SURFSara and Intel to advance the state of large-scale neural network training on Intel Xeon CPU-based servers, highlighting improved time to solution on extended training of pretrained models and exploring how various storage and interconnect options lead to more efficient scaling.
Talk Title Efficient neural network training on Intel Xeon-based supercomputers
Speakers Vikram Saletore (Intel), Lucas Wilson (Dell EMC)
Conference Artificial Intelligence Conference
Conf Tag Put AI to Work
Location San Francisco, California
Date September 5-7, 2018
URL Talk Page
Slides Talk Slides
Video

Vikram Saletore and Luke Wilson discuss a collaboration between SURFSara and Intel as part of the Intel Parallel Computing Center initiative to advance the state of large-scale neural network training on Intel Xeon CPU-based servers. SURFSara and Intel evaluated a number of data and model parallel approaches and synchronous versus asynchronous SGD methods with popular neural networks, such as ResNet50 using large datasets on the TACC (Texas Advanced Computing Center) and Dell HPC supercomputers. Vikram and Luke share insights on several best-known methods, including CPU core, memory pinning, and hyperparameter tuning, that were developed to demonstrate top-one/top-five state-of-the-art accuracy at scale. They then detail real-world problems that can be solved by utilizing models efficiently trained at large-scale and present tests performed at Dell EMC on CheXNet, a Stanford University project that extends a DenseNet model pretrained on the large-scale ImageNet dataset to detect pathologies in chest X-ray images, including pneumonia. Vikram and Luke highlight improved time to solution on extended training of this pretrained model and the various storage and interconnect options that lead to more efficient scaling.

intel dataset large-scale network stanford neural network
comments powered by Disqus
Operationalize deep learning models for fraud detection with Azure Machine Learning Workbench

Operationalize deep learning models for fraud detection with Azure Machine Learning Workbench

December 6, 2019

Advancements in computing technologies and ecommerce platforms have amplified the risk of online fraud, which results in billions of dollars of loss for the financial industry. This trend has urged companies to consider AI techniques, including deep learning, for fraud detection. Francesca Lazzeri and Jaya Mathew explain how to operationalize deep learning models with Azure ML to prevent fraud.
Distributed deep learning in the cloud: Build an end-to-end application involving computer vision and geospatial data

Distributed deep learning in the cloud: Build an end-to-end application involving computer vision and geospatial data

January 13, 2020

High-resolution land cover maps help quantify long-term trends like deforestation and urbanization but are prohibitively costly and time intensive to produce. Mary Wahl and Banibrata De demonstrate how to use Microsofts Cognitive Toolkit and Azure cloud resources to produce land cover maps from aerial imagery by training a semantic segmentation DNNboth on single VMs and at scale on GPU clusters.
Evaluate deep Q-learning for sequential targeted marketing with 10-fold cross-validation

Evaluate deep Q-learning for sequential targeted marketing with 10-fold cross-validation

January 13, 2020

Jian Wu discusses an end-to-end engineering project to train and evaluate deep Q-learning models for targeting sequential marketing campaigns using the 10-fold cross-validation method. Jian also explains how to evaluate trained DQN models with neural network-based baseline models and shows that trained deep Q-learning models generally produce better-optimized long-term rewards.
How to use transfer learning to bootstrap image classification and question answering (QA)

How to use transfer learning to bootstrap image classification and question answering (QA)

January 11, 2020

Transfer learning enables you to use pretrained deep neural networks and adapt them for various deep learning tasks (e.g., image classification, question answering, and more). Join Wee Hyong Tok and Danielle Dean to learn the secrets of transfer learning and discover how to customize these pretrained models for your own use cases.
Neural Network Distiller: A PyTorch environment for neural network compression

Neural Network Distiller: A PyTorch environment for neural network compression

January 11, 2020

Neta Zmora offers an overview of Distiller, an open source Python package for neural network compression research. Neta discusses the motivation for compressing DNNs, outlines compression approaches, and explores Distiller's design and tools, supported algorithms, and code and documentation. Neta concludes with an example implementation of a compression research paper.
Scaling AI Inference Workloads with GPUs and Kubernetes

Scaling AI Inference Workloads with GPUs and Kubernetes

January 7, 2020

Deep Learning (DL) is a computational intense form of machine learning that has revolutionize many fields including computer vision, automated speech recognition, natural language processing and artif …