February 24, 2020

194 words 1 min read

Advanced model deployments with TensorFlow Serving

Advanced model deployments with TensorFlow Serving

Hannes Hapke leads a deep dive into deploying TensorFlow models within minutes with TensorFlow Serving and optimizing your serving infrastructure for optimal throughput.

Talk Title Advanced model deployments with TensorFlow Serving
Speakers Hannes Hapke (SAP ConcurLabs)
Conference O’Reilly TensorFlow World
Conf Tag
Location Santa Clara, California
Date October 28-31, 2019
URL Talk Page
Slides Talk Slides

TensorFlow Serving is one of the cornerstones in the TensorFlow ecosystem. It has eased the deployment of machine learning models tremendously and led to an acceleration of model deployments. Unfortunately, machine learning engineers aren’t familiar with the details of TensorFlow Serving, and they’re missing out on significant performance increases. Hannes Hapke provides a brief introduction to TensorFlow Serving, then leads a deep dive into advanced settings and use cases. He introduces advanced concepts and implementation suggestions to increase the performance of the TensorFlow Serving setup, which includes an introduction to how clients can request model meta-information from the model server, an overview of model optimization options for optimal prediction throughput, an introduction to batching requests to improve the throughput performance, an example implementation to support model A/B testing, and an overview of monitoring your TensorFlow Serving setup.

comments powered by Disqus