Deploying machine learning models on the edge

When IoT meets AI, a new round of innovations begins. Yan Zhang and Mathew Salvaris examine the methodology, practice, and tools around deploying machine learning models on the edge. They offer a step-by-step guide to creating an ML model using Python, packaging it in a Docker container, and deploying it as a local service on an edge device as well as deployment on GPU-enabled edge devices.

When IoT meets AI, a new round of innovations on big data, cloud computing, and intelligent edge begins. By 2020, it’s estimated that 250 petabytes of data will be generated by personal or enterprise IoT devices every day. Edge computing is well suited to handle this data, because it provides a means to collect and process data at local computing devices rather than in the cloud or a remote data center. It has two key benefits to IoT applications: a real-time analysis of data and reduced data transmission to the cloud. Therefore, IoT devices incur less latency and react more quickly to changes in status. As part of edge computing, intelligent edge aims to bring predictive analytics on the edge devices. Yan Zhang and Mathew Salvaris explore the methodology, practice, and tools around deploying machine learning models on the edge, offering a step-by-step guide to creating a pretrained ML model using Python, packaging it in a Docker container, and deploying it as a local service on an edge device. They outline how to test and verify each step and discover the gotchas you may encounter. You’ll see a demo of how to make calls to the deployed service to make predictions on a predeployed edge device. Yan and Mathew also discuss the consideration of deployment on GPU-enabled edge devices as well as how the edge devices can be managed in a centralized way in the cloud. Such a strategy makes it easy to train and retrain ML models in the cloud and to deploy the trained models on the multiple edge devices at the same time.