How Microsoft predicts churn of cloud customers using deep learning and explains those predictions in an interpretable way

Although deep learning has proved to be very powerful, few results are reported on its application to business-focused problems. Feng Zhu and Val Fontama explore how Microsoft built a deep learning-based churn predictive model and demonstrate how to explain the predictions using LIMEa novel algorithm published in KDD 2016to make the black box models more transparent and accessible.

Churn prediction and prevention is a critical component of CRM for Microsoft’s cloud business. Since churn is a rare event and churn patterns may vary significantly across customers, predicting churn is a challenging task when using conventional machine-learning techniques. On the other hand, Microsoft has massive, rich customer usage and billing data, which enables it to exploit advanced machine-learning techniques to discover the complex usage patterns for churn. Feng Zhu and Val Fontama explore how Microsoft’s cloud business built a deep learning-based churn predictive model in partnership with the Deep Learning team at Microsoft Research. The model identifies which customers are at high risk of churning from Microsoft Cloud. In order to fully utilize customer data, the team built a hybrid deep learning architecture by using deep DNN layers and deep RNN (LSTM) layers, which can take both static features and dynamic time series data as inputs. Compared to the current churn model in production, the deep learning model significantly improved prediction accuracy and demonstrated higher business impact. In addition to prediction accuracy, another challenge in churn prediction is how to explain the model and predictions to end users with no machine-learning background. End users (i.e., marketing and sales teams) need to understand the model and predictions in order adopt it and take actions on customers. Since deep learning models (as well as random forest models) are black box models, explaining why the score for a specific customer is high or low in an interpretable way is a challenge. Feng and Val demonstrate how to use LIME, a new algorithm published in KDD 1016, to explain the predictions of any classifier or regressor in a faithful way by approximating it locally with an interpretable model. Using this algorithm, you can explain how different features contribute to the predicted churn score (from the DL model) for each individual customer. Topics include: