User-based real-time product recommendations leveraging deep learning using Analytics Zoo on Apache Spark and BigDL

User-based real-time recommendation systems have become an important topic in ecommerce. Lu Wang, Nicole Kong, Guoqiong Song, and Maneesha Bhalla demonstrate how to build deep learning algorithms using Analytics Zoo with BigDL on Apache Spark and create an end-to-end system to serve real-time product recommendations.

User-based recommendation systems have become an important topic in ecommerce. Traditional approaches like rule-based systems can’t consider user-specific characters and the implicit relationship between users and products to provide customized recommendation solutions. The newly developed deep neural networks have shed light on a path to success by chaptering nonlinear relationships in the user-item dataset, leveraging user, product attributes, and user-item interaction history. With the huge dataset a big ecommerce company commands, a training model in the cloud is a more achievable and efficient solution. BigDL, a new distributed deep learning framework on Apache Spark, provides easy and seamlessly integrated big data and deep learning capabilities for big data users and data scientists. Analytics Zoo is an analytics and AI platform for Spark and BigDL; it helps users build and productionize deep learning apps for big data at scale. The deep learning algorithms in BigDL result in much better results compared to traditional recommendation algorithms. Lu Wang, Nicole Kong, Guoqiong Song, and Maneesha Bhalla explain how to build a user-based recommendation system with neural collaborative filtering and wide and deep models using Analytics Zoo with BigDL on Apache Spark in the cloud. They then demonstrate how to deploy the model and serve the real-time user-based recommendation on a website.