Building deep reinforcement learning applications on BigDL and Spark

Deep reinforcement learning is a thriving area and has wide applications in industry. Arsenii Mustafin shares his experience developing deep reinforcement learning applications on BigDL and Spark.

本讲话将用英语授课，同时会提供中文同声传译。中文版本摘要会在英文摘要下面给出。 Deep reinforcement learning is a thriving area—DeepMind’s AlphaGo, for instance, has drawn the attention of the entire world. But besides playing games, deep reinforcement learning (DRL) also has many practical applications in industry, such as autonomous driving, chatbots, financial investment, inventory management, and even recommendation systems. Although DRL applications are similar to supervised computer vision or natural language processing tasks, they are unique in many ways. For example, they have to interact with or explore the environment to obtain training samples along the optimization, and the method to improve the model is usually different from common supervised applications. BigDL, a well-developed deep learning library on Spark, is handy for big data users but has been mostly used for supervised and unsupervised machine learning. Arsenii Mustafin shares his experience developing deep reinforcement learning applications on BigDL and Spark, discussing extensions particularly for DRL algorithms (DQN, PG, PPO, Actor-Critic, etc.). You’ll get tips on how to build a RL application for your own use case. 深度增强学习（Deep ReinforcementLearning，DRL）是当今AI战场里一个蓬勃发展的领域。DeepMind的AlphaGO是DRL一个非常成功的应用，获得了全世界的关注。除了能玩游戏，DRL还能在多个行业有很多应用，比如无人驾驶、对话机器人、金融投资、库存管理、甚至是推荐系统。虽然DRL的应用和监督学习型的计算机视觉或自然语言处理任务在某些方面有相同点，但它在很多方面都是非常独特的。例如，DRL的应用必须要通过和环境的交互（探索）来获得训练样本并优化，而且DRL用于改进模型的方法和通常的监督学习型应用也有所不同。 在本演讲里，我们会分享我们使用BigDL/Spark来构建深度增强学习应用的经验。BigDL是一个基于Spark的完善的深度学习库，对大数据用户而言非常用以上手。通常BigDL被用于监督和无监督的机器学习。不过我们特意为DRL算法（例如，DQN、PG、PPO、Actor-Critic等算法）进行了扩展，实现了经典的DRL算法，并用它们构建了DRL的应用，还进行了性能调优。我们很高兴来分享在做这些的过程中学到的东西。希望我们的经验能帮助听众学会如何自己为它们的业务构建一个增强学习的应用。