Bargava Subramanian and Harjinder Mistry share data engineering and machine learning strategies for building an efficient real-time recommendation engine when the transaction data is both big and wide. They also outline a novel way of generating frequent patterns using collaborative filtering and matrix factorization on Apache Spark and serving it using Elasticsearch in the cloud.

Data is all sales and marketing. The reality of data work is pain. Most data projects fail and are horrible experiences to work on. Phil Harvey explains that data is just too hardthe world needs to talk about real challenges so that we can start tackling them to deliver data projects that work. This is DataOps; there will be tears before bedtime.

Data science insights are undoubtedly transforming organizations, but the challenges of setting up and maintaining a data science stack that generates those insights are rarely discussed. Elaine Lee describes how Avants Data Engineering team built a system with open source projects, centered around Docker, to support data science R&D, continuous integration, and scaling in production.

Despite Python's popularity throughout the data-engineering and data science workflow, the principles behind its performance and scaling behavior are less understood. Travis Oliphant explains best practices and modern tools to scale Python to larger-than-memory and distributed workloads without sacrificing its ease of use or being forced to adopt heavyweight frameworks.