Bargava Subramanian and Harjinder Mistry share data engineering and machine learning strategies for building an efficient real-time recommendation engine when the transaction data is both big and wide. They also outline a novel way of generating frequent patterns using collaborative filtering and matrix factorization on Apache Spark and serving it using Elasticsearch in the cloud.

In the past few years, there has been a proliferation of production-ready open source databases, giving developers and operators more choices than ever. Joseph Lynch explores how Yelp has combined complimentary data stores to provide a powerful data tier for our developers. Along the way, Joseph shares lessons learned about deployment, configuration, and monitoring from a production environment.

David Talby and Claudiu Branzan lead a live demo of an end-to-end system that makes nontrivial clinical inferences from free-text patient records. Infrastructure components include Kafka, Spark Streaming, Spark, Titan, and Elasticsearch; data science components include custom UIMA annotators, curated taxonomies, machine-learned dynamic ontologies, and real-time inferencing.

There's a software culture revolution inside the government. User-centric design, lean, and agile are first citizens, but the increased velocity in development and testing requires a change in the way the government does deployment, security, and compliance. Diego Lapiduz shows how the cloud.gov team is building tools to achieve faster deployments and continuous compliance in a secure environment.

Currently, multitenancy in Hadoop is limited to organizations running separate Hadoop clusters, and the secure sharing of resources is achieved using virtualization or containers. Jim Dowling describes how HopsWorks enables organizations to securely share a single Hadoop cluster using projects and a new metadata layer that enables protection domains while still allowing data sharing.