Operational analytic databases are a new class of data systems that are emerging to store and analyze various types of operational data, including netflows. Popula …

Recently there was a blog post on Linkedin talking about the skills required to be a successful network engineer in the future. While the skills listed were all re …

Have you ever tuned a Spark or MR job? If the answer is yes, you already know how difficult it is to tune more than hundred parameters to optimize the resources used. Manoj Kumar, Pralabh Kumar, and Arpan Agrawal offer an overview of TuneIn, an auto-tuning tool developed to minimize the resource usage of jobs. Experiments have shown up to a 50% reduction in resource usage.

John Dohoney and Kaitlin Carter walk you through deploying the SMACK stack on DC/OS. This architecture enables you to create modern streaming applications that make use of NoSQL databases with Cassandra and message streaming with Apache Kafka using analytics streaming with Apache Spark, all running under Apache Mesos implemented with Akka streaming and asynchronous Java libraries under DC/OS.

Burcu Baran, Wei Di, Michael Li, and Chi-Yi Kuan walk you through the big data analytics and data science lifecycle and share their experience and lessons learned leveraging advanced analytics and machine learning techniques such as predictive modeling to drive and grow business at LinkedIn.

Spark applications need to be well tuned so that individual applications run quickly and reliably and cluster resources are efficiently utilized. Edwina Lu, Ye Zhou, and Min Shen outline a fast, reliable, and automated process used at LinkedIn for tuning Spark applications, enabling users to quickly identify and fix problems.

LinkedIn runs more than 1,800+ Kafka brokers that deliver more than two trillion messages a day. Running Kafka at such a scale makes automated operations a necessity. Jiangjie Qin shares lessons learned from operating Kafka at scale with minimum human intervention.