Energy usage is a significant part of daily life, so the ability to monitor this use offers a number of benefits, from cost savings to improved safety. A key challenge is the lack of labeled data. Yiqun Hu shares a new solution: a RNN-based network trained to learn good features from unlabeled data.

Prometheus is great when it comes to monitoring and alerting. But the long term storage opportunities are comparatively weak compared to related time series databases (missing data distribution, shard …

Moty Fania shares Intels IT experience implementing an on-premises IoT platform for internal use cases. The platform was designed as a multitenant platform with built-in analytical capabilities and based on open source big data technologies and containers. Moty highlights the lessons learned from this journey with a thorough review of the platforms architecture.

Automated anomaly detection in production using simple data science techniques enables you to more quickly identify an issue and reduce the time it takes to get customers out of an outage. Tuli Nivas shows how to apply simple statistics to change how performance data is viewed and how to easily and effectively identify issues in production.

For the past year, LinkedIn has been running and iteratively improving Luminol, its anomaly detection system for real user monitoring data. Ritesh Maheshwari and Yang Yang offer an overview of Luminol, focusing on how to build a low-cost end-to-end system that can leverage any algorithm, and explain lessons learned and best practices that will be useful to any engineering or operations team.

Moty Fania shares Intels IT experience implementing an on-premises IoT platform for internal use cases. The platform was based on open source big data technologies and containers and was designed as a multitenant platform with built-in analytical capabilities. Moty highlights the key lessons learned from this journey and offers a thorough review of the platforms architecture.

Datadog provides outlier and anomaly detection functionality to automatically alert on metrics that are difficult to monitor using thresholds alone. Homin Lee discusses the algorithms and open source tools Datadog uses and lessons learned from using these alerts on its own systems, along with some real-life examples on how to avoid false positives and negatives.

The Defense Advanced Research Projects Agency (DARPA) is synonymous with transformational change, developing the seeming impossible into the practical. Matthew van Adelsberg demonstrates how collaborative teams of SMEs, data scientists, and engineers have been organized to achieve DARPA hard results for nearly a decade and offers insights into how companies can do the same.

Building a real-time monitoring service that handles millions of custom events per second while satisfying complex rules, varied throughput requirements, and numerous dimensions simultaneously is a complex endeavor. Sumeet Singh and Mridul Jain explain how Yahoo approached these challenges with Apache Storm Trident, Kafka, HBase, and OpenTSDB and discuss the lessons learned along the way.