metrics

Distributed real-time highly available stream processing

Distributed real-time highly available stream processing

January 1, 2020

Yu-Xi Lim and Michal Wegrzyn outline a high-throughput distributed software pattern capable of processing event streams in real time. At its core, the pattern relies on functional reactive programming idioms to shard and splice state fragments, ensuring high horizontal scalability, reliability, and high availability.
Delivering Services at DigitalOcean

Delivering Services at DigitalOcean

December 30, 2019

Though kubernetes itself provides a rich collection of resources to deploy services it also comes with a bundle of complexity. This is where DOCC (DigitalOcean Control Center) comes in. This talk will …
Prometheus Is Good for Your Small Startup

Prometheus Is Good for Your Small Startup

December 29, 2019

ShuttleCloud is a small startup specialized in email and contacts migrations. We developed a reliable migration platform in high availability used by clients like Gmail, Gcontacts and Comcast. For exa …
Smart cities, the smart grid, the IoT, and big data

Smart cities, the smart grid, the IoT, and big data

December 29, 2019

Smart cities and the electricity smart grid have become leading examples of the IoT, in which distributed sensors describe mission-critical behavior by generating billions of metrics daily. Mark Donsky and Syed Rafice show how smart utilities and cities rely on Hadoop to capture, analyze, and harness this data to increase safety, availability, and efficiency across the entire electricity grid.
Suffering Withdrawal; an automated approach to connectivity evaluation

Suffering Withdrawal; an automated approach to connectivity evaluation

December 28, 2019

Todays routers generally make themselves more- or less-attractive to transit traffic through operators manipulation of their interfaces IGP metrics or overload s …
The Mushroom Cloud Effect or What Happens When Containers Fail?

The Mushroom Cloud Effect or What Happens When Containers Fail?

December 28, 2019

Micro service architectures result in up to 20 times larger environments than their monolithic counterparts. In such big and interconnected environments container metrics will tell you about infrastru …
Data Center Track - How Good is Your Data Center?

Data Center Track - How Good is Your Data Center?

December 25, 2019

The Data Center Track at NANOG 66, led by Gabe Cole strives to outline structures and tools to help organizations develop a framework for evaluating data center p …
Launching products at massive scale: The DevOps way

Launching products at massive scale: The DevOps way

December 22, 2019

Everything changes at scale. Launching products at a scale of 1+ billion users requires a massive cross-team, cross-functional, coordinated effort, and business, engineering, and cultural challenges must be overcome. Kishore Jalleda and Gopal Mor explain how they have applied DevOps best practices at scale to successfully launch several high-profile products at Yahoo.
The promise of Push

The promise of Push

December 20, 2019

HTTP/2 Push is the next frontier of web performance. Colin Bendell explores Push's possibilities and pitfalls as well as new metrics to measure the Push opportunity. Colin discusses when and how to use Push, testing, and several advanced techniques, reviews real-world experimentation, and shares recommendations on how to prepare for the future with Push support.
Architecting for change: LinkedIn's new data ecosystem

Architecting for change: LinkedIn's new data ecosystem

December 16, 2019

Shirshanka Das and Yael Garten describe how LinkedIn redesigned its data analytics ecosystem in the face of a significant product rewrite, covering the infrastructure changes, such as client-side activity tracking, a unified reporting platform, and data virtualization techniques to simplify migration, that enable LinkedIn to roll out future product innovations with minimal downstream impact.
Beyond Hadoop at Yahoo: Interactive analytics with Druid

Beyond Hadoop at Yahoo: Interactive analytics with Druid

December 16, 2019

Himanshu Gupta explains why Yahoo has been increasingly investing in interactive analytics and how it leverages Druid to power a variety of internal- and external-facing data applications.
Creating and evaluating a distance measure

Creating and evaluating a distance measure

December 14, 2019

Whether we're talking about spam emails, merging records, or investigating clusters, there are many times when having a measure of how alike things are makes them easier to work with (e.g., with unstructured data that isn't incorporated into your data models). Melissa Santos offers a practical approach to creating a distance metric and validating with business owners that it provides value.
Evaluating models for a needle in a haystack: Applications in predictive maintenance

Evaluating models for a needle in a haystack: Applications in predictive maintenance

December 13, 2019

In the realm of predictive maintenance, the event of interest is an equipment failure. In real scenarios, this is usually a rare event. Unless the data collection has been taking place over a long period of time, the data will have very few of these events or, in the worst case, none at all. Danielle Dean and Shaheen Gauher discuss the various ways of building and evaluating models for such data.
How a Spark-based feature store can accelerate big data adoption in financial services

How a Spark-based feature store can accelerate big data adoption in financial services

December 12, 2019

Kaushik Deka and Phil Jarymiszyn discuss the benefits of a Spark-based feature store, a library of reusable features that allows data scientists to solve business problems across the enterprise. Kaushik and Phil outline three challenges they facedsemantic data integration within a data lake, high-performance feature engineering, and metadata governanceand explain how they overcame them.
Winning with data: How ThredUp, Twilio, and Warby Parker use data to build advantage

Winning with data: How ThredUp, Twilio, and Warby Parker use data to build advantage

December 7, 2019

Daniel Mintz dives into case studies from three companiesThredUp, Twilio, and Warby Parkerthat use data to generate sustainable competitive advantages in their industries.
Performance anomaly detection at scale (sponsored by Salesforce)

Performance anomaly detection at scale (sponsored by Salesforce)

November 30, 2019

Automated anomaly detection in production using simple data science techniques enables you to more quickly identify an issue and reduce the time it takes to get customers out of an outage. Tuli Nivas shows how to apply simple statistics to change how performance data is viewed and how to easily and effectively identify issues in production.
Employee burnout and its often overlooked impact on productivity

Employee burnout and its often overlooked impact on productivity

November 26, 2019

Employee burnout is an overlooked anchor dragging down productivity and employee engagement in our industry. Courtney Kissler highlights some tactics and metrics to help leaders proactively address this issue.
Measuring ad blocker impact on site performance

Measuring ad blocker impact on site performance

November 25, 2019

Users' ad blockers are impacting your site's perceived performance, but measuring the impact of ad blockers on actual and perceived performance can be difficult. Karan Kumar offers an overview of new testing he has created that measures the overall impact ad blockers have on the quality of user experience and performance across a number of sites.
The wild west of media performance: A Vox story

The wild west of media performance: A Vox story

November 22, 2019

A little over a year ago, Vox Media created a dedicated performance team, which immediately set out to make all Vox Media sites as fast as possibleand has since made significant progress. Ian Carrico and Jason Ormand discuss what the team has done, how it did it, and what it's still working on.
Using machine learning to determine drivers of bounce and conversion

Using machine learning to determine drivers of bounce and conversion

November 21, 2019

Google partnered with SOASTA to train a machine-learning model on a large sample of real-world performance, conversion, and bounce data. Patrick Meenan and Tammy Everts offer an overview of the resulting modelable to predict the impact of performance work and other site metrics on conversion and bounce rates.
Your hero images need you: Save the day with HTTP/2 image loading

Your hero images need you: Save the day with HTTP/2 image loading

November 21, 2019

Tobias Baldauf explains how to use HTTP/2's superpowers to optimize image delivery, thereby increasing the perceived performance of your page, reducing load times, and driving conversions.
Pixels and place: What online experiences can borrow from offline spaces and vice versa

Pixels and place: What online experiences can borrow from offline spaces and vice versa

November 17, 2019

The metaphors used online have always borrowed heavily from the offline world, but as our online and offline worlds converge, the biggest opportunities for innovative experiences will come from blending them intentionally. Kate ONeill examines how the meaning and understanding of place relates to identity, culture, and intent and how we can shape our audiences' experiences more meaningfully.
Detecting outliers and anomalies in real-time at Datadog

Detecting outliers and anomalies in real-time at Datadog

November 13, 2019

Datadog provides outlier and anomaly detection functionality to automatically alert on metrics that are difficult to monitor using thresholds alone. Homin Lee discusses the algorithms and open source tools Datadog uses and lessons learned from using these alerts on its own systems, along with some real-life examples on how to avoid false positives and negatives.
Managing a large open source community with smart tools

Managing a large open source community with smart tools

November 8, 2019

Increasing your online presence in the open source community involves more than just writing and committing code. We need to create safe places to collaborate and communicate freely in order to involve more people than just our closest team members. Jonas Rosland and Stephanie Carlson outline a few of the tools they use to tackle this big task and discuss failures, successes, and lessons learned.
Road traffic analysis and agile architectures

Road traffic analysis and agile architectures

October 29, 2019

Kristoffer Dyrkorn outlines an infrastructure that reduces the steep cost of road construction and maintenance. The system provides high-quality and near real-time information by integrating an unusual combination of sensors, devices, protocols, and software. Kristoffer discusses architectural challenges met and choices and mistakes made while trying to keep the architecture receptive to change.
How to use your data science team: Becoming a data-driven organization

How to use your data science team: Becoming a data-driven organization

October 24, 2019

Youve decided you need data scientists. You know who to hire. Now, what do you do with them? Yael Garten offers examples of how companies like LinkedIn use data to make business and product decisions. Yael reviews the spectrum of data science, and discusses the culture, process and tools needed to transform companies into data-driven organizations.
Designing with data

Designing with data

October 12, 2019

The data shows that we can't escape data. Whether designer, developer, or researcher, this workshop is for everyone who wants to learn how to get more out of the ever-growing mountain of data to create a better user experience. Pamela Pavliscak explains how to combine data from analytics, social media, public datasets, diaries, usability tests, and more to inform design.