large-scale

Doing DropBox the Cloud Native Way

January 23, 2020

Cloud Native architecture has slowly become the default way to build robust, scalable applications. How would you rebuild a large scale storage service such as Dropbox (please note Dropbox is just an …

IPv6, the DNS and Big Packets

January 23, 2020

We are now at that point in the transition to IPv6 when we are hearing some ISPs talk about an IPv6-only service in the near future. But are we really ready? This …

Inside the bad actor's studio

January 22, 2020

Using research from more than one billion users, 500 billion events, and 50 million malicious accounts collected from global online services, Julian Wong details some of the sophisticated attack techniques being used by modern day online criminals and demonstrates how these types of attacks can be detected and mitigated by leveraging artificial intelligence.

Full Lifecycle Lessons from a Large Scale Government OpenStack Private Cloud

January 20, 2020

This talk will draw on lessons learnt through the full lifecycle of large scale government OpenStack private cloud, from design and implementation, through to operation and upgrade. How to bring into …

Interconnection Track: Creating a Centralized Database for Peering and Colocation Services

January 20, 2020

Everyone in the industry needs access to reliable and accurate colocation and network data. PeeringDB is the most commonly used source for this information but its …

Leading By Experience: The Unvarnished Truth about Community Leadership

January 19, 2020

As community manager for a large-scale collaborative project involving over 20 organizations, many of whom compete directly with each other, I have learned a lot of lessons and made a huge number of m …

Network Telemetry at Yahoo!

January 18, 2020

Providing 1 billion monthly active users with responsive, rich applications requires a large scale network. Locked within processes running on network devices are …

Open Undersea Cable Systems

January 18, 2020

Undersea bandwidth demand continues to increase for large-scale cloud providers, driven by phenomenal growth of cloud services and the globally distributed nature …

Watch out! The nanoservices are comIng.

January 15, 2020

Welcome to the world of nanoservices: smaller than a microservice, bigger than a function, they are the perfect unit of software. Nanoservices are flexible, manageable, and scalable and a great way to do serverless computing. Matthew Clark explains how to get nanoservices right, drawing on his experience at the BBC, which now has over a thousand in production.

Testing Software Performance and Scalability Using Containers

January 13, 2020

Software performance and scalability is critical to the success of an enterprise software and a great amount of effort and resources are spent in testing the performance and scalability for various us …

Automated Out-Of-Band Management with Ansible and Redfish

January 2, 2020

Ansible is an open source automation engine that automates complex IT tasks. It is a one-to-many agentless mechanism where complex deployment tasks can be controlled and monitored from a central contr …

How to Automate a Large Scale Government OpenStack Private Cloud

January 2, 2020

Lightweight Containerization at Facebook

January 1, 2020

In Facebook's new container system we started to heavily utilize Btrfs, cgroups2 and systemd. The combination of these tools and some additional internal code allowed us to create a lightweight, fast …

MacroBase: A search engine for fast data streams

December 31, 2019

Sahaana Suri offers an overview of MacroBase, a new analytics engine from Stanford designed to prioritize the scarcest resource in large-scale, fast-moving data streams: human attention. MacroBase allows reconfigurable, real-time root-cause analyses that have already diagnosed issues in production streams in mobile, data center, and industrial applications.

Mistakes were made, but not by us: Lessons from a year of supporting Apache Kafka

December 31, 2019

Dustin Cote shares his experience troubleshooting Apache Kafka in production environments and explains how to avoid pitfalls like message loss or performance degradation in your environment.

Monitoring At Scale: What was Recently Done and Whats Next in oVirt

December 31, 2019

Monitoring the state of high number of running virtual machine instances is crucial for effective management of large-scale virtual environments. It provides invaluable information for system administ …

Securely building deep learning models for digital health data

December 30, 2019

Josh Patterson, Vartika Singh, David Kale, and Tom Hanlon walk you through interactively developing and training deep neural networks to analyze digital health data using the Cloudera Workbench and Deeplearning4j (DL4J). You'll learn how to use the Workbench to rapidly explore real-world clinical data, build data-preparation pipelines, and launch training of neural networks.

The journey to Einstein: Building a multitenancy AI platform that powers hundreds of thousands of businesses

December 28, 2019

Salesforce recently released Einstein, which brings AI into its core platform to power every business. The secret behind Einstein is an underlying platform that accelerates AI development at scale for both internal and external data scientists. Simon Chan shares his experience building this unified platform for a multitenancy, multibusiness cloud enterprise.

How to escape saddle points efficiently

December 25, 2019

Many new theoretical challenges have arisen in the area of gradient-based optimization for large-scale data analysis, driven by the needs of applications and the opportunities provided by new hardware and software platforms. Michael Jordan shares recent research on the avoidance of saddle points in high-dimensional nonconvex optimization.

Using Docker Containers to Serve Deep Learing Predictions at Booking.com

December 24, 2019

Each day, over 1.2 million room nights are reserved on Booking.com. That gives us access to huge amount of data which we can utilise in order to provide a better experience to our customers. We under …

Conversational AI at large scale

December 19, 2019

There has been a quantum leap in the performance of conversational AI. From speech recognition to machine translation and language understanding, deep learning made its mark. However, scaling and productizing these breakthroughs remains a big challenge. Yishay Carmiel shares techniques and tips on how to take advantage of large datasets, accelerate training, and create an end-to-end product.

Your Careers Biggest Ally: You

December 17, 2019

Some things are under our control. And some things arent. Lifes like that for every one of us. For those belonging to an underrepresented demographic however, age-old societal beliefs about money, f …

Nomad and next-generation application architectures

December 15, 2019

Armon Dadgar offers an overview of Nomad, an application scheduler designed for both long-running services and batch jobs. Along the way, Armon explores the benefits of using schedulers for empowering developers and increasing resource utilization and how schedulers enable new next-generation application architectures.

Orchestrating chaos: Applying database research in the wild

December 14, 2019

Lineage-driven fault injection (LDFI), a novel approach to automating failure testing, can greatly reduce the number of faults that must be explored via fault injection. Peter Alvaro explores LDFIs theoretical roots in the database research notion of provenance and presents early results from the field and opportunities for near- and long-term future research.

Unified Monitoring of Containers and Microservices [I]

December 14, 2019

Microservices are become critical for enterprise strategy towards simplifying their IT landscape. For a successful journey of microservice adoption, Container management, DevOps and Monitoring play an …

Using Containers for Continuous Integration and Continuous Delivery [I]

December 14, 2019

Building and testing is a great use case for containers, both due to the dynamic and isolation aspects, but it increases complexity when scaling to multiple nodes and clusters. Jenkins is an example …

Scaling a user delivery network for real-time audience targeting

December 13, 2019

Adam Shepard peels back the covers on a user delivery networka worldwide distributed data store powering over 80 billion transactions a day at millisecond speed. Join in to learn about eventually consistent data architectures, tiered and hybrid storage layers, and what it takes to manage that much data at scale.

Would You Like Some Tracing With Your Monitoring?

December 13, 2019

Understanding how your microservices based application is executing in a highly distributed and elastic cloud environment can be complicated. Distributed tracing has emerged as an invaluable technique …