api

Who Needs Network Management in a Cloud Native Environment?

Who Needs Network Management in a Cloud Native Environment?

January 3, 2020

Network management (and virtual network in particular) is hard. Cloud app developers find themselves dealing with too many options and too many settings, which make no sense. This is because Cloud API …
Automated Out-Of-Band Management with Ansible and Redfish

Automated Out-Of-Band Management with Ansible and Redfish

January 2, 2020

Ansible is an open source automation engine that automates complex IT tasks. It is a one-to-many agentless mechanism where complex deployment tasks can be controlled and monitored from a central contr …
Lessons from an AWS migration

Lessons from an AWS migration

January 1, 2020

if(we)'s batch event processing pipeline is different from yours, but the process of migrating it from running in a data center to running in AWS is likely pretty similar. Chris Mills explains what was easier than expected, what was harder, and what the company wished it had known before starting the migration.
High Performance Microservices on Linux with Apache Thrift

High Performance Microservices on Linux with Apache Thrift

December 31, 2019

The world is rapidly adopting cloud native approaches to software development, reaping the synergies produced by combining microservices, Linux container technology and dynamic application orchestrati …
OpenSDS and Storage Transformation for Cloudification at Vodafone

OpenSDS and Storage Transformation for Cloudification at Vodafone

December 30, 2019

OpenSDS is an open-source project created to address the storage challenges, particularly in scale-out cloud native environments, with heterogeneous storage platforms. The project promotes the use of …
SETL: An efficient and predictable way to do Spark ETL

SETL: An efficient and predictable way to do Spark ETL

December 30, 2019

Common ETL jobs used for importing log data into Hadoop clusters require a considerable amount of resources, which varies based on the input size. Thiruvalluvan M G shares a set of techniquesinvolving an innovative use of Spark processing and exploiting features of Hadoop file formatsthat not only make these jobs much more efficient but also work well with fixed amounts of resources.
Spark camp: Apache Spark 2.0 for analytics and text mining with Spark ML

Spark camp: Apache Spark 2.0 for analytics and text mining with Spark ML

December 30, 2019

Brooke Wenig introduces you to Apache Spark 2.0 core concepts with a focus on Spark's machine learning library, using text mining on real-world data as the primary end-to-end use case.
Stream analytics with SQL on Apache Flink

Stream analytics with SQL on Apache Flink

December 30, 2019

Although the most widely used language for data analysis, SQL is only slowly being adopted by open source stream processors. One reason is that SQL's semantics and syntax were not designed with streaming data in mind. Fabian Hueske explores Apache Flink's two relational APIs for streaming analyticsstandard SQL and the LINQ-style Table APIdiscussing their semantics and showcasing their usage.
The columnar roadmap: Apache Parquet and Apache Arrow

The columnar roadmap: Apache Parquet and Apache Arrow

December 29, 2019

Julien Le Dem explains how Parquet is improving at the storage level, with metadata and statistics that will facilitate more optimizations in query engines in the future, how the new vectorized reader from Parquet to Arrow enables much faster reads by removing abstractions, and how standard Arrow-based APIs are paving the way to breaking the silos of big data.
Everything You Always Wanted to Know About Object Storage

Everything You Always Wanted to Know About Object Storage

December 28, 2019

The rapid growth of unstructured data is fueling the need for a next generation storage thats flexible, economical, and scalable enough to handle the petabytes of data being created every day. Object …
Panel Discussion: If APIs are like Snowflakes, How Do You Make them More "Standardized" - or Do You? - Moderated by Jeff Borek, IBM

Panel Discussion: If APIs are like Snowflakes, How Do You Make them More "Standardized" - or Do You? - Moderated by Jeff Borek, IBM

December 26, 2019

APIs form the connecting glue between modern applications and are used to connect third-party data services, public/private data sources, and other applications. The Open API Initiative (OAI) came tog …
Serverless - Is It For Your Organization?

Serverless - Is It For Your Organization?

December 26, 2019

This talk will examine the Serverless phenomenon, popularized by AWS Lambda but now proposed by many public cloud providers such as Google and Microsoft. Well look at what Serverless is, what advant …
The Linux Cryptographic API for Fun and Profit - Gilad Ben-Yossef, Arm Holdings

The Linux Cryptographic API for Fun and Profit - Gilad Ben-Yossef, Arm Holdings

December 25, 2019

The Linux kernel has a rich cryptographic API which provides access to a modular implementation of symmetric and asymmetric block ciphers, hashes and digests which are either software implemented, use …
Ten Layers of Container Security

Ten Layers of Container Security

December 24, 2019

In this session, we'll identify the 10 most common layers in a typical container deployment, and the best ways to build security into each layer. These layers include: 1) container host: Multi-tenancy …
Using Persistent Memory to Build a High-Performance, Fully User Space File System - Krzysztof Czurylo, Intel

Using Persistent Memory to Build a High-Performance, Fully User Space File System - Krzysztof Czurylo, Intel

December 24, 2019

The pmemfile project is an attempt to provide low-overhead, high-performance implementation of a POSIX-like file APIs on top of persistent memory. The pmemfile builds on libpmemobj, a transactional ob …
Panel Discussion: Outreachy Kernel Internship Report

Panel Discussion: Outreachy Kernel Internship Report

December 23, 2019

Come learn about the great work our kernel interns have accomplished! Outreachy provides a 3-month paid internship for women, trans men, genderqueer people, and US members of other underrepresented gr …
syscall_intercept - A User Space Library for Intercepting System Calls - Krzysztof Czurylo, Intel

syscall_intercept - A User Space Library for Intercepting System Calls - Krzysztof Czurylo, Intel

December 22, 2019

The syscall_intercept library provides a low-level interface for hooking Linux system calls in user space. This is achieved by disassembling the code of the standard C library, looking for syscall ins …
Mapping data in Jupyter notebooks with PixieDust (sponsored by IBM)

Mapping data in Jupyter notebooks with PixieDust (sponsored by IBM)

December 21, 2019

Raj Singh offers an overview of PixieDust, a Jupyter Notebook extension that provides an easy way to make interactive maps from DataFrames for visual exploratory data analysis. Raj explains how he built mapping into PixieDust, putting data from Apache Spark-based analytics on maps using Mapbox GL.
Mixing cgroupfs v1 and cgroupfs v2: Finding Solutions for Container Runtimes

Mixing cgroupfs v1 and cgroupfs v2: Finding Solutions for Container Runtimes

December 21, 2019

With the release of kernel 4.5 the new cgroupfs v2 API was declared non-experimental. But the missing feature parity between cgroupfs v2 with cgroupfs v1 makes it nearly impossible for container runti …
Death of the ESB: 10 Pitfalls in the Trasition to Cloud-Native Integration

Death of the ESB: 10 Pitfalls in the Trasition to Cloud-Native Integration

December 20, 2019

As we move from an ESB world to an API focused microservices world, the need for integration does not go away. In fact, it is arguably far more important today than ever. Based on customer engagements …
Everything You Always Wanted to Know About Object Storage

Everything You Always Wanted to Know About Object Storage

December 20, 2019

The rapid growth of unstructured data is fueling the need for a next generation storage thats flexible, economical, and scalable enough to handle the petabytes of data being created every day. Object …
Replacing the Radix Tree

Replacing the Radix Tree

December 20, 2019

Last year I gave a talk extolling the benefits of the Linux radix tree. This year I am talking about its shortcomings, what I did to improve things, and how I came to the conclusion that it had to be …
Integration of Flexible Storage with the API of Gluster

Integration of Flexible Storage with the API of Gluster

December 19, 2019

No matter where an application is running, it will most likely need some form of storage. Gluster is a distributed scale-out filesystem that offers interfaces for many different use-cases and workload …
Machine learning on Google Cloud Platform (sponsored by Google)

Machine learning on Google Cloud Platform (sponsored by Google)

December 18, 2019

Amy Unruh offers a quick overview of machine learning on Google Cloud Platform and demonstrates a couple of the Google Cloud ML APIs. She then briefly highlights a few OSS TensorFlow models and explains how to use transfer learning to fine-tune them with your own data.
Achieve predictable performance (sponsored by Intel)

Achieve predictable performance (sponsored by Intel)

December 17, 2019

Alex Grbic explains how a single FPGA can deliver significant acceleration for multiple workloads. This new approach of integrating data analytics frameworks and existing databases enables enterprise customers to run unmodified applications without requiring any FPGA expertise and can be used with unstructured, NoSQL, and traditional relational databases, such as Swarm64.
The future of AI is now (sponsored by IBM)

The future of AI is now (sponsored by IBM)

December 17, 2019

Damion Heredia explores how augmented intelligence is helping companies disrupt industries and enabling them to make better decisions.
Tutorial: Service Orchestration - Way Beyond SDNFV Orchestration - Presented by MEF

Tutorial: Service Orchestration - Way Beyond SDNFV Orchestration - Presented by MEF

December 16, 2019

While service providers are quickly ramping up their capabilities to orchestrate Virtualized Network Functions and SDN components, many are starting to realize that work gets them only part way to the …
How Shutterfly migrated 10+ billion photos to the cloud

How Shutterfly migrated 10+ billion photos to the cloud

December 15, 2019

Jack Chan describes how Shutterfly migrated metadata from over 10B photos from a private data center into AWS in 100 days and explores designs to absorb mountains of metadata, on-premises ecommerce integration, and parallel user experiences, all in a highly scalable fashion. Shutterfly Photos is now a hybrid cloud solution with images hosted on-premises and client-facing photos metadata on AWS.