[TALK]@Telematika

database

Learning how to perform ETL data migrations with open source tool Embulk

January 9, 2020

The Embulk data migration tool offers a convenient way to load data in to a variety of systems with basic configuration. Jason Bell offers an overview of the Embulk tool and outlines some common data migration scenarios that a data engineer could employ using the tool.

Processing 10M samples a second to drive smart maintenance in complex IIoT systems

January 7, 2020

Geir Engdahl and Daniel Bergqvist explain how Cognite is developing IIoT smart maintenance systems that can process 10M samples a second from thousands of sensors. You'll explore an architecture designed for high performance, robust streaming sensor data ingest, and cost-effective storage of large volumes of time series data as well as best practices learned along the way.

The changing face of ETL: Event-driven architectures for data engineers

January 6, 2020

Robin Moffatt discusses the concepts of events, their relevance to software and data engineers, and their ability to unify architectures in a powerful way. Join in to learn why analytics, data integration, and ETL fit naturally into a streaming world. Along the way, Robin will lead a hands-on demonstration of these concepts in practice and commentary on the design choices made.

The Presto Cost-Based Optimizer for interactive SQL on anything

January 5, 2020

Tars for FinTech

December 26, 2019

Tars, a high-performance microservice framework, supports mainstream and customized protocols and provides an integrated solution of microservices governance for developers and operators. With support …

Database migrations don't have to be painful, but the road will be bumpy

December 25, 2019

Adrian Lungu and Serban Teodorescu explain howinspired by the green-blue deployment techniquethe Adobe Audience Manager team developed an active-passive database migration procedure that allows them to test database clusters in production, minimizing the risks without compromising the innovation.

Faster ML over joins of tables

December 24, 2019

Arun Kumar details recent techniques to accelerate ML over data that is the output of joins of multiple tables. Using ideas from query optimization and learning theory, Arun demonstrates how to avoid joins before ML to reduce runtimes and memory and storage footprints. Along the way, he explores open source software prototypes and sample ML code in both R and Python.

From flat files to deconstructed databases: The evolution and future of the big data ecosystem

December 24, 2019

Big data infrastructure has evolved from flat files in a distributed filesystem to an efficient ecosystem to a fully deconstructed and open source database with reusable components. Julien Le Dem discusses the key open source components of the big data ecosystem and explains how they relate to each other and how they make the ecosystem more of a database and less of a filesystem.

ROCKSET: The design and implementation of a data system for low-latency queries for search and analytics

December 22, 2019

Most existing big data systems prefer sequential scans for processing queries. Igor Canadi and Dhruba Borthakur challenge this view, offering an overview of converged indexing: a single system called ROCKSET that builds inverted, columnar, and document indices. Converged indexing is economically feasible due to the elasticity of cloud-resources and write optimized storage engines.

Solving the enterprise data dilemma (sponsored by erwin)

December 21, 2019

Adam Famularo showcases erwin's combination of data management and data governance to produce actionable insights. Erwin customer Nasdaq then shares a real-world use case. You'll learn how to answer tough data questions, how to maintain a metadata landscape, and how to use data management and governance to produce actionable insights.

Scaling Resilient Systems: A Journey into Slack's Database Service

December 20, 2019

Monitoring and observability are important concepts, especially in complex and distributed systems. Redundancy and defensive programming are important as well, but sometimes they are not enough. Desig …

Five Things You Didnt Know You Could Do with SPIFFE and SPIRE

December 19, 2019

Zero Trust networking and secure authentication are hot topics in security team meetings all over the world. But how do you actually get started? The open-source SPIFFE and SPIRE projects are your fou …

How to Migrate a MySQL Database to Vitess

December 19, 2019

Vitess is a cloud-native storage solution that can scale indefinitely. This session will cover a high level overview of all the Vitess features, the architecture, and what database workloads are a goo …

Intro to SIG Service Catalog

December 18, 2019

This is an introduction to the Kubernetes Service Catalog extension project. Service Catalog lets you provision cloud services directly from the comfort of native Kubernetes tooling, regardless of whe …

Building a Database as a Service on Kubernetes

December 15, 2019

Stateful, scalable storage on Kubernetes is an unsolved problem. Creating it as a service is even more difficult. The cloud-native ecosystem offers many tools such as the operator-sdk, Prometheus, Gra …

Geo-partitioning with Vitess

December 5, 2019

Geo-partitioning is an emerging requirement for multinational corporations. Laws such as the GDPR in Europe stipulate where a users data needs to be physically located, as well as when it can be tran …

How Container Networking Affects Database Performance

December 3, 2019

Through benchmarks, Percona Labs explores the effects of different container networking drivers used in Kubernetes when hosting database workloads. For this talk, we will perform benchmarks using Perc …

Intro: Prometheus

December 1, 2019

Prometheus is an open-source monitoring system and time series database. It features a multi-dimensional data model with a powerful query language and integrates many aspects of systems and service mo …

Intro: Service Catalog SIG

November 15, 2019

This is an introduction to the Kubernetes Service Catalog project. Service Catalog lets you provision cloud services, regardless of where they are hosted, directly from the comfort of native Kubernete …

Databases on Kubernetes Using a Custom Operator: Day 1, Day 2, and Beyond

November 14, 2019

We started the journey of building a managed cloud version of the graph database Neo4j. A bit later we started developing an operator to manage multiple database clusters in k8s.Handling persistence a …

Scale Kubernetes Service Endpoints 100x

November 12, 2019

Service and endpoints have been core primitives in K8s since 1.0. As k8s deployments gets larger, the scalability of k8s service endpoints has become a bottleneck, causing high overhead on network, cp …

Keynote: Debunking the Myth: Kubernetes Storage is Hard

November 8, 2019

We often hear Storage on Kubernetes is hard or Dont deploy stateful applications on Kubernetes! Is there any truth to these statements? In this talk I separate fact from fiction. Storage on Kuber …

Intro: Vitess

November 6, 2019

This session will cover a high level overview of all the Vitess features. Following this, we'll go over a tutorial on how to bring up a fully functional Vitess cluster in Kubernetes. Once up, we'll go …

Benchmarking Cloud Native Databases Performance on Kubernetes

November 3, 2019

CNCF defines cloud native technologies as, "technologies that empower organizations to build and run scalable applications in modern and dynamic environments. Cloud native technologies enable loosely …

Large Synoptic Survey Telescope: From Cloud Native to Dark Energy

October 30, 2019

Relying on the largest camera sensor ever designed, the Large Synoptic Survey Telescope (LSST) will revolutionize astronomy. The LSST data processing pipeline requires computational power of thousands …

The Spotify Story: Moving from NIH to an Active CNCF Community Member

October 28, 2019

Four years ago, at Spotify, we built and managed the majority of our infrastructure ourselves. We managed our own data centres. We built our own time series database. We built our own orchestration te …

Running eBays High-Performance Workloads with Kubernetes

October 25, 2019

In the past two years weve been expanding our k8s deployments significantly fast by moving more and more production workloads into kubernetes.Were now running multiple thousand-node k8s clusters fro …

Intro + Deep Dive: Prometheus

October 24, 2019