February 3, 2020

188 words 1 min read

Open Source at Uber

Open Source at Uber

Uber officially began its open source journey in 2015 when three passionate engineers decided to contribute Ubers work back to the community. In only three years, Ubers open source program has foste …
Talk Title Open Source at Uber
Speakers Jesus M. Gonzalez-Barahona (Bitergia), Brian Hsieh (Head of Open Source, Uber)
Conference Open Source Summit North America
Conf Tag
Location Vancouver, BC, Canada
Date Aug 27-31, 2018
URL Talk Page
Slides Talk Slides
Video

Uber officially began its open source journey in 2015 when three passionate engineers decided to contribute Uber’s work back to the community. In only three years, Uber’s open source program has fostered several outstanding open source projects like Jaeger, deck.gl, and Pyro. Uber has also been contributing to existing open source projects such as Hadoop, Spark, and more. In this presentation, Brian Hsieh will share Uber’s open source journey, its program structure, open source best practices, and how Uber enables open innovation by fostering a healthy and collaborative open source culture. He will also discuss the challenges of leading an open source program in a hyper-growth global company and Uber’s long-term open source vision.

health spark hadoop open source uber collaborative
comments powered by Disqus
Bighead: Airbnb's end-to-end machine learning platform

Bighead: Airbnb's end-to-end machine learning platform

January 26, 2020

Atul Kale and Xiaohan Zeng offer an overview of Bighead, Airbnb's user-friendly and scalable end-to-end machine learning framework that powers Airbnb's data-driven products. Built on Python, Spark, and Kubernetes, Bighead integrates popular libraries like TensorFlow, XGBoost, and PyTorch and is designed be used in modular pieces.
Hudi: Unifying storage and serving for batch and near-real-time analytics

Hudi: Unifying storage and serving for batch and near-real-time analytics

January 22, 2020

Uber has a real need to provide faster, fresher data to its data consumers and products, which are running hundreds of thousands of analytical queries every day. Nishith Agarwal, Balaji Varadarajan, and Vinoth Chandar share the design, architecture, and use cases of the second-generation of Hudi, an analytical storage engine designed to serve such needs and beyond.
Marmaray: A generic, scalable, and pluggable Hadoop data ingestion and dispersal framework

Marmaray: A generic, scalable, and pluggable Hadoop data ingestion and dispersal framework

January 20, 2020

Danny Chen, Omkar Joshi, and Eric Sayle offer an overview of Marmaray, a generic Hadoop ingestion and dispersal framework recently released to production at Uber. You'll learn how Marmaray can meet a team's data needs by ensuring that data can be reliably ingested into Hive or dispersed into online data stores and take a deep dive into the architecture to see how it all works.
TuneIn: How to get your jobs tuned while you are sleeping

TuneIn: How to get your jobs tuned while you are sleeping

January 17, 2020

Have you ever tuned a Spark or MR job? If the answer is yes, you already know how difficult it is to tune more than hundred parameters to optimize the resources used. Manoj Kumar, Pralabh Kumar, and Arpan Agrawal offer an overview of TuneIn, an auto-tuning tool developed to minimize the resource usage of jobs. Experiments have shown up to a 50% reduction in resource usage.
Pangeo: Big data climate science in the cloud

Pangeo: Big data climate science in the cloud

January 6, 2020

Climate science is being flooded with petabytes of data, overwhelming traditional modes of data analysis. The Pangeo project is building a platform to take big data climate science into the cloud using SciPy and large-scale interactive computing tools. Join Ryan Abernathey and Yuvi Panda to find out what the Pangeo team is building and why and learn how to use it.
Distributed TensorFlow on Hops

Distributed TensorFlow on Hops

December 30, 2019

Fabio Buso offers demonstrations of frameworks for building distributed TensorFlow applications on the Hops platform and walks you through the whole model lifecycle, from debugging and visualizing models on TensorBoard to parallel experimentation and distributed training (with the help of Spark) to model deployment and inferencing using TensorFlow Serving and Kubernetes.