February 7, 2020

171 words 1 min read

Open Source at Uber

Open Source at Uber

Uber officially launched its open source journey in 2015 which started with three passionate engineers. In only three years, Ubers open source program has grown and fostered several outstanding open …
Talk Title Open Source at Uber
Speakers Brian Hsieh (Head of Open Source, Uber)
Conference Automotive Linux Summit & Open Source Summit Japan
Conf Tag
Location Tokyo, Japan
Date Jun 19-22, 2018
URL Talk Page
Slides Talk Slides
Video

Uber officially launched its open source journey in 2015 which started with three passionate engineers. In only three years, Uber’s open source program has grown and fostered several outstanding open source projects like Jaeger, deck.gl, and Pyro. Uber has also been contributing to open source projects like Hadoop, Spark, and more. In the presentation, Brian Hsieh will share Uber’s open source journey, its program office structure, best practices, and how Uber enables open innovation by embracing open source culture. He will also share the challenges of leading open source program in a hyper-growth global company, and Uber’s long-term open source vision.

spark uber hadoop open source
comments powered by Disqus
Open Source at Uber

Open Source at Uber

February 3, 2020

Uber officially began its open source journey in 2015 when three passionate engineers decided to contribute Ubers work back to the community. In only three years, Ubers open source program has foste …
Marmaray: A generic, scalable, and pluggable Hadoop data ingestion and dispersal framework

Marmaray: A generic, scalable, and pluggable Hadoop data ingestion and dispersal framework

January 20, 2020

Danny Chen, Omkar Joshi, and Eric Sayle offer an overview of Marmaray, a generic Hadoop ingestion and dispersal framework recently released to production at Uber. You'll learn how Marmaray can meet a team's data needs by ensuring that data can be reliably ingested into Hive or dispersed into online data stores and take a deep dive into the architecture to see how it all works.
TuneIn: How to get your jobs tuned while you are sleeping

TuneIn: How to get your jobs tuned while you are sleeping

January 17, 2020

Have you ever tuned a Spark or MR job? If the answer is yes, you already know how difficult it is to tune more than hundred parameters to optimize the resources used. Manoj Kumar, Pralabh Kumar, and Arpan Agrawal offer an overview of TuneIn, an auto-tuning tool developed to minimize the resource usage of jobs. Experiments have shown up to a 50% reduction in resource usage.
Pangeo: Big data climate science in the cloud

Pangeo: Big data climate science in the cloud

January 6, 2020

Climate science is being flooded with petabytes of data, overwhelming traditional modes of data analysis. The Pangeo project is building a platform to take big data climate science into the cloud using SciPy and large-scale interactive computing tools. Join Ryan Abernathey and Yuvi Panda to find out what the Pangeo team is building and why and learn how to use it.
Distributed TensorFlow on Hops

Distributed TensorFlow on Hops

December 30, 2019

Fabio Buso offers demonstrations of frameworks for building distributed TensorFlow applications on the Hops platform and walks you through the whole model lifecycle, from debugging and visualizing models on TensorBoard to parallel experimentation and distributed training (with the help of Spark) to model deployment and inferencing using TensorFlow Serving and Kubernetes.
Architecting data platforms for cybersecurity

Architecting data platforms for cybersecurity

December 12, 2019

Data is becoming a crucial weapon to secure an organization against cyber threats. Charaka Goonatilake shares strategies for designing effective data platforms for cybersecurity using big data technologies, such as Spark and Hadoop, and explains how these platforms are being used in real-world examples of data-driven security.