dataset

"CASE": A Mesos Scheduler for Distributed Machine Learning

"CASE": A Mesos Scheduler for Distributed Machine Learning

February 4, 2020

Many machine learning frameworks support distributed training of models. Distributed training is becoming increasingly important as organizations train on larger datasets with increased importance on …
Deep learning at scale: A field manual

Deep learning at scale: A field manual

February 4, 2020

Jason Knight offers an overview of the state of the field for scaling training and inference across distributed systems from a practitioner's point of view. Along the way, Jason dives deep into available tools, resources, and venues for getting started without having to go it alone.
Executive Briefing: Best practices for human in the loopThe business case for active learning

Executive Briefing: Best practices for human in the loopThe business case for active learning

February 4, 2020

Deep learning works well when you have large labeled datasets, but not every team has those assets. Paco Nathan offers an overview of active learning, an ML variant that incorporates human-in-the-loop computing. Active learning focuses input from human experts, leveraging intelligence already in the system, and provides systematic ways to explore and exploit uncertainty in your data.
How artificial intelligence is changing advertising in China: A conversation with Bessie Lee and Ching Law

How artificial intelligence is changing advertising in China: A conversation with Bessie Lee and Ching Law

February 3, 2020

Advertising in China is on the frontline of AI adoption and innovation. Join Bessie Lee and Ching Law for a conversation on how AI is changing advertising. You'll hear how China's white-hot AI advertising applications can serve as roadmaps and spark ideas in other industries and how companies like Tencent are improving performance by leveraging AI technology.
Applying petabyte-scale analytics and machine learning to billions of news reading sessions

Applying petabyte-scale analytics and machine learning to billions of news reading sessions

January 26, 2020

What can we learn from a one-billion-person live poll of the internet? Andrew Montalenti explains how Parse.ly has gathered a unique dataset of news reading sessions of billions of devices, peaking at over two million sessions per minute on thousands of high-traffic news and information websites, and how the company uses this data to unearth the secrets behind online content.
Architecting a next-generation data platform

Architecting a next-generation data platform

January 26, 2020

Using Customer 360 and the internet of things as examples, Jonathan Seidman and Ted Malaska explain how to architect a modern, real-time big data platform leveraging recent advancements in the open source software world, including components like Kafka, Flink, Kudu, Spark Streaming, and Spark SQL and modern storage engines to enable new forms of data processing and analytics.
Deep learning on audio in Azure to detect sounds in real time

Deep learning on audio in Azure to detect sounds in real time

January 25, 2020

In this auditory world, the human brain processes and reacts effortlessly to a variety of sounds. While many of us take this for granted, there are over 360 million in this world who are deaf or hard of hearing. Swetha Machanavajhala and Xiaoyong Zhu explain how to make the auditory world inclusive and meet the great demand in other sectors by applying deep learning on audio in Azure.
Executive Briefing: Best practices for human in the loopThe business case for active learning

Executive Briefing: Best practices for human in the loopThe business case for active learning

January 24, 2020

Deep learning works well when you have large labeled datasets, but not every team has those assets. Paco Nathan offers an overview of active learning, an ML variant that incorporates human-in-the-loop computing. Active learning focuses input from human experts, leveraging intelligence already in the system, and provides systematic ways to explore and exploit uncertainty in your data.
Hudi: Unifying storage and serving for batch and near-real-time analytics

Hudi: Unifying storage and serving for batch and near-real-time analytics

January 22, 2020

Uber has a real need to provide faster, fresher data to its data consumers and products, which are running hundreds of thousands of analytical queries every day. Nishith Agarwal, Balaji Varadarajan, and Vinoth Chandar share the design, architecture, and use cases of the second-generation of Hudi, an analytical storage engine designed to serve such needs and beyond.
Job recommendations leveraging deep learning using Analytics Zoo on Apache Spark and BigDL

Job recommendations leveraging deep learning using Analytics Zoo on Apache Spark and BigDL

January 21, 2020

Can the talent industry make the job search/match more relevant and personalized for a candidate by leveraging deep learning techniques? Guoqiong Song, Wenjing Zhan, and Jacob Eisinger demonstrate how to leverage distributed deep learning framework BigDL on Apache Spark to predict a candidates probability of applying to specific jobs based on their rsum.
Leveraging Spark and deep learning frameworks to understand data at scale

Leveraging Spark and deep learning frameworks to understand data at scale

January 21, 2020

Vartika Singh, Alan Silva, Alex Bleakley, Steven Totman, Mirko Kmpf, and Syed Nasar outline approaches for preprocessing, training, inference, and deployment across datasets (time series, audio, video, text, etc.) that leverage Spark, its extended ecosystem of libraries, and deep learning frameworks.
Managing data chaos in the world of microservices

Managing data chaos in the world of microservices

January 20, 2020

When we talk about microservices, we usually focus on the communication layer. In practice, data is the much harder and often overlooked problem. Splitting applications into independent units leads to increased complexity, such as structural and semantic changes, knowledge sharing, and data discovery. Join Alexey Kachayev to explore emerging technologies created to tackle these challenges.
Marmaray: A generic, scalable, and pluggable Hadoop data ingestion and dispersal framework

Marmaray: A generic, scalable, and pluggable Hadoop data ingestion and dispersal framework

January 20, 2020

Danny Chen, Omkar Joshi, and Eric Sayle offer an overview of Marmaray, a generic Hadoop ingestion and dispersal framework recently released to production at Uber. You'll learn how Marmaray can meet a team's data needs by ensuring that data can be reliably ingested into Hive or dispersed into online data stores and take a deep dive into the architecture to see how it all works.
Real-time automated claim processing: The surprising utility of NLP methods on non-text data

Real-time automated claim processing: The surprising utility of NLP methods on non-text data

January 18, 2020

Processing claims is central to every insurance business. Amro Alkhatib shares a successful business case for automating claims processing, from idea to production. The machine learning-based claim automation model uses NLP methods on non-text data and allows auditable automated claims decisions to be made.
Self-service modern analytics on the GovCloud

Self-service modern analytics on the GovCloud

January 18, 2020

Lockheed Martin is a data-driven company with a massive variety and volume of data. To extract the most value from its information assets, the company is constantly exploring ways to enable effective self-service scenarios. Ramesh Krishnan and Steve Morgan discuss Lockheed Martin's journey into modern analytics and explore its analytics platform focused on leveraging AWS GovCloud.
Distributed deep learning in the cloud: Build an end-to-end application involving computer vision and geospatial data

Distributed deep learning in the cloud: Build an end-to-end application involving computer vision and geospatial data

January 13, 2020

High-resolution land cover maps help quantify long-term trends like deforestation and urbanization but are prohibitively costly and time intensive to produce. Mary Wahl and Banibrata De demonstrate how to use Microsofts Cognitive Toolkit and Azure cloud resources to produce land cover maps from aerial imagery by training a semantic segmentation DNNboth on single VMs and at scale on GPU clusters.
Efficient neural network training on Intel Xeon-based supercomputers

Efficient neural network training on Intel Xeon-based supercomputers

January 13, 2020

Vikram Saletore and Luke Wilson discuss a collaboration between SURFSara and Intel to advance the state of large-scale neural network training on Intel Xeon CPU-based servers, highlighting improved time to solution on extended training of pretrained models and exploring how various storage and interconnect options lead to more efficient scaling.
Evaluate deep Q-learning for sequential targeted marketing with 10-fold cross-validation

Evaluate deep Q-learning for sequential targeted marketing with 10-fold cross-validation

January 13, 2020

Jian Wu discusses an end-to-end engineering project to train and evaluate deep Q-learning models for targeting sequential marketing campaigns using the 10-fold cross-validation method. Jian also explains how to evaluate trained DQN models with neural network-based baseline models and shows that trained deep Q-learning models generally produce better-optimized long-term rewards.
Executive Briefing: Best practices for human in the loopThe business case for active learning

Executive Briefing: Best practices for human in the loopThe business case for active learning

January 12, 2020

Deep learning works well when you have large labeled datasets, but not every team has those assets. Paco Nathan offers an overview of active learning, an ML variant that incorporates human-in-the-loop computing. Active learning focuses input from human experts, leveraging intelligence already in the system, and provides systematic ways to explore and exploit uncertainty in your data.
Hit a home run making baseball decisions using artificial intelligence and machine learning

Hit a home run making baseball decisions using artificial intelligence and machine learning

January 12, 2020

Join Ari Kaplan, Erin LeDell, Chris Coad, and David Kearns to see where AI meets business intelligence, as they explore the latest ML technologies and concepts powering today's decisions, including Hortonworks, Aginity Amp, H2O.ai, IBM Data Science Experience, and moreusing real-life baseball data to illustrate the concepts.
How to use transfer learning to bootstrap image classification and question answering (QA)

How to use transfer learning to bootstrap image classification and question answering (QA)

January 11, 2020

Transfer learning enables you to use pretrained deep neural networks and adapt them for various deep learning tasks (e.g., image classification, question answering, and more). Join Wee Hyong Tok and Danielle Dean to learn the secrets of transfer learning and discover how to customize these pretrained models for your own use cases.
Open source decentralized data markets for training AI in areas of large shared risk

Open source decentralized data markets for training AI in areas of large shared risk

January 11, 2020

Paco Nathan examines decentralized data markets. With components based on blockchain technologiessmart contracts, token-curated registries, DApps, voting mechanisms, etc.decentralized data markets allow multiple parties to curate ML training datasets in ways that are transparent, auditable, and secure and allow equitable payouts that take social values into account.
Trustworthiness of machine learning applications

Trustworthiness of machine learning applications

January 10, 2020

Machine learning is eating software. As decisions are automated, model interpretability must become an integral part of the ML pipeline rather than an afterthought. In the real world, the demand for being able to explain a model is rapidly gaining on model accuracy. Mayukh Bhaowal discusses the steps Salesforce Einstein is taking to make machine learning more transparent and less of a black box.
How JupyterLab and widgets enable interactive analysis of the Earth's past, present, and future

How JupyterLab and widgets enable interactive analysis of the Earth's past, present, and future

January 8, 2020

Massive collections of data on the Earth's changing environment, collected by satellite sensors and generated by Earth system models, are being exposed via web APIs by multiple providers. Tyler Erickson highlights the use of JupyterLab and Jupyter widgets in analyzing complex high-dimensional datasets, providing insights into how our Earth is changing and what the future might look like.
JupyterHub for domain-focused integrated learning modules

JupyterHub for domain-focused integrated learning modules

January 7, 2020

The Data Science Modules program at UC Berkeley creates short explorations into data science using notebooks to allow students to work hands-on with a dataset relevant to their course. Mariah Rogers, Ronald Walker, and Julian Kudszus explain the logistics behind such a program and the indispensable features of JupyterHub that enable such a unique learning experience.
Pangeo: Big data climate science in the cloud

Pangeo: Big data climate science in the cloud

January 6, 2020

Climate science is being flooded with petabytes of data, overwhelming traditional modes of data analysis. The Pangeo project is building a platform to take big data climate science into the cloud using SciPy and large-scale interactive computing tools. Join Ryan Abernathey and Yuvi Panda to find out what the Pangeo team is building and why and learn how to use it.
Reproducible data dependencies for Jupyter: Distributing massive, versioned image datasets from the Allen Institute for Cell Science

Reproducible data dependencies for Jupyter: Distributing massive, versioned image datasets from the Allen Institute for Cell Science

January 6, 2020

Reproducible data is essential for notebooks that work across time, across contributors, and across machines. Jackson Brown and Aneesh Karve demonstrate how to use an open source data registry to create reproducible data dependencies for Jupyter and share a case study in open science over terabyte-size image datasets.
Reproducible science with the Renku platform

Reproducible science with the Renku platform

January 6, 2020

Sandra Savchenko-de Jong offers an overview of Renku, a highly scalable and secure open software platform designed to make (data) science reproducible, foster collaboration between scientists, and share resources in a federated environment.