dataset

SoS: A polyglot notebook and workflow system for both interactive multilanguage data analysis and batch data processing

SoS: A polyglot notebook and workflow system for both interactive multilanguage data analysis and batch data processing

January 6, 2020

Bo Peng offers an overview of Script of Scripts (SoS), a Python 3-based workflow engine with a Jupyter frontend that allows the use of multiple kernels in one notebook. This unique combination enables users to analyze data using multiple scripting languages in one notebook and, if needed, convert scripts to workflows in situ to analyze large amounts of data on remote systems.
The future of data-driven discovery in the cloud

The future of data-driven discovery in the cloud

January 5, 2020

Drawing on his experience with the Pangeo project, Ryan Abernathey makes the case for the large-scale migration of scientific data and research to the cloud. The cloud offers a way to make the largest datasets instantly accessible to the most sophisticated computational techniques. A global scientific data commons could usher in a golden age of data-driven discovery.
Tutorial: Kubeflow End-to-End: GitHub Issue Summarization

Tutorial: Kubeflow End-to-End: GitHub Issue Summarization

January 5, 2020

IMPORTANT NOTE:Due to the nature of tutorials, this session has been placed in a smaller capacity room to help facilitate a conducive learning environment. Space is very limited and seating will be g …
Visualizing high-dimensional biological data with Clustergrammer-Widget in the Jupyter Notebook

Visualizing high-dimensional biological data with Clustergrammer-Widget in the Jupyter Notebook

January 4, 2020

Nicolas Fernandez offers an overview of Clustergrammer-Widget, an interactive heatmap Jupyter widget that enables users to easily explore high-dimensional data within a Jupyter notebook and share their interactive visualizations using nbviewer.
Distributed TensorFlow on Hops

Distributed TensorFlow on Hops

December 30, 2019

Fabio Buso offers demonstrations of frameworks for building distributed TensorFlow applications on the Hops platform and walks you through the whole model lifecycle, from debugging and visualizing models on TensorBoard to parallel experimentation and distributed training (with the help of Spark) to model deployment and inferencing using TensorFlow Serving and Kubernetes.
Nezha: A Kubernetes Native Big Data Accelerator For Machine Learning

Nezha: A Kubernetes Native Big Data Accelerator For Machine Learning

December 27, 2019

Large training datasets used by machine learning frameworks, such as Kubeflow, are usually stored in low cost and high capacity S3 or Google Cloud Storage. However, S3s rating limiting and slow data …
Lessons learned while evolving Boxs database infrastructure

Lessons learned while evolving Boxs database infrastructure

December 21, 2019

When Tamar Bercovici joined Box, the entire platform was running on a single MySQL DB host fronted by a simple pool of memcached servers. Tamar details how the team has evolved the Box database stack to handle an ever-growing query load and dataset. It now comprises hundreds of servers serving millions of queries per second over hundreds of billions of data records.
Tracking the performance of the web with HTTP Archive

Tracking the performance of the web with HTTP Archive

December 13, 2019

Have you ever thought about how your sites performance compares to the web as a whole? Paul Calvano explores how the HTTP Archive works, how people are using this dataset, and some ways that Akamai has leveraged data within the HTTP Archive to help its customers.
Architecting a next-generation data platform

Architecting a next-generation data platform

December 12, 2019

Using Customer 360 and the IoT as examples, Jonathan Seidman and Ted Malaska explain how to architect a modern, real-time big data platform leveraging recent advancements in the open source software world, using components like Kafka, Flink, Kudu, Spark Streaming, and Spark SQL and modern storage engines to enable new forms of data processing and analytics.
Distributed training of deep learning models

Distributed training of deep learning models

December 10, 2019

Mathew Salvaris, Miguel Gonzalez-Fierro, and Ilia Karmanov offer a comparison of two platforms for running distributed deep learning training in the cloud, using a ResNet network trained on the ImageNet dataset as an example. You'll examine the performance of each as the number of nodes scales and learn some tips and tricks as well as some pitfalls to watch out for.
Keynote: CERN Experiences with Multi-Cloud Federated Kubernetes

Keynote: CERN Experiences with Multi-Cloud Federated Kubernetes

December 7, 2019

Using public cloud resources to cover for peak workloads is a practical and economical alternative to over provisioning on-premise resources. This is the case in environments like CERN where its large …
Machine learning platform lifecycle management

Machine learning platform lifecycle management

December 7, 2019

A machine learning platform is not just the sum of its parts; the key is how it supports the model lifecycle end to end. Hope Wang explains how to manage various artifacts and their associations, automate deployment to support the lifecycle of a model, and build a cohesive machine learning platform.
Macroeconomic news sentiment: Enhanced risk assessment for sovereign bond spreads

Macroeconomic news sentiment: Enhanced risk assessment for sovereign bond spreads

December 7, 2019

Christina Erlwein-Sayer explains how to enhance the modeling and forecasting of sovereign bond spreads by considering quantitative information gained from macroeconomic news sentiment, using a number of large news analytics datasets.
Operationalize deep learning models for fraud detection with Azure Machine Learning Workbench

Operationalize deep learning models for fraud detection with Azure Machine Learning Workbench

December 6, 2019

Advancements in computing technologies and ecommerce platforms have amplified the risk of online fraud, which results in billions of dollars of loss for the financial industry. This trend has urged companies to consider AI techniques, including deep learning, for fraud detection. Francesca Lazzeri and Jaya Mathew explain how to operationalize deep learning models with Azure ML to prevent fraud.
StreamDM: Advanced data science with Spark Streaming

StreamDM: Advanced data science with Spark Streaming

December 5, 2019

Heitor Murilo Gomes and Albert Bifet offer an overview of StreamDM, a real-time analytics open source software library built on top of Spark Streaming, developed at Huawei's Noahs Ark Lab and Tlcom ParisTech.
Using Siamese CNNs for removing duplicate entries from real estate listing databases

Using Siamese CNNs for removing duplicate entries from real estate listing databases

December 4, 2019

Aggregation of geospecific real estate databases results in duplicate entries for properties located near geographical boundaries. Sergey Ermolin and Olga Ermolin detail an approach for identifying duplicate entries via the analysis of images that accompany real estate listings that leverages a transfer learning Siamese architecture based on VGG-16 CNN topology.
You call it data lake; we call it Data Historian.

You call it data lake; we call it Data Historian.

December 4, 2019

There are a number of tools that make it easy to implement a data lake. However, most lack the essential features that prevent your data lake from turning into a data swamp. Naghman Waheed and Brian Arnold offer an overview of Monsanto's Data Historian platform, which can ingest, store, and access datasets without compromising ease of use, governance, or security.
Smart diagnosis in healthcare with deep learning

Smart diagnosis in healthcare with deep learning

December 3, 2019

Deep learning with ConvNet in particular has emerged as a promising tool in medical research labs and diagnostic centers to help analyze images and scans, and systems are now surpassing human capability for manual inspection. Nishant Sahay explains how to apply deep learning to analyze high-end microscope images and X-ray scans to provide accurate diagnosis.
AI-powered crime prediction

AI-powered crime prediction

November 30, 2019

What if we could predict when and where crimes will be committed? Or Herman-Saffar and Ran Taig offer an overview of Crimes in Chicago, a publicly published dataset of reported incidents of crime that have occurred in Chicago since 2001. Or and Ran explain how to use this data to explore committed crimes to find interesting trends and make predictions for the future.
Best practices for productionizing Apache Spark MLlib models

Best practices for productionizing Apache Spark MLlib models

November 29, 2019

Joseph Bradley discusses common paths to productionizing Apache Spark MLlib models and shares engineering challenges and corresponding best practices. Along the way, Joseph covers several deployment scenarios, including batch scoring, Structured Streaming, and real-time low-latency serving.
Distributed clinical models: Inference without sharing patient data

Distributed clinical models: Inference without sharing patient data

November 26, 2019

Clinical collaboration benefits from pooling data to train models from large datasets, but it's hampered by concerns about sharing data. Balasubramanian Narasimhan, John-Mark Agosta, and Philip Lavori outline a privacy-preserving alternative that creates statistical models equivalent to one from the entire dataset.
Get a farm-to-table view of your data: Track data lineage from source to analytics (sponsored by Syncsort)

Get a farm-to-table view of your data: Track data lineage from source to analytics (sponsored by Syncsort)

November 25, 2019

Chefs must be able to trust the authenticity, quality, and origin of their ingredients; data analysts must be able to do the same of their dataand what happens to it along the way. Tend Yourtu explains how to seamlessly track the lineage and quality of your dataon and off the cluster, on-premises or in the cloudto deliver meaningful insights and meet regulatory compliance requirements.
Improving user-merchant propensity modeling using neural collaborative filtering and wide and deep models on Spark BigDL at scale

Improving user-merchant propensity modeling using neural collaborative filtering and wide and deep models on Spark BigDL at scale

November 24, 2019

Sergey Ermolin and Suqiang Song demonstrate how to use Spark BigDL wide and deep and neural collaborative filtering (NCF) algorithms to predict a users probability of shopping at a particular offer merchant during a campaign period. Along the way, they compare the deep learning results with those obtained by MLlibs alternating least squares (ALS) approach.
Merging human and machine learning for everyday solutions

Merging human and machine learning for everyday solutions

November 23, 2019

Li Fan shares insights into how Pinterest improves products based on usage and explains how the company is using AI to predict whats in an image, what a user wants, and what theyll want next, answering subjective questions better than machines or humans alone could achieve.
Smart agriculture: Blending IoT sensor data with visual analytics

Smart agriculture: Blending IoT sensor data with visual analytics

November 21, 2019

Mike Prorock offers an overview of mesur.io, a game-changing climate awareness solution that combines smart sensor technology, data transmission, and state-of-the-art visual analytics to transform the agricultural and turf management market. Mesur.io enables growers to monitor areas of concern, providing immediate benefits to crop yield, supply costs, farm labor overhead, and water consumption.
Sprouted clams and stanky bean: When machine learning makes mistakes

Sprouted clams and stanky bean: When machine learning makes mistakes

November 20, 2019

At AIweirdness.com Janelle Shane posts the results of neural network experiments gone delightfully wrong. But machine learning mistakes can also be very embarrassing or even dangerous. Using silly datasets as examples, Janelle talks about some ways that algorithms fail.
Using ML to improve UX and literacy for young poets

Using ML to improve UX and literacy for young poets

November 19, 2019

Power Poetry is the largest online platform for young poets, with over 350K users. Ann Nguyen explains how Power Poetry is extending the learning potential with machine learning and covers the technical elements of the Poetry Genome, a series of ML tools to analyze and break down similarity scores of the poems added to the site.