Introduction to Hilbert AutoML with TensorFlow Extended (TFX) at Yahoo! JAPAN

Hilbert is an AI framework that works with TensorFlow Extended (TFX) at Yahoo! JAPAN, which provides AutoML to create production-level deep learning models automatically. Hilbert is currently used by over 20 services of Yahoo! JAPAN. Shin-Ichiro Okamoto details how to achieve production-level AutoML and explores service use cases at Yahoo! JAPAN.

Yahoo! JAPAN provides more than 100 services, which include advertising, search, ecommerce, auction, finance, maps, news, travel, video streaming, gaming, cloud service, data business, etc. The company’s data scientists and engineers have different skill sets, and it was necessary to develop an AI framework to meet their individual needs so everyone can make deep learning models more easily. In order to realize production-level AutoML, the company introduced a complete TFX YAML configuration interface. As a result, it’s now possible to optimize hyperparameters, input feature combinations, input cross-feature combinations, embedding sizes, model types, model architectures, model size, and more at the same time. It would be difficult to achieve all of these optimizations without a TFX configuration interface. Hilbert is an AI framework that works with TFX to provide AutoML to create production-level deep learning models automatically. Hilbert is currently used by over 20 services at Yahoo! JAPAN. Shin-Ichiro Okamoto explains how to achieve production-level AutoML and explores service use cases at Yahoo! JAPAN. He also details the company’s goal to make Hilbert open source.