December 30, 2019

150 words 1 min read

LINE's log analysis platform

LINE's log analysis platform

Data is a very important asset to LINE, one of the most popular messaging applications in Asia. Wataru Yukawa explains how LINE gets the most out of its data using a Hadoop data lake and an in-house log analysis platform.
Talk Title LINE's log analysis platform
Speakers Wataru Yukawa (LINE)
Conference Strata + Hadoop World
Conf Tag Make Data Work
Location Singapore
Date December 6-8, 2016
URL Talk Page
Slides Talk Slides
Video

Data is a very important asset to LINE, one of the most popular messaging applications in Asia. Wataru Yukawa explains how LINE gets the most out of its data using a Hadoop data lake and an in-house log analysis platform based on HDP (Hadoop, Hive), Presto, and Azkaban. Wataru offers an overview of the platform along with some useful tools created at LINE, including yanagishima, which helps analysts check data to create reports or find incorrect data.

messaging data lake hadoop
comments powered by Disqus
Hadoop and Spark at ING: An overview of the architecture, security, and business cases at a large international bank

Hadoop and Spark at ING: An overview of the architecture, security, and business cases at a large international bank

December 12, 2019

Bas Geerdink offers an overview of the evolution that the Hadoop ecosystem has taken at ING. Since 2013, ING has invested heavily in a central data lake and data management practice. Bas shares historical lessons and best practices for enterprises that are incorporating Hadoop into their infrastructure landscape.
How a Spark-based feature store can accelerate big data adoption in financial services

How a Spark-based feature store can accelerate big data adoption in financial services

December 12, 2019

Kaushik Deka and Phil Jarymiszyn discuss the benefits of a Spark-based feature store, a library of reusable features that allows data scientists to solve business problems across the enterprise. Kaushik and Phil outline three challenges they facedsemantic data integration within a data lake, high-performance feature engineering, and metadata governanceand explain how they overcame them.
Real-time Hadoop: What an ideal messaging system should bring to Hadoop

Real-time Hadoop: What an ideal messaging system should bring to Hadoop

October 21, 2019

Application messaging isnt newsolutions include IBM MQ, RabbitMQ, and ActiveMQ. Apache Kafka is a high-performance, high-scalability alternative that integrates well with Hadoop. Can modern distributed messaging systems like Kafka be considered a legacy replacement or is it purely complementary? Ted Dunning outlines Kafka's architectural benefits and tradeoffs to find the answer.
Real-world patterns for continuously deployed advanced analytics

Real-world patterns for continuously deployed advanced analytics

December 30, 2019

How can we drive more data pipelines, advanced analytics, and machine learning models into production? How can we do this both faster and more reliably? Graham Gear draws on real-world processes and systems to explain how it's possible to apply continuous delivery techniques to advanced analytics, realizing business value earlier and more safely.
Smart cities, the smart grid, the IoT, and big data

Smart cities, the smart grid, the IoT, and big data

December 29, 2019

Smart cities and the electricity smart grid have become leading examples of the IoT, in which distributed sensors describe mission-critical behavior by generating billions of metrics daily. Mark Donsky and Syed Rafice show how smart utilities and cities rely on Hadoop to capture, analyze, and harness this data to increase safety, availability, and efficiency across the entire electricity grid.
An architecture for merging fast data and enterprise applications: The SMACK stack

An architecture for merging fast data and enterprise applications: The SMACK stack

December 28, 2019

Big data architectures and enterprise/microservice architectures are slowly converging. Big data is transitioning to "fast data," emphasizing streaming over batch processing, while data processing is growing ubiquitous. Dean Wampler explores the SMACK stackSpark, Mesos, Akka, Cassandra, and Kafkaand explains how it addresses the needs of both fast data and the enterprise.