October 19, 2019

313 words 2 mins read

Toppling the mainframe: Enterprise-grade streaming under 2 ms on Hadoop

Toppling the mainframe: Enterprise-grade streaming under 2 ms on Hadoop

What if we have reached the point where open source can handle massively difficult streaming problems with enterprise-grade durability? Ilya Ganelin presents Capital Ones novel solution for real-time decisioning on Apache Apex. Ilya shows how Apex provides unique capabilities that ensure less than 2 ms latency in an enterprise-grade solution on Hadoop.

Talk Title Toppling the mainframe: Enterprise-grade streaming under 2 ms on Hadoop
Speakers Ilya Ganelin (Capital One Data Innovation Lab)
Conference Strata + Hadoop World
Conf Tag Big Data Expo
Location San Jose, California
Date March 29-31, 2016
URL Talk Page
Slides Talk Slides
Video

These days everyone is excited about big data and fast data. Capital One has embraced this new generation of technology with open arms. However, as Edward Heinlein was fond of reminding us, “TANSTAFL—There ain’t no such thing as a free lunch.” For many years, there’s been a very real battle around the standard operating model of software. Tech giants like Oracle and IBM have traditionally built massively expensive enterprise-ready products, while the open source community provides free, albeit usually inferior, software. For a product to be enterprise-ready, it must guarantee complete reliability alongside performance and flexibility. There are notable successes in the open source world such as Linux and Open SSH/SSL, but the realm of distributed stream computing has lacked comparable solutions. Capital One set out to find whether we could build or find enterprise-ready technology in the open source world to tackle difficult streaming problems that also provides equivalent performance, durability, and availability as a mainframe computer. Ilya Ganelin details Capital One’s attempt to answer this question in a rigorous and complete way, not just by making a prototype or discovering exciting new tools, but by creating an open source-based, enterprise-ready product that can transparently replace an enormously expensive proprietary solution. Ilya presents Capital One’s novel solution for real-time decisioning on Apache Apex. Topics include:

comments powered by Disqus