October 27, 2019

196 words 1 min read

Building a scalable architecture for processing streaming data on AWS

Building a scalable architecture for processing streaming data on AWS

Analyzing real-time streams of data is becoming increasingly important to remain competitive. Siva Raghupathy and Manjeet Chayel guide attendees through some of the proven architectures for processing streaming data using a combination of cloud and open source tools such as Apache Spark. Watch a live demo and learn how you can easily scale your applications with Amazon Web Services.
Talk Title Building a scalable architecture for processing streaming data on AWS
Speakers Siva Raghupathy (Amazon Web Services), Manjeet Chayel (Amazon Web Services)
Conference Strata + Hadoop World
Conf Tag Big Data Expo
Location San Jose, California
Date March 29-31, 2016
URL Talk Page
Slides Talk Slides
Video

Long gone are the days where big data was confined to batch processing. From video streaming to real-time recommendation engines, preventive maintenance to fraud detection applications, analyzing real-time streams of data is becoming increasingly important to remain competitive. Siva Raghupathy and Manjeet Chayel guide attendees through some of the proven architectures for processing streaming data using a combination of cloud and open source tools. Watch a live demo and learn how you can easily scale your applications with AWS. Topics include: This session is sponsored by Amazon Web Services.

streaming video streaming guide aws open source video big data cloud scalable
comments powered by Disqus
Scala and the JVM as a big data platform: Lessons from Apache Spark

Scala and the JVM as a big data platform: Lessons from Apache Spark

October 21, 2019

The success of Apache Spark is bringing developers to Scala. For big data, the JVM uses memory inefficiently, causing significant GC challenges. Spark's Project Tungsten fixes these problems with custom data layouts and code generation. Dean Wampler gives an overview of Spark, explaining ongoing improvements and what we should do to improve Scala and the JVM for big data.
Building a scalable data science platform with R

Building a scalable data science platform with R

October 27, 2019

Hadoop is famously scalable, as is cloud computing. R, the thriving and extensible open source data science software. . .not so much. Mario Inchiosa and Roni Burd outline how to seamlessly combine Hadoop, cloud computing, and R to create a scalable data science platform that lets you explore, transform, model, and score data at any scale from the comfort of your favorite R environment.
Designing a scalable real-time data platform using Akka, Spark Streaming, and Kafka

Designing a scalable real-time data platform using Akka, Spark Streaming, and Kafka

October 26, 2019

Alex Silva outlines the implementation of a real-time analytics platform using microservices and a Scala stack that includes Kafka, Spark Streaming, Spray, and Akka. This infrastructure can process vast amounts of streaming data, ranging from video events to clickstreams and logs. The result is a powerful real-time data pipeline capable of flexible data ingestion and fast analysis.
IoT in the enterprise: A look at Intel (IoT) Inside

IoT in the enterprise: A look at Intel (IoT) Inside

October 23, 2019

Moty Fania shares Intels IT experience implementing an on-premises big data IoT platform for internal use cases. This unique platform was built on top of several open source technologies and enables highly scalable stream analytics with a stack of algorithms such as multisensor change detection, anomaly detection, and more.
Self-service, interactive analytics at multipetabyte scale in capital markets regulation on the cloud

Self-service, interactive analytics at multipetabyte scale in capital markets regulation on the cloud

October 20, 2019

Scott Donaldson and Matt Cardillo detail the security measures and system architecture needed to bring alive a multipetabyte data warehouse via interactive analytics and directed graphs from several trillions of market events, using HBase, EMR, Hive, Redshift, and S3 technologies in a cost-efficient manner.
Toppling the mainframe: Enterprise-grade streaming under 2 ms on Hadoop

Toppling the mainframe: Enterprise-grade streaming under 2 ms on Hadoop

October 19, 2019

What if we have reached the point where open source can handle massively difficult streaming problems with enterprise-grade durability? Ilya Ganelin presents Capital Ones novel solution for real-time decisioning on Apache Apex. Ilya shows how Apex provides unique capabilities that ensure less than 2 ms latency in an enterprise-grade solution on Hadoop.