Clouds and containers: Case studies for big data

Once the data has been captured, how can the cloud, containers, and a data fabric combine to build the infrastructure to provide the business insights? Paul Curtis explores three customer deployments that leverage the best of the private clouds and containers to provide a flexible big data environment.

Today, big data has expanded from traditional big data frameworks to a range of newer technologies, such as Spark, Kafka, and SQL-on-Hadoop. But once the data has been captured, how can the cloud, containers, and a data fabric combine to build the infrastructure to provide the business insights? Paul Curtis explores how three customers built their big data environments. While the approaches were different, all had the same common requirements—reliable, scalable storage and flexibility—and the end goals for each were strikingly similar—the ability to handle large amounts of data and provide insights quickly. One customer optimized for processing speed, another optimized for absolute application portability, and the third optimized for processing application flexibility. However, in all three cases, these customers realized that there was a common need for their data to be available across all of their applications. Their storage requirements became the base that allowed their applications to access the large data sets and to preserve application state. By starting with their storage needs, each of these customers then made choices about the application environment that worked best for their requirements. For example, preserving application state became a major requirement of one customer in order to allow their containerized applications complete portability and scalability. After choosing how to store their data, each customer chose a different path to achieving the goal. The combinations of clouds and containers each of the customers chose directly impacted the success of their big data projects. Join Paul to explore different perspectives and techniques for big data deployments.