Building big data applications on Azure

As big data solutions are rapidly moving to the cloud, it's becoming increasingly important to know how to use Apache Hadoop, Spark, R Server, and other open source technologies in the cloud. Pranav Rastogi walks you through building big data applications on Azure HDInsight and other Azure services.

Apache Hadoop and Spark have proven to be a scalable solution for the enterprise, providing a large ecosystem of advanced analytics and big data tools in a unified framework. However, managing this diverse ecosystem and ensuring that its users are able to obtain maximum performance from their clusters is a difficult task in the cloud. Customers are now looking to leverage the capabilities of Apache Hadoop, Spark, and other open source technologies in the cloud. Cloud providers have infrastructure improvements over on-premises, which one can leverage to run these open source components more reliably and at enterprise scale. Pranav Rastogi walks you through building big data applications on Azure HDInsight—a fully managed cloud Hadoop distribution enabling reliable open source analytics with an industry-leading SLA—and other Azure services. Pranav begins with an overview of Azure HDInsight, which offers managed clusters for Spark, Hive, MapReduce, HBase, Storm, Kafka, and R Server and integrates with a rich ecosystem of open source tools, such as Jupyter, Zeppelin, Eclipse, and IntelliJ, making it easier for developers to get started and increasing their productivity. Pranav then demonstrates how to build different types of solutions, including warehousing, streaming and data science, using open source technologies. As you build these solutions on Azure, you’ll discover how to leverage a suite of Azure services for storage, monitoring, and securing clusters, enabling you to run your applications at scale on petabytes of data in a secure environment.