Apache Kafka + Apache Mesos = Highly scalable streaming microservices

Kai Whner shares a highly scalable, mission-critical infrastructure using Apache Kafka and Apache Mesos: Kafka brokers are used as the distributed messaging backbone; Kafkas Streams API embeds stream processing into any external application without the need for a dedicated streaming cluster; and Mesos is used as a scalable infrastructure to leverage the benefits of a cloud-native platform.

Microservices establish many benefits like Agile, flexible development, and deployment of business logic. However, a microservice architecture also creates many new challenges, including increased communication between distributed instances, the need for orchestration, new failover requirements, and resiliency design patterns. Kai Wähner offers an overview of a highly scalable, mission-critical infrastructure using Apache Kafka and Apache Mesos: Kafka brokers are used as the distributed messaging backbone; Kafka’s Streams API embeds stream processing into any external application without the need for a dedicated streaming cluster; and Mesos is used as a scalable infrastructure to leverage the benefits of a cloud-native platform like service discovery, health checks, and failover management. Kai ends with a live demo that shows how to develop real-time applications for your core business with Kafka messaging brokers and Kafka Streams API and how to deploy, manage, and scale them on a Mesos cluster using different deployment options.