Super Fast Real-time Data Processing on Cloud-Native Architecture [I]

While Cloud-Native architecture allows for elastic scaling and much simpler operation, it usually adds opinionated abstractions which may hinder performance or lead to implementation challenges. This …

While Cloud-Native architecture allows for elastic scaling and much simpler operation, it usually adds opinionated abstractions which may hinder performance or lead to implementation challenges. This presentation will go over iguazio’s use-case for building an extremely high-performance, real-time data processing platform on the foundations of a cloud-native architecture and its 12 factors. I will explain how we overcome networking and storage challenges of existing micro-services technologies to process data at the rate of 100Gb/s per node; address data persistency and affinity challenges; implement elastic “server-less” functions; integrate with outbound networking; and use hardware acceleration. The session will also cover the tradeoffs and challenges we observed in working with Kubernetes, Swarm, and DC/OS, and then explain how they are addresses.For background you can read my post in: https://thenewstack.io/hackers-guide-kubernetes-networking/