Serverless workflows for orchestration hybrid cluster-based and serverless processing

Serverless implementation of core processing is quickly becoming a production-ready solution. However, companies with existing processing pipelines may find it hard to go completely serverless. Serverless workflows unite the serverless and cluster worlds, with the benefits of both approaches. Rustem Feyzkhanov demonstrates how serverless workflows change your perception of software architecture.

Serverless is becoming increasingly popular as a means to organize core processing. However, companies with existing pipelines can find it hard to move completely serverless. Serverless workflows unite the serverless and cluster worlds, with the benefits of both approaches. Serverless workflows enable hybrid architecture with both cluster-based processing with longtime processing or high CPU load jobs and serverless functions for the rest of operations because they’re scalable, simple, and cheap. Additionally serverless workflows enable completely modular processing, where the developer defines the method of implementation, whether third-party services, open source libraries, or native cloud services. Rustem Feyzkhanov compares AWS Step Functions and Azure Logic Apps, discussing their pricing, features, and pros and cons. He also demonstrates a number of applications and their architectures, including an ML/DL pipeline, load testing, image processing, and report generation. You’ll then learn how serverless workflows allow you to conduct production tasks such as A/B testing modules, canary deployments, and error handling.