Scaling massive, real-time data pipelines with Go
Jean de Klerk explains what it takes to pipe, parse, filter, and store 10,000 messages a second with Go.
| Talk Title | Scaling massive, real-time data pipelines with Go |
| Speakers | Jean de Klerk (Pivotal) |
| Conference | O’Reilly Open Source Convention |
| Conf Tag | Making Open Work |
| Location | Austin, Texas |
| Date | May 8-11, 2017 |
| URL | Talk Page |
| Slides | Talk Slides |
| Video | |
Passing 10,000 messages a second between apps and data stores is tough. Add on filtering and parsing and you’re bound to run into problems. Pivotal Cloud Foundry handles several factors of that load each day, which means its programs need to be highly concurrent and very careful with state. Jean de Klerk explains what it takes to pipe, parse, filter, and store 10,000 messages a second with Go, sharing some patterns and best practices that can be taken advantage of in order to handle this load.
