SONATA: Scalable Streaming Analytics for Network Telemetry

Current solutions for network telemetry are ill-suited for security or performance troubleshooting as they offer limited expressibility and are harder to scale as …

Current solutions for network telemetry are ill-suited for security or performance troubleshooting as they offer limited expressibility and are harder to scale as the number of monitoring queries or the volume of data increases. Expressing queries as dataflow operations (i.e., map/ reduce/ distinct/ group, etc.) over packet tuples addresses the expressibility problem, but scaling such a system is non-trivial. We observe that network telemetry can benefit from two key observations: (1)~underlying data plane can process the packets at line rate, and (2)~small portion of the total traffic satisfies the query for most network monitoring applications. This talk presents the design and implementation of SONATA, a stream-based network telemetry system that allows an operator to express network-wide queries as dataflow operations over packet tuples. Given a query, SONATA automatically determines the optimal plan, (1)~refining the input query to iteratively zoom in over portions of traffic that satisfy the query, and (2)~partitioning the refined queries across the network switches and the stream processing system. We implement several example queries that are motivated by real-world security and troubleshooting scenarios and quantify the scalability benefits of SONATA for these queries using traffic traces from different production networks.