Command Execution in Heterogeneous Network at Facebook scale

Facebook's global network has tens of thousands of network devices from a variety of vendors. We need to run commands on these devices for monitoring, provisioning …

Facebook’s global network has tens of thousands of network devices from a variety of vendors. We need to run commands on these devices for monitoring, provisioning and deployment. Running commands on devices from different vendors poses unique technical challenges. To address these challenges, we created a FBNet Command Runner (FCR) thrift service for running commands on devices. The service takes care of vendor specific details of network devices and allows user to focus on business logic. This service now runs commands on tens of thousands of network devices in our network. We have now open sourced the FCR service. Others can now extend this service to adapt it to their network. In this tutorial, we will spend about 10 minutes to walk through an overview of this service and how it ties into network management at Facebook. After that we will do a live tutorial. In this tutorial, we will build a Command Runner service based on FBNet Command Runner. For this tutorial we will be using Python as the programming language.

• We will use linux containers to emulate a set of generic network devices
• Using small modification, we will adapt the FCR to work with these emulated device
• We will be using FCR thrift APIs to run commands on devices using our FCR service.
• In process, we will create a small cli tool to interactively run commands using the FCR.
• Run multiple instances to scale with the size of network