System software goes weird

System software has been a monoculture of C and a land of monoliths, but a new wave of open source system software has started to appear, in languages such as Rust, Go, Haskell, OCaml, and Lua. These projects are developing new sorts of applications and introducing new people to systems programming. Justin Cormack explores what's going on, where it's useful, and the new paths it's leading us down.

When the going gets weird, the weird turn pro.—Hunter S. Thompson For decades, system software has been on its own path, diverging from the wider software practice in many respects: C has been the exclusive language; operating systems are monolithic among the largest software projects; occasional releases, rather than continuous delivery, is the norm; and testing is often relatively limited given the sizes of the projects. But recently a diverse set of open source projects have taken standard programming practice and applied it to systems problems, producing software that looks very different. Justin Cormack explores a selection of these systems, in areas such as networking, security, and operating systems, and outlines what we can learn from them, how they are useful, whether they are useful, how much fun they are, and whether worse is better. Justin discusses a range of projects, including Snabb Switch, which implements a 10-gigabit ethernet driver in LuaJIT, a scripting language often used in games, and can switch multiple 10G streams on commodity servers. It aims to be an Emacs for networking applications. Justin also looks at network stacks in functional languages, perhaps an unusual idea, and explains how unikernels are turning them into a new type of application that does not need a traditional operating system. While this software is unlikely to replace our current stacks, that is not the intention—it is to try to make new, more agile applications that need what have traditionally been “system” components.