High Performance Computing: The Move from General Purpose Processors to Custom Hardware and the Implications for Linux

Ten years ago the operating system was processing data and was an efficient means of a general abstraction of computer hardware. These days accelerators bypass established kernel data paths in many wa …

Ten years ago the operating system was processing data and was an efficient means of a general abstraction of computer hardware. These days accelerators bypass established kernel data paths in many ways in order to get better performance and latency. We have seen the development of GPUs, FPGA, offload NICs, RDMA technologies, NVMe, offload storage technologies and so on and so on showing a trend that is slowly taking over. One key issue here is that we have basically reached a ceiling in what a general processing core can do. The way to higher performance and faster processing must therefore avoid general processing and move to specialized hardware that can handle data faster. In this talk we investigate the history of the development of the various offload technique and how they are supported currently and suggest a way forward to better integrate accelerators into Linux.