Considerations for hardware-accelerated machine learning platforms

The advances we see in machine learning would be impossible without hardware improvements, but building a high-performance hardware platform is tricky. It involves hardware choices, an understanding of software frameworks and algorithms, and how they interact. Mike Pittaro shares the secrets of matching the right hardware and tools to the right algorithms for optimal performance.

Data science workloads are computationally intensive and can generally benefit from increased processing horsepower. However, simply adding a GPU is not always the best choice. Building a high-performance hardware platform is tricky. It involves hardware choices, an understanding of software frameworks and algorithms, and how they interact. Mike Pittaro explores available hardware acceleration options, their unique and interesting features, and key benefits and limitations for data science, focusing on x86 processors, GPUs, Intel’s Xeon Phi processor, and field-programmable gate arrays (FPGAs). Mike then dives into the major libraries and frameworks, reviewing which hardware acceleration is supported and how, with a focus on open source frameworks. Mike also covers specific algorithms, explaining how different techniques have important differences in acceleration potential. Mike concludes with a discussion of real-world practical issues, such as developer and production workflows, training versus classification, and workstation versus cluster implementations, and outlines a summary of the current choices, combinations that work well in practice, decision guidelines, and some pointers on where the industry seems to be heading.