Analysis of the Xen code review process: An example of software development analytics

The Xen Project's code contributions have been growing 10% a year. However, during this period of growth, the code review process became much slower, leading to issues in the community. Lars Kurth and Daniel Izquierdo explain how software development analytics came to the rescue: it provided surprising insights that allowed the project members to understand issues and take corrective action.

The Xen Project’s code contributions have been growing 10% a year. However, during this period of growth, the code review process became much slower, leading to issues in the community. Code review in the Xen Project—as in many other FOSS projects—is performed on mailing lists. During the last few years, the project observed an increase in the number of messages devoted to code review—in particular, an increase in the number of code review messages per patch series or individual patch. Everyone in the community had a different theory as to the root causes of the issues based on their observations: some developers believed we didn’t have enough reviewers, some felt the project’s maintainers had become more aggressive, and some felt code review was not coordinated enough. Many observations contradicted each other and were based only on opinions. Consequently, key members of the project could not agree on how to deal with the perceived issues. Lars Kurth and Daniel Izquierdo explain why the project decided to use data mining techniques using software development analytics to address the issue. The project needed a detailed analysis to verify which theories were valid, which were not, and which were missed. To do this, the team defined a number of parameters in the code review process to determine if it was deteriorating in some way and pinpoint the root causes of this deterioration, if any. Lars and Daniel cover the project’s journey through a number of stories and explore the techniques that enabled the community to improve their review process. Topics include: All the tools used for the analysis are free/open source software and could be used by anyone willing to reproduce the analysis or perform a similar analysis.