Can I read Does the Tool Matter? Exploring Some Causes of Threats to Validity in Mining Software Repositories on EtoBox?

Does the Tool Matter? Exploring Some Causes of Threats to Validity in Mining Software Repositories by Hoess, Nicole; Paradis, Carlos; Kazman, Rick; Mauerer, Wolfgang is a scholarly article available to read on EtoBox.

What is Does the Tool Matter? Exploring Some Causes of Threats to Validity in Mining Software Repositories about?

Software repositories are an essential source of information for software engineering research on topics such as project evolution and developer collaboration. Appropriate mining tools and analysis pipelines are therefore an indispensable precondition for many research activities. Ideally, valid results should not depend on technical details of data collection and processing. It is, however, widely acknowledged that mining pipelines are complex, with a multitude of implementation decisions made by tool authors based on their interests and assumptions. This raises the questions if (and to what extent) tools agree on their results and are interchangeable. In this study, we use two tools to extract and analyse ten large software projects, quantitatively and qualitatively comparing results and derived data to better understand this concern. We analyse discrepancies from a technical point of view, and adjust code and parametrisation to minimise replication differences. Our results indicate that despite similar trends, even simple metrics such as the numbers of commits and developers may differ by up to 500%. We find that such substantial differences are often caused by minor technical d

Author

Hoess, Nicole; Paradis, Carlos; Kazman, Rick; Mauerer, Wolfgang

Published

2025

Language

EN