LAUSR

In “Sharing of science is most likely among male scientists”1 Massen et al. report on an intriguing study. They were interested in the positive response rate of academics whom they asked to share either a paper or a dataset. Controlling for a number of variables, the authors conclude that male scientists are more likely to share but mainly with other male scientists, which they putatively ascribe to an “Old Boy network.” However, upon close inspection, the data do not warrant their conclusions. To begin with, there is a problem with the interaction term on which the authors base their main claim: in our view, it is both misinterpreted and misspecified. Their claim (i.e., “Sharing of science is most likely among male scientists”) involves the effects of Sex of Requester and Sex of Participant. In regression terms, this translates into a two-way interaction (i.e., Sex of Requester: Sex of Participant). Massen et al., however, base their claim on a three-way interaction of the above with Condition (i.e., paper vs. dataset requests). This is questionable because a model with just a two-way interaction and a control for condition is not only more parsimonious and conceptually closer to what they set out to show but also yields a superior Akaike Information Criterion (AIC) value (i.e., a lower one). AIC is the benchmark used by the authors themselves, and it indicates that the three-way interaction is in fact superfluous. Now crucially, in the parsimonious model, the interaction between Sex of requester and Sex of Participant is not significant (p = 0.061). Beyond the specification of this crucial interaction term, we take issue with the authors’ modelling procedure. Upon request, they reported that their model does not include lower-order terms (in particular, the aforementioned crucial interaction term Sex of Requester: Sex of Participant). There are two problems with this. First, this approach violates the ‘Principle of marginality’2,3, constituting an “arbitrary imposition on the model.”4 In principle, there can be reasons to deviate from this principle under specific (and very rare) circumstances5, but there should be an independent motivation for doing so. No such motivation was given, and we cannot see a reason ourselves. Second, omitting the crucial lower-order term sex of requester: sex of participant performs a different test than implied. The three-way interaction without the two-way interaction contrasts male-male paper requests with every other combination of the three predictors. Since this also includes male-male data requests, the statistical meaning of the interaction term is obfuscated. (As an aside: the interaction in the regression pits everything against male-male sharing in the low-cost condition, i.e., paper requests, and not, as implied in the results section, against male-male sharing in the high-cost condition, i.e., data requests). In short, considering both the misinterpretation and the misspecification of the crucial interaction term, no clear connection between the authors’ analysis and their claims remains, in our view. Next, we noticed some problems in the regression model in the supplementary materials and failed to replicate the analysis following the same protocol. The authors use a backward model selection procedure based on AIC. Although one could take issue with such a procedure in hypothesis-driven research (as opposed to exploratory research), we did follow the same procedure, and arrived at different conclusions. Starting from a model that contains all effects that the article lists as significant and using an AIC-based backwards selection procedure, open