LAUSR

K AT E R Y N A K O N / S H U TT E R S TO C K .C O M I n November 2021, a collaborative effort called the Reproducibility Project: Cancer Biology announced the results of its eight-year-long attempt to replicate 193 experiments from 53 highimpact preclinical cancer studies. Among the group’s troubling conclusions, the researchers were able to complete only 50 of the experiments from 23 studies.1 Why so few? The researchers reported extreme difficulty in even conducting many of the necessary experiments. None of them were described in enough detail to design a replication without clarification from the original authors, the report asserted, and 32% of those authors never responded to inquiries or declined to share reagents, code, or data. In some cases, extreme clarifications were needed to attempt the replications, and model systems often did not behave as they had been originally described. “Unnecessary friction in the research process may be slowing the advancement of knowledge, solutions, and treatments,” a write-up of the findings concluded. For cancer research, the ability to move exciting findings from bench to bedside “requires us to understand what we’re moving along the way,” project leader Timothy Errington, PhD, director of research at the nonprofit Center for Open Science in Charlottesville, Virginia, tells CytoSource. If other researchers are not able to replicate the conditions under which the initial observations were made, he says, it is difficult to assess their credibility or whether the findings merit further investment. Past efforts had pointed toward a replicability problem in the field of cancer biology. Dr Errington says that the finer level of detail from this study, including why some of the studies could not be replicated, has highlighted potential bottlenecks in the research process that could be targets for corrective action to improve the flow of knowledge. As for the original experiments that the project could complete, Dr Errington says, the replication attempts resulted in significantly weaker evidence for the findings. Positive results, in fact, were, on average, half as likely to be replicable as null results, and the replication effect sizes for those positive results were 85% smaller than the original effect sizes (based on a relatively small sample size, however).2 “Surprising things should be less replicable than non-surprising things,” says Brian Nosek, PhD, executive director of the Center for Open Science, who notes that it was impossible to calculate a precise reproducibility rate because of the number of variables. Replication failures could be due to a false-positive result in the initial experiment, a false-negative result in the replication attempt, or differing conditions between the initial experiment and the replication attempt that created site-specific effects. Even so, Dr Nosek says that it was reasonable to suggest that the credibility of published findings in cancer biology may be less certain than was previously recognized. Marcia McNutt, PhD, president of the US National Academy of Sciences in Washington, DC, notes that experiments using animal models can be especially tricky for independent groups to replicate because of the additional layer of variables in how the animals are housed, fed, and used in the research. Dr Errington adds that the replication project’s collaborators tried to recreate the same experimental conditions and steps as the original studies. “But A troubling lack of replicability for cancer biology studies