Research: increasing value, reducing waste

"The Lancet" started the new year with a unique special that looks at the recurring criticism that a lot of the current research is simply irrelevant and wasteful. Under the title "Increasing Value, Reducing Waste" The Lancet published a Series of five papers about research. In the first report Iain Chalmers et al. discuss how decisions about which research to fund should be based on issues relevant to users of research. Next, John Ioannidis et al consider improvements in the appropriateness of research design, methods, and analysis. Rustam Al-Shahi Salman et al. then turn to issues of efficient research regulation and management. Next, An-Wen Chan et al. examine the role of fully accessible research information. Finally, Paul Glasziou et al. discuss the importance of unbiased and usable research reports. These papers set out some of the most pressing issues, recommend how to increase value and reduce waste in biomedical research, and propose metrics for stakeholders to monitor the implementation of these recommendations.

Admittedly this is all written from the perspective of biomedical research but a lot of the issues described can be found in all scientific disciplines and unfortunately represent a more general trend. What makes this rather special is the fact that all studies don't stop at stating the facts and telling us inconvenient truths but make suggestions how the sciences can improve. All contributions consequently focus on the system and do not look at external factors. Although my initial reaction was skeptical as there are certainly examples how misdirection through external factors can create irrelevant research and hamper important scientific work. Canadians can tell you a thing or two about that. But instead of simply complaining the approach is rather "Here is what we think is wrong and here is what we think we can do to make it better".

Nevertheless the criticism is harsh: institutional incentive systems are counterproductive, money is wasted, patients put at risk, and the entire system skewed. In the editorial commentary Sabine Kleinert and Richard Horton state There is clearly a strong feeling among many scientists ... that something has gone wrong with our system for assessing the quality of scientific research.

And they go on citing two Nobel prize laureates, Randy Schekman, and Peter Higgs, both giving interviews in The Guardian. Schekman used the attention he received as Nobel laureate in 2013 and started a ferocious attack against the lead journals: These luxury journals [Nature, Science, and Cell] are supposed to be the epitome of quality, publishing only the best research. Because funding and appointment panels often use place of publication as a proxy for quality of science, appearing in these titles often leads to grants and professorships. But the big journals' reputations are only partly warranted. While they publish many outstanding papers, they do not publish only outstanding papers. Neither are they the only publishers of outstanding research.

Peter Higgs (yes, that's the guy who predicted the Higgs Boson) actually described himself as an embarrassment to his University department because he published so little: Today I wouldn't get an academic job. It's as simple as that. I don't think I would be regarded as productive enough.

Back in 2009 two of the senior authors of publications in this new series published a paper titled Avoidable waste in the production and reporting of research evidence.  They made the extraordinary claim that as much as 85% of research investment was wastefully spend. An incredible figure. According to the authors researchers often start with wrong objectives, work with insufficient study designs, and - a cardinal sin - they ignore similar research that has already been done. The study also states clearly that this is not necessary the result of sloppy or bad research but rather a problem of the system, e.g. all too often results are not accessible to everyone.

I think it is worth to look closer at all of the five papers and in the next posts I will take a stab at transferring their messages to research in general and to biodiversity science in particular.