Rigor and reproducibility: What you need to know
Starting in 2012, Brian Nosek, Ph.D., of the University of Virginia, began a project that would attempt to reproduce the results of 100 psychology studies that had been published in three scientific journals. Enlisting the help of several hundred colleagues across the country, the scientists simply re-ran the studies described in the journals to see how many of them would match up to the results of the original publications.
Only 39 of them did.
Nosek’s findings, published in the journal Nature in 2015, are among the reasons that the National Institutes of Health are placing an added emphasis on rigor and reproducibility.
The former begets the latter – or so the hope is – and that’s why new requirements are popping up on grant applications to ensure that researchers are using rigorous, robust methodology that has been carefully planned out.
Among the changes to NIH grants:
- The addition of a “scientific premise” section that discusses the strengths and weaknesses of previous research. Researchers must show how their study is going to fill in some areas of weakness or expand on areas of strength.
- Added emphasis on the research approach, including biostatistical methods, inclusion and exclusion criteria, and diversity in study subjects.
- Requirements that researchers authenticate key biological or chemical resources, to protect against false findings caused by cross contamination.
“The NIH really wants to make sure that the methods you’re using have been planned out correctly,” said Carrie Dykes, Ph.D., research engagement specialist at the CTSI.
Fortunately, the Research Help Desk is available to answer questions and provide expertise about the grant changes. View a presentation detailing the new requirements or email ResearchHelp@urmc.rochester.edu for more information.
Nature also recently posted an article on the subject here.
Michael Hazard | 1/28/2016
