Tips for Reproducibility

 

"A growing chorus of concern, from scientists and laypeople, contends that the complex system for ensuring the reproducibility of biomedical research is failing and is in need of restructuring" 

Policy: NIH plans to enhance reproducibility

 

What is Reproducibility?

“refers to the ability of a researcher to duplicate the results of a prior study using the same materials and procedures as were used by the original investigator. So in an attempt to reproduce a published statistical analysis, a second researcher might use the same raw data to build the same analysis files and implement the same statistical analysis to determine whether they yield the same results.” 

Report of the Subcommittee on Replicability in Science Advisory Committee to the National Science Foundation Directorate for Social, Behavioral, and Economic Sciences

 

What is Replicability?

“refers to the ability of a researcher to duplicate the results of a prior study if the same procedures are followed but new data are collected. That is, a failure to replicate a scientific finding is commonly thought to occur when one study documents relations between two or more variables and a subsequent attempt to implement the same operations fails to yield the same relations with the new data.” 

Report of the Subcommittee on Replicability in Science Advisory Committee to the National Science Foundation Directorate for Social, Behavioral, and Economic Sciences

 

Rigor and Reproducibility at NIH:

https://www.nih.gov/research-training/rigor-reproducibility
https://grants.nih.gov/reproducibility/index.htm

 

Do you have a “Recipe for Reproducibility?"

1. Quality Assurance: Collecting and Validating Data

2. Describing Data & Providing Documentation: Contextual information and details about data sets needed for discovery, access, use, and reuse

3. Code & scripts: Code associated with analyses and results

4. Deposit & Disseminating Data (and code): Submission to open data repositories 

See README Files for more best practices 

 

Practical Tips for Reproducibility

  1. Plan for reproducibility before you start
    • Write a study plan or set up an electronic lab notebook
    Keep track of things
    • Track changes using version control and document everything in a README file 
  2. Report your research transparently
    • Share your protocols and write manuscripts collaboratively  
  3. Archive & share your materials
    • Share and license your research products 

Download the "Reproducibility Resources & Tools" Handout compiled by Countway Library.

Last Updated: 2018-11-15