What is reproducible research?

Various scientific disciplines have defined and used the terms "replication" and "reproducible" in different (and sometimes contradictory) ways. But in recent years there have been efforts to standardise these terms, particularly in the context of computational research. Here we will use the definitions from Reproducibility and Replicability in Science:

• Replication: obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data.

• Reproducible: obtaining consistent computational results using the same input data, computational steps, methods, code, and conditions of analysis.

Question

If you can't explain your research well enough for someone else to reproduce the results, can you really call what you did "research"?

Creating reproducible research requires good work habits and practices, and also a way to verify that the results are reproducible.

It is sometimes said that reproducibility can be achieved by sharing all of the code and data with a published article, but this is not necessarily sufficient for computationally complex studies. There are many reasons why running the same code with the same data may produce different results.

Tip

It is easier to make our research reproducible by starting at the planning stage, rather than waiting until we've produced results.