IEEE Computer – Reproducible Research
The July/August 2012 issue of IEEE Computer is dedicated to:
Reproducible Research
(http://www.flickr.com/photos/gr33n3gg/3445868551)
The Editorial:
Reproducible Research: Tools and Strategies for Scientific Computing
by Victoria Stodden, long term advocate of Reproducibility Research and Open Science, introduces a set of fascinating articles on multiple practical approaches to restoring reproducibility verification in scientific research.
“This special issue is comprised of articles contributed by participants in a workshop held in Vancouver in July 2011 called “Reproducible Research: Tools and Strategies for Scientific Computing.”
…
One of our aims was to improve the visibility of the nascent group of tool builders working to facilitate really reproducible research in computational science.
…
This special issue focuses on tools and strategies for reproducible computational science.”
The articles cover a diverse set of topics, and describe tools that are available today (many of them as Open Source software) to empower researchers to facilitate the reproducibility of their work.
They include:
- Making Computations and Publications Reproducible with VisTrails
- Three Dream Applications of Verifiable Computational Results
- CDE: A Tool for Creating Portable Experimental Software Packages
- Virtual Appliances, Cloud Computing, and Reproducible Research
- Code Sharing Is Associated with Research Impact in Image Processing
- Automated Capture of Experiment Context for Easier Reproducibility in Computational Research
- Designing and Implementing a Computing Framework for Image-Guided Radiation Therapy Research
- Testing Scientific Programs
The requirement for research publications to be reproducible goes back to the Dawn of Science, and the early days of the Royal Society, whose motto was (and still is):
“Nullius in Verba”
“Take Nobody’s word for it”
Unfortunately, this essential characteristic of scientific work has been lost today, in large measure due to the misplaced reward system that demands academics to publish large numbers of papers without consideration of whether the work is reproducible or not. The result of this sad state of affairs is that about 90% of scientific papers are not reproducible.
The Reproducible Research movement is working hard on restoring the practice of reproducibility verification by raising awareness, and by providing practical mechanisms for making this part of the daily routine of scientific work once again.
From this collection of articles, two that are of great practical utilitiy are:
- The CDE tool by Philip Guo, which enables the creation of portable Linux (executable) applications as self-contained packages. Think of a minimal Linux platform that is fully-customized for the sole purpose of running one executable.
- The use of Virtual Appliances and Cloud Computing, by Bill Howe, as platforms for enabling the distribution of applications and data used during research work. This would enable others to verify the reproducibility of the research.
This special issue is a very valuable contribution to the Reproducible Research movement and by logical implication, to the Open Science movement.