[Insight-users] REPRODUCIBLE RESEARCH: Article in Science Journal by Jill P. Mesirov

[Luis Ibanez]

"Accessible Reproducible Research"

by Jill P. Mesirov
Broad Institute of Massachusetts Institute of Technology and Harvard
University, Cambridge, MA 02142, USA

Science 22 January 2010:
Vol. 327. no. 5964, pp. 415 - 416
DOI: 10.1126/science.1179653

http://www.sciencemag.org/cgi/content/full/327/5964/415


<quote>

Scientific publications have at least two goals: (i) to announce a
result and (ii) to convince readers that the result is correct.
Mathematics papers are expected to contain a proof complete enough to
allow knowledgeable readers to fill in any details. Papers in
experimental science should describe the results and provide a clear
enough protocol to allow successful repetition and extension.

[...]

Computer and computational scientists refer to this goal as
"reproducible research" (5), a coinage attributed to the geophysicist
Jon Claerbout in 1990, who imposed the standard of makefiles for
construction of all the figures and computational results in papers
published by the Stanford Exploration Project (6). Since that time,
other approaches have been proposed (7–14), including the ability to
insert active scripts within a text document (15) and the use of a
markup language that can produce all of the text, figures, code,
algorithms, and settings used for the computational research (16).

[...]

Toward this end, we propose a Reproducible Research System (RRS),
consisting of two components. The first element is a Reproducible
Research Environment (RRE) for doing the computational work. An RRE
provides computational tools together with the ability to
automatically track the provenance of data, analyses, and results and
to package them (or pointers to persistent versions of them) for
redistribution. The second element is a Reproducible Research
Publisher (RRP), which is a document-preparation system, such as
standard word-processing software, that provides an easy link to the
RRE. The RRS thus makes it easy to perform analyses and then to embed
them directly into a paper. A reader can readily reproduce the
analysis and, in fact, can extend it within the document itself by
changing parameters, data, filters, and so on.

[...]

Similarly, a reader of the document can open a dashboard within the
word processor to view the [...] pipelines. When one selects a table
or figure, the word processor displays the pipeline that produced it.
Just as with opening a spreadsheet, the reader can directly connect to
a [...] server to rerun the calculation, change parameters, or apply
the method to other data. The reader can save the exploratory results
within the document, along with their provenance (for replication) and
annotated text. The document can then be sent to a colleague.

[...]

The centrality of the role of computation in science—from molecular
biology to the social sciences (24)—calls out for a new model for the
way we publish our results. Just as it is routine to include
references in our papers, we should also include our complete
computational methods. Journals can play a key role in making this a
requirement for publication. To facilitate this, we need simple,
intuitive ways to both capture and embed our computational work
directly into our papers. The value of such tools goes beyond mere
documentation. They will encourage the next generation of scientists
to become "active" consumers of scientific publications—not just
looking at the figures and tables, but running computational
experiments to probe the results as they read the paper.

</quote>

Full Article at:
http://www.sciencemag.org/cgi/content/full/327/5964/415