[Insight-developers] REPRODUCIBILITY: IEEE CVPR 2010, Moved to the Bright Side !!!

[Luis Ibanez]

Hi David,

I agree, it will be fascinating to see how this reshapes the field.

Regarding your concern, I would argue that if the paper is "really"
reproducible, then the review should become trivial.

It should come down to a one-hour exercise of:

1) Download the data
2) Download the software (and potentially build it).
3) Download the scripts with parameters that run the software
4) Going for lunch
5) Coming back and comparing the results with the paper.

There shouldn't be ANY thing left for the reviewer (or the reader)
to guess, or to figure out. The instructions should be quite explicit.

All figures in the paper must be regenerable by doing "make"
on the  materials downloaded from (1,2,3).

On the other hand, preparing the paper will become more involved,
but, again at the benefit of the practices on the field.

Even for the authors themselves, it will be great to have under
CVS an entire structure of the paper, that they can download
and rerun in a matter of minutes to hours.

What we all will learn is that, reproducibility leads to a full
set of good practices.


     Luis

--------------------------------------------------------------------------------------------------
On Fri, Nov 20, 2009 at 3:07 PM, David Doria <daviddoria+itk at gmail.com> wrote:
> This is fantastic! I can't wait to see the acceleration of research this
> causes. The only problem now is getting reviewers to actually use the new
> sections properly... which may not be trivial.
>
> Thanks,
>
> David
>