February 27, 2012 at 17:41If we want reproducible science, program code must be open
The scientific journal Nature published an article in which it is argued that the source code for programs written for scientific experiments must be open. At the moment, the magazine writes, a paradoxical situation has arisen: the results of a scientific study are published in the public domain, but it can be difficult to repeat the experiment, because the authors of the scientific work do not open the source codes of the programs written by them. But modern scientific research almost always includes intensive computer calculations, analysis of experimental data, and modeling.
Code closure threatens the fundamental principle of science, calling into question the repeatability of a scientific experiment. Given the seriousness of the problem, the authors of the article propose to solve it at a fundamental level, that is, oblige all scientists to publish the source codes of programs under a free license.
Moreover, the authors of scientific papers are required to carefully and thoroughly document the source code so that it is suitable for use by other experimenters, as well as for modifications.
At the moment, there are discussions in the scientific community on this issue. Some magazines have already changed the rules for accepting articles and require the obligatory opening of source codes. But not everyone did that. There is an alternative point of view that it is not necessary to publish the full source code, but just a verbal description of the main algorithm. For example, the same journal Nature does not require the publication of code, but only “a detailed description that is sufficient for others to write similar code and do the same experiment.”
According to the authors of the article, such half measures do not fully ensure the repeatability of the experiment. They also give examples.when organizations published scientific data and a description of the algorithm, but when other experimenters recreated the program code for the description of the algorithm, they received results that were slightly different from the original results. The trial revealed that the cause was errors in the original program. This situation could be avoided if the authors of the scientific work immediately published the source code - and then the errors could be corrected at an early stage.
Code closure threatens the fundamental principle of science, calling into question the repeatability of a scientific experiment. Given the seriousness of the problem, the authors of the article propose to solve it at a fundamental level, that is, oblige all scientists to publish the source codes of programs under a free license.
Moreover, the authors of scientific papers are required to carefully and thoroughly document the source code so that it is suitable for use by other experimenters, as well as for modifications.
At the moment, there are discussions in the scientific community on this issue. Some magazines have already changed the rules for accepting articles and require the obligatory opening of source codes. But not everyone did that. There is an alternative point of view that it is not necessary to publish the full source code, but just a verbal description of the main algorithm. For example, the same journal Nature does not require the publication of code, but only “a detailed description that is sufficient for others to write similar code and do the same experiment.”
According to the authors of the article, such half measures do not fully ensure the repeatability of the experiment. They also give examples.when organizations published scientific data and a description of the algorithm, but when other experimenters recreated the program code for the description of the algorithm, they received results that were slightly different from the original results. The trial revealed that the cause was errors in the original program. This situation could be avoided if the authors of the scientific work immediately published the source code - and then the errors could be corrected at an early stage.