AGU Letter: Journals Should Ask Authors to Publish Results

Dr. Leif Svalgaard directs my attention via email to this letter in Eos, Vol. 93, No. 29, 17 July 2012, which calls for more “open science” in the role of peer review, always a good thing.

I’m repeating parts of it here, a link to the full letter follows.

By Duncan Carr Agnew

Institute of Geophysics and Planetary Physics, University of California, San Diego, La Jolla

The title of this Forum is meant to sound paradoxical: Isn’t the publication of results what AGU journals are for? I argue that in some ways they aren’t, and suggest how to

fix this.

Explaining this apparent paradox requires that we look at the structure of a published

paper and of the research project that produced it. Any project involves many

steps; for those using data to examine some problem the first step (step A) is for

researchers to collect the relevant raw data. Next (step B), they analyze these

data to learn about some phenomenon of interest; this analysis is very often

purely computational. Then (step C), the researchers (we can now call them “the

authors”) arrange the results of this analysis in a way that shows the reader the evidence

for the conclusions of the paper. Sometimes these results appear as a table, but more

often they are shown pictorially, as, for example, a plot of a time series, a map, a

correlation plot, or a cross-section. Finally (step D), the authors state the conclusions to

be drawn from the results presented.

I would claim that almost always what is published (that is, made available to others)

is just the conclusions (step D).

The actual results produced in step B from the data

collected in step A are usually not available: All that a paper usually contains is pictures

made from these results (step C), and only rarely can the actual results (the numbers)

be determined from these pictures.

…

The whole point of papers is to allow scientists to benefit from each others’ research; we could do this much more easily if AGU journals strongly encouraged authors to provide, as text files with some annotation, the information on which they based their conclusions.

Just as with the current option of providing data files in an electronic supplement, these text files would be kept by AGU. Then, a future author could not only cite the paper but also use its results—and researchers could build on each others’ work in ways that now can be very difficult.

One objection to this proposal is the cost of having AGU store all this information—a concern reflected in the existing AGU data policy, which states that AGU will

not archive data sets. But that policy was written in 1996, and the cost of storage is

5 × 10-4 of what it was then. For many papers it is quite likely that the size of the files

for the results would occupy much less storage space than what is now used for the

PDF files for the figures.

…

I suggest two changes in AGU’s data policy to help make results available:

1. It should be considered standard practice (though not an absolute requirement) for authors to include files of numerical results, which would go in an electronic supplement.

…

For some papers the results might be too large to store, but that should not be a reason to omit them in others. In accordance with present policy for materials

in a supplement these numerical files, or thorough descriptions of them, should be

made available to the reviewers.

2. Any numerical information put in the supplement should be provided in forms that

can easily be accessed by future readers. It is common to convert tabular information in

supplements to pdf files. This format is not an absolute barrier to using the numbers

given, but it requires more work than a text file would to convert the contents to

a useful (that is, machine-readable) form.

Also, authors should be allowed to bundle compressed text files using some common utility, such as zip, rather than having to keep them separate, which requires users (and reviewers) to download them one by one.

Much discussion of data policy has been about making raw data openly available. This is certainly desirable but would not, practically speaking, make the results of research available, since getting the authors’ results from raw data would require complete replication of the authors’ original processing. Making such replication easy, called “reproducible research,” was proposed by geophysicist Jon Claerbout in the early 1990s; it has made some, but slow, progress since then [Mesirov, 2010].

Pending the arrival of this utopia, the more modest steps I propose would do much to

make at least some aspects of our work easily reproduced. AGU’s (former) motto,

“unselfish cooperation in research,” would be perfectly exemplified by papers that

contained results that could be used by others.

Why not take this step?

See the full letter here