An Open Letter to Bruce Alberts of Science Magazine

In the February 11, 2011 issue of Science Magazine, an editorial called Making Data Maximally Available has announced a new policy regarding requirements for authors, from the pen of Bruce Alberts, the Editor-in-Chief and two Deputy Editors.

Figure 1. Why it is necessary to see all the data and code to understand a given result. XKCD, 

Dr. Alberts, as I have criticized Science in the past for your its lax enforcement of your its own policies, I wish to equally commend you for the new policy. This is a welcome although as-yet untested step. The new policy, says (formatting and emphasis mine):

Science’s policy for some time has been that “all data necessary to understand, assess, and extend the conclusions of the manuscript must be available to any reader of Science” (see http://www.sciencemag.org/site/feature/contribinfo/). Besides prohibiting references to data in unpublished papers (including those described as “in press”), we have encouraged authors to comply in one of two ways: either by depositing data in public databases that are reliably supported and likely to be maintained or, when such a database is not available, by including their data in the SOM.

However, online supplements have too often become unwieldy, and journals are not equipped to curate huge data sets. For very large databases without a plausible home, we have therefore required authors to enter into an archiving agreement, in which the author commits to archive the data on an institutional Web site, with a copy of the data held at Science. But such agreements are only a stopgap solution; more support for permanent, community-maintained archives is badly needed.

To address the growing complexity of data and analyses, Science is extending our data access requirement listed above to include computer codes involved in the creation or analysis of data.

To provide credit and reveal data sources more clearly, we will ask authors to produce a single list that combines references from the main paper and the SOM (this complete list will be available in the online version of the paper).

And to improve the SOM, we will provide a template to constrain its content to methods and data descriptions, as an aid to reviewers and readers.

We will also ask authors to provide a specific statement regarding the availability and curation of data as part of their acknowledgements, requesting that reviewers consider this a responsibility of the authors.

We recognize that exceptions may be needed to these general requirements; for example, to preserve the privacy of individuals, or in some cases when data or materials are obtained from third parties, and/or for security reasons. But we expect these exceptions to be rare.

This is indeed excellent news. It is a huge step, from Michael Mann claiming that to ask for his code was “intimidation” and he would not reveal it, to Science Magazine requiring code as a condition of publication. As someone who has condemned actions of past Editors of Science Magazine, I give full marks and plaudits to Bruce Alberts and the Deputy Editors of Science Magazine for bringing your requirements in line with the computer-based nature of so much of contemporary science. I encourage Nature and all other scientific journals to adopt this policy.

If I had to pick one person who was responsible for this sea change at Science Magazine, I would point to the long and untiring fight by Steve McIntyre for full disclosure of scientific data, code, and results. Much of my own activism in this field is a direct result of watching him struggle for good science. With rare exceptions he has remained calm and restrained despite an unending string of vitriolic personal attacks and public slurs by many who fatuously claimed to be his scientific betters, and by scientists who foolishly claimed that his math was wrong and should be ignored. He has written a number of letters to the Editors of Science and NSF officials and individuals encouraging them to follow their own policies and require authors to archive datasets.

His generally unruffled demeanor and unwillingness to engage in the gutter tactics of too many leading AGW scientific luminaries has inspired in me an occasionally successful attempt to become less cowboy and more … more … well … let me call it “more Canadian”. Science Magazine owes him a debt, as do we all.

Next, Dr. Alberts, Science Magazine had a very poor record of compliance with your own stated policies under the previous Editorship of Donald Kennedy. Some of these issues are still unresolved, with the magazine repeatedly refusing to take a principled stand and require the authors to either put up or shut up — that is to say, produce the data required under those policies (stated above) in effect at the time of publication, or withdraw the papers. I’m sure Steve M. can tell you exactly which ones are still pending. The fact that Donald Kennedy was willing to give his friends a free pass is absolutely no reason for you to continue his reprehensible actions. In fact, it should give you more reason to clean this up, because you would be removing a stain from your reputation.

Finally, it is important to note that there is a new guardian who will be overseeing your future actions. That guardian does not sleep, does not rest, and looks at everything. It is something called the “AEI”. Some people say that the AEI is a net-roaming, self-replicating spiderbot species named the “Argus-Eyed Intarweb”. Others stoutly maintain it is a more mystical entity, the “All-seeing Eye of the Internet”.

In either case, as I have too often found to my cost, the AEI doesn’t miss anything. Something might not get noticed on day one, but sooner rather than later the AEI gets around to it, and my errors get exposed, my overstatements get trimmed back, my misunderstandings get explained, or my goose gets cooked.

I say this because as we saw with Donald Kennedy, having a policy and rigorously enforcing a policy are very different things. You have the opportunity to clean up past mistakes and to hold authors to true scientific principles. The AEI is watching to see if you do that, and it sees through walls, I’ve got the bite marks on my aspiration to prove it.

My best to you, and congratulations again, very well done and long overdue.

w.