An excerpt: Open your minds and share your results
An open approach is the best way to maximize the benefits of research for both scientists and the public, says Geoffrey Boulton
There is a compelling case for having open access to scientific papers, to enhance the efficacy and reach of scientific communication. But important though this is, the open-access debate has drawn attention away from a deeper issue that is at the heart of the scientific process: that of ‘open data’. In an attempt to focus much-needed attention on this subject, I chaired a group that produced Science as an Open Enterprise, a policy report from the Royal Society in London, published last week.
Open enquiry has been at the heart of science since the first scientific journals were printed in the seventeenth century. Publication of scientific theories — and the supporting experimental and observational data — permits others to identify errors, to reject or refine theories and to reuse data. Science’s capacity for self-correction comes from this openness to scrutiny and challenge.
…
In the Royal Society report, we argue that this procedure must become the norm, required by journals and accepted by the scientific community as mandatory. As scientists, we have some way to go to achieve this. A recent study of the 50 highest-impact journals in biomedicine showed that only 22 required specific raw data to be made available as a condition of publication. Only 40% of papers fully adhered to the policy and only 9% had deposited the full raw data online (A. A. Alsheikh-Ali et al. PLoS ONE 6, e24357; 2011).
We also need to be open towards fellow citizens. The massive impact of science on our collective and individual lives has decreased the willingness of many to accept the pronouncements of scientists unless they can verify the strength of the underlying evidence for themselves. The furore surrounding ‘Climategate’ — rooted in the resistance of climate scientists to accede to requests from members of the public for data underlying some of the claims of climate science — was in part a motivation for the Royal Society’s current report. It is vital that science is not seen to hide behind closed laboratory doors, but engages seriously with the public.
Full editorial at Nature here: http://www.nature.com/news/open-your-minds-and-share-your-results-1.10895
==============================================================
Public Availability of Published Research Data in High-Impact Journals
Alawi A. Alsheikh-Ali, Waqas Qureshi, Mouaz H. Al-Mallah, John P. A. Ioannidis2,6,7,8,9*
Background
There is increasing interest to make primary data from published research publicly available. We aimed to assess the current status of making research data available in highly-cited journals across the scientific literature.
Methods and Results
We reviewed the first 10 original research papers of 2009 published in the 50 original research journals with the highest impact factor. For each journal we documented the policies related to public availability and sharing of data. Of the 50 journals, 44 (88%) had a statement in their instructions to authors related to public availability and sharing of data. However, there was wide variation in journal requirements, ranging from requiring the sharing of all primary data related to the research to just including a statement in the published manuscript that data can be available on request. Of the 500 assessed papers, 149 (30%) were not subject to any data availability policy. Of the remaining 351 papers that were covered by some data availability policy, 208 papers (59%) did not fully adhere to the data availability instructions of the journals they were published in, most commonly (73%) by not publicly depositing microarray data. The other 143 papers that adhered to the data availability instructions did so by publicly depositing only the specific data type as required, making a statement of willingness to share, or actually sharing all the primary data. Overall, only 47 papers (9%) deposited full primary raw data online. None of the 149 papers not subject to data availability policies made their full primary data publicly available.
Conclusion
A substantial proportion of original research papers published in high-impact journals are either not subject to any data availability policies, or do not adhere to the data availability instructions in their respective journals. This empiric evaluation highlights opportunities for improvement.
================================================================
h/t to Dr. Leif Svalgaard
Related articles
- Royal Society urges era of open research data (blogs.nature.com)
- Open Knowledge Foundation: Opening up scientific data with CKAN (okfn.org)
- Published research should be free and open to all, says academic panel (independent.co.uk)
- Set science free from publishers’ paywalls (newscientist.com)
- Science ‘should be open to all’ (bbc.co.uk)
UPDATE: Steve McIntyre weighs in here
This would be the only reason to justify paywalls: for journals to maintain such a server that would hold this data in electronic form and provide it free of charge. The researchers themselves should also be required to hold paper or electronic raw data for a number of years (say 50) and agree to make such raw data available. I have no quarrel with a researcher charging for the costs incurred to copy that raw data and send it. Refusals should result in closed doors to further journal publication. And when I say data, that includes proprietary computer code used to crunch the numbers. Since public money is being used for this research, I believe this is a federal issue that requires political action in the house and senate to pass such a federal regulation. Public schools who want federal money, must adhere to mountains of regulations in order to use public funds. Researchers should also be under that thumb.
Finally!! What took Nature so long to reach this blindingly obvious conclusion? It’s unfortunate the blogoshpere had to drag them kicking and screaming to recommend doing the right thing. Now let’s see how many journals actually follow these recommendations.
We are witnessing a sea change within the scientific community.
The truth will set us free.
Actions speak louder than words.
I like the “highlights opportunities for improvement” bit. Puts a very positive spin on the conclusion. Very non-offensive and a clever choice of phrase.
Data storage is incredibly cheap these days and there is plenty of open source and freeware software available, so cost is not really an issue and hasn’t been an issue for quite some time. The data can easily be stored as XML files.
I like this part:
“Yet this, too, presents a problem. Too often, we scientists seek patterns in data that reflect our preconceived ideas. And when we do publish the data, we too frequently publish only those that support these ideas. This cherry-picking is bad practice and should stop.”
I am very happy to see this subject is finally being addressed. It is about time!
Thanks to M&M and the others who have been raising a dust about the availability of data. Without you this subject would never have seen light of day and science would be much worse off down the road as “No Supporting Data” became the norm and entrenched through the years.
Elaine Benes responds: “Get out!!!”
I’ll believe when I see it.
In one sense it is sad that this post, and thread in general, has to be asserted. Clearly the implication is that science has become the stuff of secrecy and avarice. We’ll see how the Climate Druids respond, but I don’t have high hopes.
“Data on Request” invites reactions like the well known Phil Jones rejoinder!
It is time for researchers with integrity to shun submission to Journals that don’t make complete data sets available, to anyone, credentialed or not.
Paywall elimination is a next worthy goal, although in all honesty I don’t quite know how that paradigm can work.
Suggestions???
My first reaction was that they want to be seen to be first amongst the pack of warmist rats who are bolting at full tilt out of the sinking and burning ship of alarmism in a mad and desperate attempt to keep ahead of the rest of the vermin.
Or am I being unfair?
A fraction of the money our governments spend, building databases to retroactively snoop on us, would provide the greatest public information respurce the world has ever seen – a Library of Alexandria for the Digital Age.
It shows you what our governments real objectives are that they prefer to spend our money on things that destroy, impoverish, and enslave, rather than on those things that construct, enrich, and liberate.
A fraction of the money our governments spend, building databases to retroactively snoop on us, would provide the greatest public information respurce the world has ever seen – a Library of Alexandria for the Digital Age.
It shows what our governments real objectives are that they prefer to spend our money on things that destroy, impoverish, and enslave, rather than on those things that produce, enrich, and liberate.
Is this the Geoffrey Boulton who hid his UEA background prior to his involvement in the ‘independent’ CRU investigation, and ‘lost’ all the emails and notes associated with that inquiry, while still finding time to be the UK Government’s Chief Climate Change advisor?
http://youtu.be/2VFWYfBEtJ8
All sound convenient, slick, opportunistic, and hypocritical…
…Boulton will be remembered solely for his whitewashing dexterity.
Will Mike Mann call Nature a tool of the oil industry?
So let us now support Natures suggestions and welcome them to the camp of open data and debate, to the benefit of all, scientists and public alike. Let the data (and code) tell its own truths
more soylent green! says:
June 28, 2012 at 6:51 am
Data storage is incredibly cheap these days and there is plenty of open source and freeware software available, so cost is not really an issue and hasn’t been an issue for quite some time.
============
Not according to Forest et al.
“data archiving was not feasible given resources available in 2003”
However, in 2000 we were sailing the world with a low cost CD burner aboard. Each CD was capable of storing 650MB and blanks were about $2 each. We were mailing them back home from each port of call as off-site backup.
650MB is huge for storing numerical data such as used in climate models. It is about the same as 5 file cabinets of paper. For a few hundred dollars you could store the equivalent of mountains of paper in a small case.
Apparently the technology to archive data was lost to science in the passage of time between 2000 and 2003.
Just lip service to address skeptic’s main complaint. I doubt any real visible movement. GK
Amen to Pamela Gray’s comment. Anthony, you should be proud of what you have accomplished – and I don’t know how you keep this high level of dedication to WUWT! By providing the world platform for breaking the Climategate scandal and keeping the spotlight on the issues it raised you and your wonderful moderators have shifted the issues in the right direction and, as I’ve said before, you’ve made history.
While Jeff Mitchell comments “I’ll believe it when I see it!” (and to some extent I agree) the fact that the scientific community is being dragged by the neck ( largely by Steve McIntyre and WUWT) into this issue is significant.
Scientists in all fields should be concerned that rather than decreasing, scientific fraud, scandals, and studies that have later proved to be incompetent have only increased in frequency since the issue arose on the public and scientific radar back in the 1980s! Open access to data would decrease the temptation by ‘scientists’ to cheat. This would affect both those who are trying to work beyond their competence, and those who deliberately seek prestige or money by producing ‘science’ that plays on public fears. Not only CAGW but the whole fraudulent MMR vaccine scare come to mind.
What would also help is if members of the mainstream media would stir from their lethargic, slug-like state and do some active research of their own. Publicize more of the wastage of money, warping of public policy, and outright threats to public health and safety by those who distort science for personal gain!
Thank God for the blogosphere.
About time.
Let’s see if Nature enforces this as policy. Otherwise, it’s just noise.
Pamela Gray says:
June 28, 2012 at 6:41 am
This would be the only reason to justify paywalls: for journals to maintain such a server that would hold this data in electronic form and provide it free of charge. The researchers themselves should also be required to hold paper or electronic raw data for a number of years (say 50) and agree to make such raw data available…..
_________________________
Pamela, while I agree, I have a nit to pick, Data should NEVER be tossed. That is why we have a Library of Congress and the equivalent in other countries.
Records kept by the Egyptians, the Chinese and others are now being mined for information.
Is this going to be retroactive? Or does all the garbage previously published stand without being examined.
On a related issue of open information, I was surprised to catch the last part of an item on the Weather Channel last evening regarding “Urban Heat Islands”. In the part which I caught they did not discuss how these are impacting over-all temperature mismeasurment but they at least did a reasonable job of explaining the how and why of their existence to the extent that for many the glaring absence of discussion of their effect upon measured temperature may be obvious.
As soon as science became secretive, it ceased to be science.
If the promoters of CAGW choose to start doing science, and engage in open, reasoned debate, allowing free, unrestricted access to their data, methods, sources, etc, and seeking unfettered publication of dissenting opinion, then we will have science on display for all to see.
I do not believe I will be holding my breath for that day … … …
Regards,
Mark H.
Let us hope that Steve McIntyre is wrong :
http://climateaudit.org/2012/06/28/boultons-nature-editorial/
I look forward to the hockey team, UVA, UEA and IPCC being as transparent as Donald Duck’s pants from now on.
Data retention and availability for the duration of the copyright protection sounds fair and reasonable.
And this is after the term “denier” was plastered all over Bain et al….
Crocodile tears imo.
Nature has been among the worst offenders. It is guilty of embargoing papers, filled with data, criticizing CAGW, while publishing the most shallow, sometimes data-free papers and even pure political critiques from its stable of CAGW writers.
This is but an EDITORIAL. It calls for change, but no one at Nature is home upstairs. They never answered the call in 2009, nor 2011, so why think they will answer this call in 2012?
Jimmy Haigh says:
June 28, 2012 at 7:16 am
My first reaction was that they want to be seen to be first amongst the pack of warmist rats who are bolting at full tilt out of the sinking and burning ship of alarmism in a mad and desperate attempt to keep ahead of the rest of the vermin. Or am I being unfair?
=======
This certainly belongs in our album of funny texts. When one year ago a Dutch Professor had to resign because of a mega fraud, our academy of sciences (KNAW) was quick in assuring us that this was very, very exceptional and that we should not loose, therefore, our trust in science. In the mean time several other Dutch scientists had to resign for similar reasons, suggesting that the KNAW did not have the correct statistics. Of course, we keep our trust till the bitter end.
Boulton seems to get it, more or less.
Wake me when the Royal Society officially disavows pseudoscience and officially kicks out Paul Nurse. I’m expecting a long nap.
Brady says (June 28, 2012 at 8:40 am): “Let us hope that Steve McIntyre is wrong :
http://climateaudit.org/2012/06/28/boultons-nature-editorial/”
Thanks for the link. I’d say Steve nails it.
The unique factor that allowed humans to progress beyond the stage of paleolithic hunter-gatherers is External Memory – the ability to create and preserve information to be shared with others.
The scientific method and mathematics allow others to independently analyze information and to test it for validity.
Data hoarding may temporarily protect the egos of individuals (such as Tycho Brahe), but it does nothing that endures to support society, to advance the state of the science or the material arts, or to benefit mankind (or ‘Mann-kind’, either).
All claims are conditional until independently validated. Even Einstein was a ‘crank’ until his predictions were verified as correct by Arthur Eddington (1919), and again by Robert Pound and Glen A. Rebka Jr. (1959).
I do hope that Messrs Mann, Forest and Jones and all their obstructive little data squirrelling chums read this and squirm. Their childish and unprofessional playground games have done a huge amount to bring climatology into disrepute, and have heaped ordure upon their personal reputations.
History will not look kindly upon them.
@tomwys says:
“Paywall elimination is a next worthy goal, although in all honesty I don’t quite know how that paradigm can work. Suggestions???”
++++++++
I believe the plan is that you pay to get published. The feeling is that institutions will pay to get a paper out there, after which it will be freely available. As electronic publishing is cheap, it may appeal more as the year go by.
Universities often pay the Department for each paper published so it is likely this is not going to be a big problem. It is tax deductible (it’s an expense) so it could be made even more attractive.
The huge advantage is it will be on balance far cheaper than other methods of giving people access to publicly funded research. Readership would expand. The success of Professor Sebastian Thrun, formerly of Stanford University is really causing heads to turn. Education is heading for a huge transition in an online universe.
He explains why he gave up his tenure at Stanford to start a new online university that offers all courses for free. http://www.youtube.com/watch?v=SkneoNrfadk
Such things will be common in future and open access to knowledge a prerequisite.
Ignorance not being a virtue, we must seek to elevate the masses with mass methods. CAGW is nearly the last gasp of ‘science as priestcraft’ – the anointed insiders plotting against the majority using the majority’s resources to do it. Loved the reference above to the Druids. They refused point blank to share their information with the Romans and the info died with them – thousands of years of collected astronomical and physical science, slaughtered for nothing. I doubt there are many climate martyrs plotting at their desks. You can already identity the lifeboats they are scrambling to get into: smoke inhalation, Black Carbon, water shortages, meteor impact, SARS…anything with a big budget upside for research.
JayHd above asked:
“Is this going to be retroactive? Or does all the garbage previously published stand without being examined.”
Bingo.
I can’t think of any compelling case to NOT have open access to all scientific research data.
So will “Nature” change?
AlexS says:
June 28, 2012 at 12:38 pm
So will “Nature” change?
_____________________________
NO
What is needed is new Journals that do insist on data and meta data availability to compete against the older journals. Unfortunately it is a closed loop clique at this point. With the increase in Science Fraud publicity we may just start seeing those newer journals though.
For example the New York Times editiorial: A Sharp Rise in Retractions Prompts Calls for Reform from 4/17/2012
I tend to agree with Jimmy Haigh @ June 28, 2012 at 7:16 am
The public at large is beginning to see the pedestal that “Science” stands upon is built of bull feces and I think we are seeing the start of a mad dash to plaster over the bull feces.
Richard Drake at Climate Audit has a rather interesting take on Boulton’s motive.
I have added Dr Phil Jones famous little rejoinder to the Blessed Steve in the Nature comments section. Let’s see how long it stays there…. 🙂
Crispin in Waterloo says, June 28, 2012 at 11:31 am:
” [ … ] Universities often pay the Department for each paper published so it is likely this is not going to be a big problem. [ … ] ”
===========
I discussed this with my wife, an academic and researcher who much maligns university publishing requirements. Some of this money flows back to the individual’s ‘research’ account to offset costs of attendance at conferences, employment of research assistants, etc. Hence the process becomes indelibly corrupted … and we see much very poor quality ‘research’ pronouncements coming out of universities around the world. Look in any any given newspaper on any given day to find some declaration that according to so-and-so university something bad is going to happen to you because you do whatever … I tell you what immediately goes through the mind of the average Joe in the street, “Bullshyte !”
And science in general wonders why it is held in open contempt by the populace?
Dodgy Geezer says:
June 28, 2012 at 2:08 pm
“I have added Dr Phil Jones famous little rejoinder to the Blessed Steve in the Nature comments section. Let’s see how long it stays there…. :)”
Should have included the one where he says he’ll destroy the data before turning it over.
What a novel concept, why didn’t someone think of this sooner? 😉
I could support paywalls for the data with the papers themselves being free.
I hope this editorial is more than “just one more” calling for openness that ends up having no impact.
I wonder by open they mean the data which supporters ‘the cause’ and by access they mean those that support ‘the cause ‘
Fair enough, given no one else clearly counts and has AGW is a self evident truth, or so we are told, there can be no valid evidenced that does not support it.
But if they want to improve things they could start by asking these professionals and well qualified scientists to meet the standards expected of students at university for data collection , version control and statistical validity , if that is not to much for them of course .
from Steve Mc Intyre’s take:
I have to hold Steve’s trustworthy statement of what actually happened, against the large number of nice welcoming comments here saying
and the like.
Nice and welcoming, but naive I think.
Lucy Skywalker says:
June 28, 2012 at 4:24 pm
I have to hold Steve’s trustworthy statement of what actually happened, against the large number of nice welcoming comments here saying
“We are witnessing a sea change within the scientific community. The truth will set us free.”
and the like.
Nice and welcoming, but naive I think.
____________________
Hello Lucy,
I wrote the comment which you quoted (“…seachange…”), early this morning.
It didn’t take long for my hopes to fade and after a long day, not only do I agree with you, but I’ve moved considerably beyond the sentiments which you and Mr. McIntyre have expressed.
Now viewing the editorial in Nature more as subterfuge, considering the day’s profound (for U.S. citizens) political events- which have no bearing on climate change- a conclusion could be drawn that we are not only losing this battle, but that the war itself may have already been lost.
I pretty much reject this approach. Just exposing the data is insufficient. All these climate scientists use models to validate their theories; models which are programs. As a programmer I know any program can do anything it wants including lie. Yes the data should be open. And so should the programs. Without both, someone trying to prove or disprove a theory cannot do it conclusively.
While every Tom Dick and Harry sees an advantage in appending pronouncements about global warming to their research, then there will be publishers who are only too happy to earn their crust by reselling the apocalypse to others.
ferdberple says:
June 28, 2012 at 7:43 am
Electronic storage of data has always been cheap.
In 1976 as a graduate student, tired of carrying boxes of punch cards back and forth to and from the computer center, I bought a 6 inch diameter magnetic tape for $12 out of my grad student stipend. I barely used any of it, but had 4 boxes of punch cards on it.
I also checked the price of hard drive storage. The hard drive was about the size of a cake pan and cost $700 for 1 MG of storage.
The computer runs for my thesis are in the basement. If climate science is an example, I think they can be recycled now.
Jeff Mitchell says: June 28, 2012 at 6:58 am
I’ll believe when I see it.
Climate Scientists says:
I’ll see it when I believe it.
Bob Diaz says:
“I can’t think of any compelling case to NOT have open access to all scientific research data.”
And ‘open access’?? Who do they think they’re really dealing with? The government censors ‘open access’:
http://www.humanevents.com/2012/06/27/bird-group-sues-obama-administration-wind-power
Congratulations Anthony, McIntyre and McKitrick, and others. This would not have happened without you. However, I regret that it is the weasel Boulton washing the whitewash off his fingers in his effort at self rehabilitation.
We swim in the sea of lies…and climate villains like Thomas Stocker (Swiss) aim
at concealing data to conceal their traces/lies…. whereas the traditional honest
British gentleman way (Adam Smith , John Locke and Thomas Hobbes) is
to open up sources and fair discussion…..and here we have a case where
a editorial writer goes back to his good heritage…. 3 x Plus…JS
Why do I get the feeling this a “Do as I say, not as I do” article?
No data, no replication.
No code, no replication.
No replication, no science.
As “science” has moved ever more toward computer driven model fantasies, the ONLY way to validate them is to have complete access to the code and the input data / initializing state. If you don’t have that, all you have is a nice computer video game.
It can work through philanthopic support. There are a lot of immensely rich people in the US and other countries, who could fund journals via endowments without even noticing a dip in their bank accounts.
Having uttered/printed these pieties, they will consider the problem solved, or at least dealt with for the near to medium future. When it again becomes egregiously obvious that the problem was never dealt with, the article will be reworded and reprinted. And so on.
The first test will be the next issue of Nature: will the data and model packages and algorithms be located on open source (or even by-request) archives? For all the articles? Some of them? Any of them?
I’ll put money on the answer to all the above being, “Not a chance.”