My sincere thanks to everyone who has provided widespread review of our draft paper. There have been hundreds of suggestions and corrections submitted in comments and in email, and for that I am very grateful. That sort of input is exactly what we hoped for, and such input can only make the paper better, and so far it has.
Edits are being made based on many of those suggestions. My sincere thanks go to WUWT moderator Bob Phelan for help in collating the online comments to remove duplicates and group comments and corrections by category. Using that, I’m hoping to post up a revised draft, addressing many of those comments and corrections in the next day or two. I had hoped to have an update ready today, but the editing is taking more time than I thought initially. I will likely create a separate dedicated page for Watts et al 2012 so that it gets separated from the press release, and can be managed better.
An issue has been identified in the processing of the data used in Watts et al. 2012 that was placed online for review. We thank critics, including Zeke Hausfather and Steve Mosher for bringing that to attention. Particular thanks go to Zeke who has been helpful with emailed suggestions. Thanks also go to Dr. Leif Svalgaard, who has emailed helpful suggestions.
The authors are performing detailed reanalysis of the data for the Watts et al. 2012 paper and will submit a revised paper to a journal as soon as possible, and barring any new issues discovered, that will likely happen before the end of September.
The idea of online pre-peer review, and likely peer review itself, is in my opinion where the future of science publishing lies. I think we’ll all learn useful lessons for that future from this experiment. As the saying goes, nothing ventured, nothing gained.
My sincerest thanks to everyone for their input and consideration.
Look for future updates, along with some technical discussions as we proceed.
UPDATE: A work page has been established for Watts et al 2012 for the purpose of managing updates. You can view it here.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
I believe here is where the line has been drawn and from now on newspapers will have to listen.
In Australia two newspapers have scrapped items by Mr Muller.
Good luck with the journals, Anthony
If the purpose of peer-review is to ensure and improve the quality of published research, then this sort of pre-peer-review certainly should be the future of academic publishing, Anthony. I wish I shared your optimism that it will be widely embraced. Unfortunately, authors & editors who are less concerned with quality of published research than with promoting an ideological agenda are unlikely to be enthusiastic about a process that would weed out the junk science that is their bread and butter.
REPLY: When Johannes Gensfleisch zur Laden zum Gutenberg invented the printing press, he broke the monopoly the Catholic Church had on publishing with their army of scribes. There was much wailing and gnashing of teeth then too. In a few years, we’ll wonder how we ever managed to do effective review in “the old days”. – Anthony
I don’t understand why so many people are getting upset about Anthony releasing his paper online before submitting it to a journal. We have been doing just that for years in economics. When I get a new version of a working paper completed, I upload it to ssrn.com and often get many helpful comments before I submit the paper to a journal.
Can only agree with Daveburton…for the simple reason that the big brains won’t like mere mortals questioning their work. Egos are involved here so I expect that nothing will actually change in regards to “professional” peer review.
Regards
Mailman
Part of the reason this got a large and effective response is the popularity of this website, as well as the topic. Crowd pre-review on more obscure scientific topics on less frequented sites would not fare as well and would not be as effective, though someone should try it to see.
Was the paper submitted to a journal in time for the IPCC deadline, July 31?
If not, why the rush and holiday-cancelling last week?
You can Bet NATURE, Elsevier Publishing will not be happy with this. It is of course the way to go re publishing. BTW interpreting climate is mainly statistics, plus a knowledge of most basic sciences not to hard. I Think meteorology/forecasting demands a lot more specialized knowledge.
I think You should make a press release about this novel process.
“Online peer review, the future of science publishing. Now in process ”
That are maybe the best way to get the original paper noticed in mainstream press.
I completely agree with this form of public sourced editing prior to journal submission and peer review. My first experience with submission left me jaded. We submitted to a journal who’s editor was also experimenting with the same kind of signal and process we were using and sent back a scathing review along with a rejection and suggestion to go elsewhere. So we brought on board another scientist who was experienced in the process we were developing but was not actively researching in this area. He re-worked the paper with us and gave valuable input as to which journal we should submit to next. It worked.
Accepting that kind of help requires a humble attitude and the willingness to broaden authorship, even though the actual research may have been done by just one of the authors. If the work is important enough, build a team around you that is willing to separate the chaff from the wheat and then be prepared to share both accolades and rebuke (you will get both no matter how good the paper is in the end). You, Anthony, appear to have that attitude. I predict you will get published in a peer-reviewed journal. Not by your first choice of journals (a common experience), but you will get published.
I wonder if in the future they will be talking about limited closed reviews and open unlimited reviews.
Just think of the vast number of scientists, mathematicians, academics, engineers and others available in open review. Of course some academics will hate it as it removes their stranglehold and control over the dissemination of knowledge (BTW I am an academic and researcher).
“REPLY: When Johannes Gensfleisch zur Laden zum Gutenberg invented the printing press, he broke the monopoly the Catholic Church had on publishing with their army of scribes. There was much wailing and gnashing of teeth then too. In a few years, we’ll wonder how we ever managed to do effective review in “the old days”. – Anthony”
– The printing press example is probably more apropos than I think many might realise. It caused about a century of turmoil and chaos as it upset the established order at the time. It arguably, for that time, broke more things than it fixed. A similar pattern is following – though much more rapidly – with modern ICT and social media technology (described extremely well by the way in Yochai Benkler’s ‘Wealth of Networks’ available for free online (http://www.benkler.org/Benkler_Wealth_Of_Networks.pdf) – I’d highly recommend it as a read). Obstacles to collective action and participation are now being removed and outside the purview of established monopolies and gatekeepers. What was once a service now becomes a bottleneck in trying to maintain its existence. It’s up to all of us to keep up the pressure and put our full weight behind efforts like Anthony’s!
Just discovered your blog, well done. Good luck with your draft!
Kudos, Anthony. We need more scientists like you and your co-authors willing to take the hits to make the system work better to yield higher quality final results. And for standing up to the “establishment” that wants to quelch debate so they can dominate the discussion for their own reputations and funding/financial rewards. However, you and your supporters (me included) have a long way to go. Billions in government funding for the “right” climate scientists, plus political and taxing power for dishonest, stupid or gullible politicians, means our opponents will not go gently into the good night! As the web becomes even more pervasive, your method will be adopted more and more.
Bill
It’s a brave new world.
Well done Anthony!
Let’s think who would really not like this concept of Anthony’s having his draft paper reviewed online before submitting it for publication and peer review.
Well, all those submitting papers to the IPCC for a start, but Hansen and Mann would stand at the very top of the heap. The public shredding of ‘climate science’ practices, data manipulation and unfounded conclusions is not something these people would want to see.
“Taking one for the team” is an age old expression, meaning doing the right thing by everybody – something the notorious Team would never dream of contemplating.
Keep up the great work, Mr. Watts! Absolutely love your blog.
Its all for naught, McKibbon’s published a paper in the peer reviewed journal “rolling stone” that says its all over
http://m.rollingstone.com/?redirurl=/politics/news/global-warmings-terrifying-new-math-20120719?print=true
crowd sourcing is the way of the future
The alternative to this public vetting is what Mann has done in secret with his “trick”…. scheming with P Jones to gerrymander the paper to serve a political objective… as in HIDE THE DECLINE.
Anthony, you are making scientific history.
@richard pound
Seems like a clear admission that cAGW isn’t about science, that story was under “Politics”
If the peer-review process was working as it is supposed to, and the status quo researchers want it to, there would have been no reason for all the climate wars to-date. Climategate would not have occurred, the FOIA challenges wouldn’t have been necessary and the New Zealand lawsuits would not have happened.
When the printing press products were available to the public, the reason for turmil, chaos and challenges to the Catholic Church flowed from the same reason: the equally peer-reviewed and vetted information process was not working … except for the established order. There was no consideration for “free” information as in these days. The restriction of information to the masses was explicit and public. Today the explicit message is that the masses have a (more than less) right to see the information they want. But it wasn’t practically true.
Mann, Jones et al have only themselves to blame for what has happened. If they hadn’t been using the status quo for purposes good for them and bad for us, we’d still be submitting to JCR and wandering through the (frustrating) procedures for getting a contrary idea published.
EVERY branch of science has been doing this same thing for decades – instead of the internet, however, we have gone to conferences sponsored by our respecitve professional/academic societies and ‘publish’ abstracts and poster sessions so we can get feedback from our colleagues before sending our research out for consideration by the journals. this is absolutely no differnet. And, btw, our pre-publication ‘publications’ do get assigned citable titles and ‘publication’ date by going through that public vetting in a pre-publication mode.
GET OVER IT. If it is the way of the future, why postpone it? Let’s get on with our respective discourses in whatever venue/format and stop all of this idiocy.
I have to hand it to you Anthony, Mr Muller, when he threw his chaff up in the air, apparently never thought about where it might land.
Very cool of you and Professor John Christy to adopt the Muller protocol, in pre-releasing your paper.
Just imagine if the US CONGRESS, was required to publish the verbatim text of EVERY bill on line 30 days before they were allowed to vote on it, so that the bloggers could find the gremlins first so the elected ones, wouldn’t need their unelected lackeys to spend taxpayer money fixing all the typos, and brain coughs as well.
Only raises my opinion of Dr Christy, that he as an academic, would embrace this new paradigm.
Good luck on the paper Anthony, I look forward to having an autographed copy of the final published version.
George