The Guardian criticises Peer Review – Open Review yields, with the help of bloggers, better discovery of scientific flaws

PeerReviewCartoon[1]Eric Worrall writes: The Guardian, a green UK newspaper, has published a fascinating article about the shortcomings of peer review – and praised the growing new model of open review, in which papers are pre-published on the internet, giving anyone an opportunity to comment. Naturally the Guardian author was not talking about global warming, which in Guardian circles remains settled science which cannot be questioned, but the point is well made, and well worth reading.

According to The Guardian;

“some scientists would prefer … that results are announced only after they have passed peer review, ie been checked by experts and published in a reputable journal.

There are many reasons why this will no longer wash. Those days of deference to patrician authority are over, and probably for the better. We no longer take on trust what we are told by politicians, experts and authorities. There are hazards to such scepticism, but good motivations too. Few regret that the old spoonfeeding of facts to the ignorant masses has been replaced with attempts to engage and include the public.

But science itself has changed too. Information and communications technologies mean that not only is it all but impossible to keep hot findings under wraps, but few even try. In physics in particular, researchers put their papers on publicly accessible pre-print servers before formal publication so that they can be seen and discussed, while specialist bloggers give new claims an informal but often penetrating analysis. This enriches the scientific process and means that problems that peer reviewers for journals might not notice can be spotted and debated. Peer review is imperfect anyway – a valuable check but far from infallible, and notoriously conservative.”

The Guardian article then discusses the super luminal neutrino issue from 2011, a successful example of the application of open review, and provides this gem of a conclusion;

“How much more informative it [open review] was than the tidy fictions that published papers often become.”

Related: Recently, The Open Atmospheric Society was announced, with a goal of open peer review.

0 0 votes
Article Rating
Newest Most Voted
Inline Feedbacks
View all comments
September 27, 2014 8:29 pm

Just in case anyone thinks that The Guardian might finally seeing sense..
“Is Alaska the new Florida?
Rising temperatures could spark massive population shifts across the United States”

Reply to  artwest
September 28, 2014 12:58 am

If people are moving to Alaska it’s to get as far away from Washington DC (and America in general) as possible.

The Ghost Of Big Jim Cooley
Reply to  artwest
September 28, 2014 1:30 am

Indeed, but the Guardian is thankfully finished anyway – well, they can’t hold out for much longer:

Reply to  The Ghost Of Big Jim Cooley
September 28, 2014 6:23 am

Obviously a clear correlation with CO2 levels.
Quick, start your combustion engines. We have a chance to finish them off!

Reply to  The Ghost Of Big Jim Cooley
September 28, 2014 7:06 am

That’s probably print runs. You need to consider online readership nowadays – all UK ‘national’ titles are now going global and the Guardian online has specialist sections for US and Australian readers.
This is true of every single traditional ‘newspaper’, but you need to see what they are doing online to determine whether they are losing ‘readers’ or not.

Gerry Lightfoot
Reply to  The Ghost Of Big Jim Cooley
September 28, 2014 8:29 am

They need their mate Mikey to hide the decline – he’s good at that.

Gary Pearse
Reply to  The Ghost Of Big Jim Cooley
September 28, 2014 10:03 am

Wow, the decline in Guardian readership is not asymptotic, its even curved slightly doward since the beginning of the temp “hiatus”, a trajectory with an abrupt end. Probably the free-fall is being shored up by bots or some other artifice perfected by Gore’s reality project. An analysis for a statistical type.

The Ghost Of Big Jim Cooley
Reply to  The Ghost Of Big Jim Cooley
September 28, 2014 12:24 pm

Rtj, what’s going on with newspapers in the UK is interesting (I assume the same is happening globally). They rely on income from advertising. Readership is down, so ad costs have to follow suit. Less money in means less real journalistic quality. Less quality means less readers.

Andrew S
Reply to  The Ghost Of Big Jim Cooley
September 29, 2014 1:14 am

Just INVERT that guardian readership graph and the result is fine. Thats the sceintific way to go about it right?

Reply to  The Ghost Of Big Jim Cooley
September 29, 2014 1:54 am

Before getting too excited all the printed press is facing the same trend. The Guardian however has the largest share – of broadsheet – online readership. The Guardian is largely puerile drivel with silly good vs evil narratives in their articles. It is very hard to read them without feeling condescended.

Reply to  The Ghost Of Big Jim Cooley
September 29, 2014 8:50 am

I bet that graph can be massaged to look like a hockey stick.

Reply to  artwest
September 28, 2014 3:07 am

That Guardian piece about Alaska being the new Florida missed this graph, it’s cooling.

Reply to  Jimbo
September 28, 2014 7:07 am

Maybe more accurate to say it shows oscillatory patterns and is currently on a downtrend.

Billy Liar
Reply to  Jimbo
September 28, 2014 8:56 am

There’s a time constant involved in people moving.

Reply to  Jimbo
September 28, 2014 10:24 am

But it is still the coldest state in America, so the slight warming doesn’t mean much at all. Plus to get to Florida levels would require massive warming, that has never happened before and even the predictions are predicting would happen. That art is just the stupidest thing ever.

September 27, 2014 8:31 pm

I agree. What’s to find fault with other than increased time to acceptance but an excellent trade off?

Joel O’Bryan
September 27, 2014 8:35 pm

The Bicep cosmic inflation team should have spent some more time in Preprint review.
But I agree with Eric W. AGW vested Climate scientists are unlikely to submit to open preprint review. As much arrogance as it is CYA on AGW.

Joel O’Bryan
Reply to  Joel O’Bryan
September 27, 2014 8:48 pm

on deeper reflection, the two examples in the Guardian story (originally published in NYT), are perfect examples of a preconceived verification of expectations (Bicep team), and a surprising unexpected result (FTL neutrinos).
Today, reputation vested climate scientists expect AGW verification and try to find it. And somehow with enough data torture or model tuning, they find it.

Reply to  Joel O’Bryan
September 27, 2014 9:46 pm

….that’s post-normal science. The fix is in before data is even collected.

Reply to  Joel O’Bryan
September 28, 2014 5:45 am

comment image
One of my Dimocrat buddies that I debated posted that. 😉

Reply to  Ed Martin
September 28, 2014 6:24 am

The cartoon strongly reminds me [of] the Wegman dismissal by the alarmists.

Reply to  Ed Martin
September 28, 2014 6:36 am

If that were all that skeptics had to say I wouldn’t be one.
Notice how the cartoonist mocks the obese.

Reply to  Ed Martin
September 28, 2014 6:55 am

The cartoon only shows the end of the conversation since they would have started at the upper left and worked there way to the lower right.
The earlier part probably included:
“Look, they claimed CO2 change always precedes temperature change.”
“Now they are claiming Polar Bears are being threatened by “Global Warming”!”
“Here they are claiming we’ve never been this warm before.”
“Oh, not that Mt. Kilimanjaro icecap melting again.”
“Yikes! The Ocean’s are gonna boil!”
“Look the warming stopped so they changed it to “Climate Change” which will never stop.”
“What? Floating ice melting will cause catastrophic sea level rise?”
(Readers can add more, but you get the idea)

September 27, 2014 8:48 pm

I still haven’t quite figured out Eric’s writing style.
God help the newcomers.

September 27, 2014 8:50 pm

Open per review is one thing that allways calls for correct not estimated or corrected figures to be presented. Not to mention that the person who put a paper for per review must be willing to follow Theories of Science. In other word not to use Fallacies and/or miss taking ALL premisses needed to be true from thesis to conclusion.
One more thing. Before 1995 I never heard that the person or institution who sent a paper to a science journal had to pay for the “old” per review and/or for having the paper published. Friends of mins, Ph.D. in Physic and Medicine then told me that their institution had to pay so much that not all important papers could be published world wide in science journals. Is that really so? When money is needed for a paper to be published and the per review’s critic scholary reading is made by a present paradigm’s scholar, that in itself calls for special observation by all interested in correct information re. any paper published in science.

September 27, 2014 8:54 pm

I like the acknowledgement the that the masses won’t be treated as ignorant, relying only on what the experts dole out as fact. Elites (political and accede if) have always underestimated the intelligence of the general population.

Reply to  Dave
September 27, 2014 8:56 pm

Academic= accede if

Reply to  Dave
September 29, 2014 5:57 am

The Guardian piece doesn’t really talk about the “masses” but about “specialist bloggers”. To me it has more than a whiff of post-normalism about it.

September 27, 2014 9:05 pm

Gee, they just discovered that? (Open review yields better discovery)

September 27, 2014 9:11 pm

Don’t forget to patent your billion dollar business idea the same way. (winky face)

Reply to  Sparks
September 28, 2014 6:25 am

A patent means disclosure, Sparks.

September 27, 2014 9:14 pm
Reply to  Eric Worrall
September 27, 2014 9:21 pm

27 September 2014 11:01am
If debate about these things is to be out in the open, then I has to be properly out in the open. That means no more headlines such as “Scienists prove x”, or “Boffins find y”. The headlines have to be “A team has found data that may provide some support for hypothesis z”.
You’ll very rarely find a paper that uses strong claims, its almost always: “This may be interpreted this way” or “our data are consistent with” or “we believe this could suggest”.
Science by press release is a problem because when debate happens within science, claims are qualified, bets are hedged and caveats liberally scattered across everything. But the general press doesn’t like this. They what simple, strong, confident facts.

O you had to bring that up.

Reply to  Eric Worrall
September 27, 2014 9:23 pm
NZ Willy
Reply to  Eric Worrall
September 27, 2014 9:42 pm

Mind boggling, can’t tell if it’s a self-parody or a troll.

Joel O'Bryan
Reply to  Eric Worrall
September 27, 2014 11:16 pm

Got a hoot out of the “Big Oil allied to the Tea Party is greasing the passage…”
Thanks for the laugh Eric.

Reply to  Eric Worrall
September 27, 2014 11:50 pm

The only “Koch” that JamesMackelroy has to be concerned about are those white lines that keep going up his nose. What an ignorant rant … this is when I do believe in abortion as a means of birth control and licensing of parents to have children.

Reply to  Eric Worrall
September 28, 2014 12:47 am

It’s a wind-up. The only question is how long will it be before the Gaurdian’s gatekeepers catch on and remove it.

Geoff Shorten
Reply to  Eric Worrall
September 28, 2014 12:53 am

Here’s another comment by the same poster…
“I am not a potato! I am a member of the foissil fueled think tank the Northern Latitudes Global Warming Alliance. We are paid by the oil companies to produce and promote Global Warming. Every fraction of a degree is worth millions to us.
More CO2 Now!.
That is our motto and rallying cry. We fund UKIP and other denialists for short term profit. We want to fry the planet. We do not believe in Evolution which is a Communist Plot, and God is going to save us by revealing His plans for our planet shortly, when it is warm enough for Him to come out of Hiding.
Thank you for listening. Watch this spot for more news from the Northern Latitudes Global Warming Alliance (NLGWA). This is a registered trademark.”
…so I think we can assume it’s tongue in cheek.

Reply to  Eric Worrall
September 28, 2014 4:44 am

ROFLMAO – please, that has to be a troll… and it’s well done! I’m sure the true believers would be reading that and thinking, “wait, what? Is this what I believe?”

Reply to  Eric Worrall
September 28, 2014 5:28 am

That’s hilarious!

Reply to  Eric Worrall
September 28, 2014 5:39 am

so much at steak 🙂
I also was hard presses to work out if its parody or theyre off tap

Reply to  Eric Worrall
September 28, 2014 5:45 am

I hate grease in my Tea

Reply to  Eric Worrall
September 28, 2014 7:44 am

It’s satire. Here’s the commenters’ history at the Guardian. Quite a variety.

Michael Wassil
Reply to  Eric Worrall
September 27, 2014 11:34 pm

LOL. One of my posts over here at WUWT got modded yesterday for mentioning N u r e m b u r g and the CAGW perps in the same sentence. Quite different standards of propriety at the Guardian, what!

Reply to  Eric Worrall
September 28, 2014 3:06 am

@Eric Worrall
The penny just dropped with The Guardian readership 🙂
I can’t see how anybody will be surprised at that outcome…….once Dana and Cook started to propagate and preach their “enlightment” of the unquestionable ACC-AGW…..once the Guardiang became another forefront of the SKS on preaching the ACC-AGW as a dogma with a feverish zeal, under the cloack of a scientific certanty, the outcome was quite predictable and expectable…….no any surprise really there.
I used to be a regular commenter at the Guardian for about 4 years…

Reply to  Eric Worrall
September 28, 2014 4:53 am

Thanks, that made my morning!

September 27, 2014 9:16 pm

That change will do good.
Insert, redacted, Sherly Crow video here.

Reply to  ossqss
September 27, 2014 10:19 pm

Is that “one sheet” Sheryl Crow? I thought that was just a rumour.

September 27, 2014 10:28 pm

Now all they need to do is look into mirror, apply their own reasoning, and the transformation will be complete.
Probably not ever going to happen but keep an open thought.

September 27, 2014 10:34 pm

You can buy qualifications, for only £750-£2500 you can become a physicist, or for £5000 you can be a game designer, curtsy of “train to game”. look at what you can buy in education, then look how it functions.

September 27, 2014 10:54 pm

This is serious. If the klimatariat lose the Guardian, they lose the whole upper-middle bedwetter demographic.

lemiere jacques
September 27, 2014 11:32 pm

many researchers are scared that if, by any means , their job was assessed through the importance of its scientific value, they would lose their jobs.
Most of the papers have no scientific value, they don’t contain any science at all. After the paper is published the knowledge is exactly the same. ( it is almost proven in medecine).
Science is the knowledge obtained by the work of people in respect of scientific method. the majority of these people are researchers. I hate the word scientist.

lemiere jacques
Reply to  lemiere jacques
September 27, 2014 11:46 pm

public research have thrived because people used to think that research means more wealth for the society eventually.
When you ask a researcher why should i give you money to do this?most of the time he answers this….research => wealth
researcher must be assessed by researchers….because…only researcher can understand research.
Do you really think that the fact that aper are so hard to read is by chance?
I worship science, i do not worship researchers.

Eyal Porat
September 27, 2014 11:47 pm

All is well but even here sometimes a “review” turns into personal attack on the writers.
When preaching to others one has to keep the standards he preaches for.
But the idea of “crowd review” is the future whether they want it or not. We saw it happen again and again, even with the “Hockey Stick” fiasco.

Scottish Sceptic
September 28, 2014 2:14 am

I suspect the main benefit of “crowd review” is that by pre-publicising papers, it makes it highly embarrassing for established people to reject work just because it questions their own conclusions.

Scottish Sceptic
September 28, 2014 2:18 am

Just a thought – when are the Guardian going to start suggesting “pre-print review” of its own articles?

Reply to  Scottish Sceptic
September 28, 2014 2:44 am

I was just thinking the same thing.
And why stop with the Guardian? Time magazine? NYT? Aren’t all of these publications advocating the notions of equality and diversity? Shouldn’t everyone have an equal chance to review a NYT article on any subject? Shouldn’t stupid and ill informed people have the same opportunity as others? Isn’t that the definition of diversity? Why should just the editor review an article?

September 28, 2014 3:36 am

To me this reads like:
-Peer Review on climate science sucks…
Peer Review = Pal Review
Peer Review as it stands leading to a lot of scientific bullshit accredited as an unquestionable “art thow science”.
Solution to this problem:
“Lets have an “open review” first. Have the public [aka the sceptics] have a go at it with their analysing and scrutinizing [which seems by far better than the Peer Review] and somehow have a better fence guard against the bullshit becoming a scientific certanty and very hard to be retractable.”-
please, someone tell me I got this wrong…

September 28, 2014 5:25 am

I agree with Open Review. Pal review has utterly failed, particularly in the climate sciences.
This was, in effect, the review process for my 2008 paper, published at by Joe d’Aleo. The paper as well as all data and calculations was published for everyone to take a kick at.
To my knowledge, I initiated in early January 2008 the still-heretical notion that dCO2/dt changed ~contemporaneously with temperature and therefore CO2 lagged temperature by about 9 months, and thus CO2 could not primarily drive temperature.
I later learned from Richard Courtney that others (Kuo et al 1990, Keeling et al 1995) had noted the lag but apparently not the dCO2/dt relationship with T.
There was strong objection to my hypo in ClimateAudit, including a paper by Willis entitled “Data Smoothing and Spurious Correlation”, which was much appreciated, but wrong.
We’’ll see where this hypo goes…
Best, Allan

September 28, 2014 6:14 am

One of the main problems, in my view, is plagiarism. There are many unscrupulous people out there, who will rip off your ideas before they are formally printed (and thus formally copyrighted). Yes, it happens, and I have experienced this myself.
In which case, the website that hosts the pre-publication papers will have to be secure and internationally regarded as a proper place of publication, so that the copyright is established at the pre-publication stage, rather than at formal publication.

Reply to  ralfellis
September 28, 2014 6:33 am

This is just a matter of proving you published something. So, it would suffice to make a webcitation, and put a copy into the university archives, and you can prove that it was you who published a certain text on a certain date. Or any other way of getting witnesses.

September 28, 2014 6:16 am

Reblogged this on gottadobetterthanthis and commented:
Good point.

September 28, 2014 7:35 am

I would think that Anthony would have a lot to say about “crowd review”. We all remember this and the flurry it caused:

Reply to  Anthony Watts
September 28, 2014 8:22 am

I seem to remember that “Time Of Observation” was one of the criticisms. Was that even resolvable?

September 28, 2014 7:52 am

“IF” it were actually “unbiased” peer-review by unbiased and truly knowledgeable competent peers of the original writer – unedited and unchanged (unmarred) by unbiased and honest, professional editors presented in journals with unlimited space for as many papers as could be written, then “peer-reviewed”” scientific literature would be a valid source of new knowledge.
But NONE of those ideals are true.
We have biased editors selecting specific articles to wedge into a limited number of journals – vetting those articles by an ANONYMOUS and hidden Star Chamber group of reviewers for an editor who filters not only content, but reviewers and publication delays and precedence!

September 28, 2014 12:23 pm

I don’t know if anyone else brought it up (I skipped through the comments, sorry) but The Economist in the past year ran a a couple of corrosive expositions, like the Guardian, of careless preparation and review of scientific papers, but hasn’t shifted its warmist editorial outlook a speck.

September 28, 2014 12:28 pm

the guardian author is greenpeace activist Phillip Ball – he spent time in a russian jail over a greenpeace protest in the artic seas – to me – his writing has a sober quality even when he might be incensed – so it’s probably not surprising he would take this contrariain position – it surely will rankle many of his comrades
not surprising – he and i are at odds on many issues – for example – i found his article on Richard Mueller’s “volte face” on climate change a cheat – he treated Mueller’s scientific uncertainty (fence sitting) before he took a warmist position the same as GW skepticism – to him – leaping off the fence on the warmist side was treated the same a leaping over the fence from the skeptic side – (yes – i know that its controversial to think that Mueller was ever a fence sitter)
so now i wonder – was Phillip Ball a fence sitter on peer review – or does the guardian article he wrote indicate a volte face on his part

September 28, 2014 12:54 pm

Has everyone missed it? The INVERSE correlation between CO2 and the Guardian’s circulation.
CO2 is OBVIOUSLY destroying the Guardian. They need to have MORE WINDMILLS AND SOLAR COLLECTORS, promptly, before they become extinct.

Lloyd Martin Hendaye
September 28, 2014 4:00 pm

Apparently The Guardian et al. do not understand the essential, primary –indeed, sole– scientific purpose of so-called Peer Review: Not to pass on the truth or falsity of an empirical hypothesis, for only Nature can do that, but to assure that, first, an assertion at-issue is not in prima facie violation of basic principles –no anti-entropic, paranormal, supernatural exegeses need apply– and second, that all analytical procedures, pro-and-con empirical evidence plus relevant documentation are complete, consistent, reported without resort to spurious “adjustments”, ideological bias, wishful thinking.
One glance at any deviant Warmist “paper” will blast these criteria on every count. The fact that backscratching grant-mongers routinely parrot each others’ coprophagous views is not a recommendation but an index of corruption so ingrained, pervasive, that no-one associated with these puerile poseurs in any way retains a shred of personal or professional, never-mind “scientific”, credibility.

September 28, 2014 7:21 pm

The objective is to find the truth (which may be somewhat different that even we think) and hand our kids/grandkids a better world. Anthony did eggzakly the right thing and the Serendipity Surprise was in his meetings later.
We should not try to destroy the Gores/Manns etcetc. But we’ve also found that the Paleo stuff (70 million yrs. of iceball at 4000-12,000 ppm CO2) has no traction as does not 18 yrs. of ‘no-temp.-change’ or ‘minimal real sea-level rise.’ It’s also worthless going after things like 10.10org with their splattered children except to offer to pay their way to meet up with their kindred ISIS. We have to find things which resonate with us and ordinary folks. We are all feeling beings who think, not t’other way round.
Looking @ ‘warmer’ sites, some of them are clearly ‘bought-and-paid-for’ but others really believe the warmista-stuff. Debate will not do it. The bedmap2 data is interesting, especially since the ‘warmers’ say, “yes, the ice in Antarctica is increasing in extent, but is thinner, so still a net loss.” We should push for another such survey in a year. We need hard, incontrovertible data. Annddd… can we put our heads together and come up with simple, obvious experimental ways to show it’s nonsense?
At the end of ICCC9, I suggested to Joe Bast that we invite the Gores/Manns etc. to submit their stuff to a judge and we’d do the same. Ifn they decline, we just use their published works. The judge would highlight conflicting evidences [HADCRUT-goalpost-moving?] and would render 1 of 4 opinions The evidence is strongly in favor of/against cAGW or the evidence leans to or away from same. The judge would be a plain-vanilla or a self-aware, supercomputer.
AR5 final draft p.7: “Due to a low level of scientific understanding there is low confidence in attributing the causes of the observed loss of mass from the Antarctic ice sheet since 1993.” Translation: We have no clue what’s going on down there. But certainty begins at Punta del Fuego, right? Riiight.
The Guardian article is a great gift: We might (gently) ask them if this sentiment also applies to ‘Global Warming’ and if not, why not.

September 28, 2014 7:25 pm

Is this the Guardian starting a very wide and slow 180 degree about turn?

September 28, 2014 9:02 pm

The ivory towers are falling down. Falling down. Falling down. The ivory towers are falling down.

September 29, 2014 1:34 am

The Guardian is a useless rag that transplants articles from other rags. The Alaska/Florida piece first appeared in the New York Times. To all dead tree press scribblers, be careful what you wish for: You might be posted to Alaska.

September 29, 2014 3:16 am

rtj1211 September 28, 2014 at 7:06 am
That’s probably print runs
It is. However, that’s what matters, as they have no paywall online (yet), hence the GMG’s revenues are spiralling downwards. Whilst Rusbridger’s salary spirals ever upwards.

Darkinbad the Brighdayler
September 29, 2014 3:32 am

Opinionated Bloggers may be useful as indeed they are in their sphere but they are no substitute for your peers. Those that compete with you to discover first, for the same funding and academic posts. They know your strengths and weaknesses and from the semi anonymity of the peer review process will happily shoot you down should you fail in rigour. Ain’t no old boys network of back-slappers or idea buggins’ turn in my area of Science. Thankfully

James Strom
Reply to  Darkinbad the Brighdayler
September 29, 2014 8:27 am

Open review is review by everyone or anyone. Peer review is review by a small subset. There is a possibility of bias in creating the subset. Which is more likely to uncover mistakes? The challenge is to create a proper mode of organization for open review.

September 30, 2014 10:12 am

Just to reassure you all
In an article the other day stating that the greens should give up on black Carbon and focus on CO2 as the primary driver of global warming, I posted (as Strativarius) that RSS is on 18 years now and easily passes the ‘Santer’ test.
5 minutes later it was removed
I was obviously off-message

Nick W
September 30, 2014 3:21 pm

It’s funny that the Guardian has been mentioned, and yes, it’s about as sad and prehistoric as it gets> it’s as though they’ve gone back thirty years to when the scam started and it’s readers have ignored all the latest facts like no warming since 1998. It’s difficult if you try telling any of their readers about the facts
I had a three day online argument with a warmist after entering a comment on a Guardian climate article about Alaska becoming the new Florida! He sent me links to sites with all his ‘evidence’ and didn’t see a problem when I pointed out that all his websites were run by environmental activists. He thought it was stupid when I mentioned that science was meant to be neutral and stick to the facts. He then tried to get the upper hand by suggesting that his evidence was imperial as he had quoted the Royal Society in his argument. Again, he didn’t like it when I pointed out that the former chairman Lord Rees was an environmental activist. And again, he didn’t see a problem with this.
As you’ll all know, it’s very difficult to debate with a warmist becuase they see nothing wrong with a one sided and purely environmental activist agenda.

%d bloggers like this: