The Guardian criticises Peer Review – Open Review yields, with the help of bloggers, better discovery of scientific flaws

PeerReviewCartoon[1]Eric Worrall writes: The Guardian, a green UK newspaper, has published a fascinating article about the shortcomings of peer review – and praised the growing new model of open review, in which papers are pre-published on the internet, giving anyone an opportunity to comment. Naturally the Guardian author was not talking about global warming, which in Guardian circles remains settled science which cannot be questioned, but the point is well made, and well worth reading.

According to The Guardian;

“some scientists would prefer … that results are announced only after they have passed peer review, ie been checked by experts and published in a reputable journal.

There are many reasons why this will no longer wash. Those days of deference to patrician authority are over, and probably for the better. We no longer take on trust what we are told by politicians, experts and authorities. There are hazards to such scepticism, but good motivations too. Few regret that the old spoonfeeding of facts to the ignorant masses has been replaced with attempts to engage and include the public.

But science itself has changed too. Information and communications technologies mean that not only is it all but impossible to keep hot findings under wraps, but few even try. In physics in particular, researchers put their papers on publicly accessible pre-print servers before formal publication so that they can be seen and discussed, while specialist bloggers give new claims an informal but often penetrating analysis. This enriches the scientific process and means that problems that peer reviewers for journals might not notice can be spotted and debated. Peer review is imperfect anyway – a valuable check but far from infallible, and notoriously conservative.”

http://www.theguardian.com/commentisfree/2014/sep/26/scientists-gravitational-waves-science

The Guardian article then discusses the super luminal neutrino issue from 2011, a successful example of the application of open review, and provides this gem of a conclusion;

“How much more informative it [open review] was than the tidy fictions that published papers often become.”


Related: Recently, The Open Atmospheric Society was announced, with a goal of open peer review.

Advertisements

80 thoughts on “The Guardian criticises Peer Review – Open Review yields, with the help of bloggers, better discovery of scientific flaws

  1. The Bicep cosmic inflation team should have spent some more time in Preprint review.
    But I agree with Eric W. AGW vested Climate scientists are unlikely to submit to open preprint review. As much arrogance as it is CYA on AGW.

    • on deeper reflection, the two examples in the Guardian story (originally published in NYT), are perfect examples of a preconceived verification of expectations (Bicep team), and a surprising unexpected result (FTL neutrinos).
      Today, reputation vested climate scientists expect AGW verification and try to find it. And somehow with enough data torture or model tuning, they find it.

      • The cartoon only shows the end of the conversation since they would have started at the upper left and worked there way to the lower right.
        The earlier part probably included:
        “Look, they claimed CO2 change always precedes temperature change.”
        “Now they are claiming Polar Bears are being threatened by “Global Warming”!”
        “Here they are claiming we’ve never been this warm before.”
        “Oh, not that Mt. Kilimanjaro icecap melting again.”
        “Yikes! The Ocean’s are gonna boil!”
        “Look the warming stopped so they changed it to “Climate Change” which will never stop.”
        “What? Floating ice melting will cause catastrophic sea level rise?”
        (Readers can add more, but you get the idea)

  2. Open per review is one thing that allways calls for correct not estimated or corrected figures to be presented. Not to mention that the person who put a paper for per review must be willing to follow Theories of Science. In other word not to use Fallacies and/or miss taking ALL premisses needed to be true from thesis to conclusion.
    One more thing. Before 1995 I never heard that the person or institution who sent a paper to a science journal had to pay for the “old” per review and/or for having the paper published. Friends of mins, Ph.D. in Physic and Medicine then told me that their institution had to pay so much that not all important papers could be published world wide in science journals. Is that really so? When money is needed for a paper to be published and the per review’s critic scholary reading is made by a present paradigm’s scholar, that in itself calls for special observation by all interested in correct information re. any paper published in science.

  3. I like the acknowledgement the that the masses won’t be treated as ignorant, relying only on what the experts dole out as fact. Elites (political and accede if) have always underestimated the intelligence of the general population.

    • Sudders
      27 September 2014 11:01am
      If debate about these things is to be out in the open, then I has to be properly out in the open. That means no more headlines such as “Scienists prove x”, or “Boffins find y”. The headlines have to be “A team has found data that may provide some support for hypothesis z”.
      You’ll very rarely find a paper that uses strong claims, its almost always: “This may be interpreted this way” or “our data are consistent with” or “we believe this could suggest”.
      Science by press release is a problem because when debate happens within science, claims are qualified, bets are hedged and caveats liberally scattered across everything. But the general press doesn’t like this. They what simple, strong, confident facts.

      O you had to bring that up.

      • Got a hoot out of the “Big Oil allied to the Tea Party is greasing the passage…”
        Thanks for the laugh Eric.

      • The only “Koch” that JamesMackelroy has to be concerned about are those white lines that keep going up his nose. What an ignorant rant … this is when I do believe in abortion as a means of birth control and licensing of parents to have children.

      • Here’s another comment by the same poster…
        “I am not a potato! I am a member of the foissil fueled think tank the Northern Latitudes Global Warming Alliance. We are paid by the oil companies to produce and promote Global Warming. Every fraction of a degree is worth millions to us.
        More CO2 Now!.
        That is our motto and rallying cry. We fund UKIP and other denialists for short term profit. We want to fry the planet. We do not believe in Evolution which is a Communist Plot, and God is going to save us by revealing His plans for our planet shortly, when it is warm enough for Him to come out of Hiding.
        Thank you for listening. Watch this spot for more news from the Northern Latitudes Global Warming Alliance (NLGWA). This is a registered trademark.”
        …so I think we can assume it’s tongue in cheek.

      • ROFLMAO – please, that has to be a troll… and it’s well done! I’m sure the true believers would be reading that and thinking, “wait, what? Is this what I believe?”

    • LOL. One of my posts over here at WUWT got modded yesterday for mentioning N u r e m b u r g and the CAGW perps in the same sentence. Quite different standards of propriety at the Guardian, what!

    • @Eric Worrall
      The penny just dropped with The Guardian readership 🙂
      ————————-
      I can’t see how anybody will be surprised at that outcome…….once Dana and Cook started to propagate and preach their “enlightment” of the unquestionable ACC-AGW…..once the Guardiang became another forefront of the SKS on preaching the ACC-AGW as a dogma with a feverish zeal, under the cloack of a scientific certanty, the outcome was quite predictable and expectable…….no any surprise really there.
      I used to be a regular commenter at the Guardian for about 4 years…
      cheers

  4. Now all they need to do is look into mirror, apply their own reasoning, and the transformation will be complete.
    Probably not ever going to happen but keep an open thought.

  5. You can buy qualifications, for only £750-£2500 you can become a physicist, or for £5000 you can be a game designer, curtsy of “train to game”. look at what you can buy in education, then look how it functions.

  6. many researchers are scared that if, by any means , their job was assessed through the importance of its scientific value, they would lose their jobs.
    Most of the papers have no scientific value, they don’t contain any science at all. After the paper is published the knowledge is exactly the same. ( it is almost proven in medecine).
    Science is the knowledge obtained by the work of people in respect of scientific method. the majority of these people are researchers. I hate the word scientist.

    • public research have thrived because people used to think that research means more wealth for the society eventually.
      When you ask a researcher why should i give you money to do this?most of the time he answers this….research => wealth
      researcher must be assessed by researchers….because…only researcher can understand research.
      Do you really think that the fact that aper are so hard to read is by chance?
      I worship science, i do not worship researchers.

  7. All is well but even here sometimes a “review” turns into personal attack on the writers.
    When preaching to others one has to keep the standards he preaches for.
    But the idea of “crowd review” is the future whether they want it or not. We saw it happen again and again, even with the “Hockey Stick” fiasco.

  8. I suspect the main benefit of “crowd review” is that by pre-publicising papers, it makes it highly embarrassing for established people to reject work just because it questions their own conclusions.

    • I was just thinking the same thing.
      And why stop with the Guardian? Time magazine? NYT? Aren’t all of these publications advocating the notions of equality and diversity? Shouldn’t everyone have an equal chance to review a NYT article on any subject? Shouldn’t stupid and ill informed people have the same opportunity as others? Isn’t that the definition of diversity? Why should just the editor review an article?

  9. To me this reads like:
    -Peer Review on climate science sucks…
    Peer Review = Pal Review
    Peer Review as it stands leading to a lot of scientific bullshit accredited as an unquestionable “art thow science”.
    Solution to this problem:
    “Lets have an “open review” first. Have the public [aka the sceptics] have a go at it with their analysing and scrutinizing [which seems by far better than the Peer Review] and somehow have a better fence guard against the bullshit becoming a scientific certanty and very hard to be retractable.”-
    please, someone tell me I got this wrong…
    cheers

  10. I agree with Open Review. Pal review has utterly failed, particularly in the climate sciences.
    This was, in effect, the review process for my 2008 paper, published at icecap.us by Joe d’Aleo. The paper as well as all data and calculations was published for everyone to take a kick at.
    http://icecap.us/index.php/go/joes-blog/carbon_dioxide_in_not_the_primary_cause_of_global_warming_the_future_can_no/
    To my knowledge, I initiated in early January 2008 the still-heretical notion that dCO2/dt changed ~contemporaneously with temperature and therefore CO2 lagged temperature by about 9 months, and thus CO2 could not primarily drive temperature.
    I later learned from Richard Courtney that others (Kuo et al 1990, Keeling et al 1995) had noted the lag but apparently not the dCO2/dt relationship with T.
    There was strong objection to my hypo in ClimateAudit, including a paper by Willis entitled “Data Smoothing and Spurious Correlation”, which was much appreciated, but wrong.
    http://climateaudit.org/2008/02/12/data-smoothing-and-spurious-correlation/
    We’’ll see where this hypo goes…
    Best, Allan

  11. One of the main problems, in my view, is plagiarism. There are many unscrupulous people out there, who will rip off your ideas before they are formally printed (and thus formally copyrighted). Yes, it happens, and I have experienced this myself.
    In which case, the website that hosts the pre-publication papers will have to be secure and internationally regarded as a proper place of publication, so that the copyright is established at the pre-publication stage, rather than at formal publication.
    Ralph

    • This is just a matter of proving you published something. So, it would suffice to make a webcitation, and put a copy into the university archives, and you can prove that it was you who published a certain text on a certain date. Or any other way of getting witnesses.

  12. “IF” it were actually “unbiased” peer-review by unbiased and truly knowledgeable competent peers of the original writer – unedited and unchanged (unmarred) by unbiased and honest, professional editors presented in journals with unlimited space for as many papers as could be written, then “peer-reviewed”” scientific literature would be a valid source of new knowledge.
    But NONE of those ideals are true.
    We have biased editors selecting specific articles to wedge into a limited number of journals – vetting those articles by an ANONYMOUS and hidden Star Chamber group of reviewers for an editor who filters not only content, but reviewers and publication delays and precedence!

  13. I don’t know if anyone else brought it up (I skipped through the comments, sorry) but The Economist in the past year ran a a couple of corrosive expositions, like the Guardian, of careless preparation and review of scientific papers, but hasn’t shifted its warmist editorial outlook a speck.

  14. the guardian author is greenpeace activist Phillip Ball – he spent time in a russian jail over a greenpeace protest in the artic seas – to me – his writing has a sober quality even when he might be incensed – so it’s probably not surprising he would take this contrariain position – it surely will rankle many of his comrades
    not surprising – he and i are at odds on many issues – for example – i found his article on Richard Mueller’s “volte face” on climate change a cheat – he treated Mueller’s scientific uncertainty (fence sitting) before he took a warmist position the same as GW skepticism – to him – leaping off the fence on the warmist side was treated the same a leaping over the fence from the skeptic side – (yes – i know that its controversial to think that Mueller was ever a fence sitter)
    so now i wonder – was Phillip Ball a fence sitter on peer review – or does the guardian article he wrote indicate a volte face on his part

  15. Has everyone missed it? The INVERSE correlation between CO2 and the Guardian’s circulation.
    CO2 is OBVIOUSLY destroying the Guardian. They need to have MORE WINDMILLS AND SOLAR COLLECTORS, promptly, before they become extinct.

  16. Apparently The Guardian et al. do not understand the essential, primary –indeed, sole– scientific purpose of so-called Peer Review: Not to pass on the truth or falsity of an empirical hypothesis, for only Nature can do that, but to assure that, first, an assertion at-issue is not in prima facie violation of basic principles –no anti-entropic, paranormal, supernatural exegeses need apply– and second, that all analytical procedures, pro-and-con empirical evidence plus relevant documentation are complete, consistent, reported without resort to spurious “adjustments”, ideological bias, wishful thinking.
    One glance at any deviant Warmist “paper” will blast these criteria on every count. The fact that backscratching grant-mongers routinely parrot each others’ coprophagous views is not a recommendation but an index of corruption so ingrained, pervasive, that no-one associated with these puerile poseurs in any way retains a shred of personal or professional, never-mind “scientific”, credibility.

  17. The objective is to find the truth (which may be somewhat different that even we think) and hand our kids/grandkids a better world. Anthony did eggzakly the right thing and the Serendipity Surprise was in his meetings later.
    We should not try to destroy the Gores/Manns etcetc. But we’ve also found that the Paleo stuff (70 million yrs. of iceball at 4000-12,000 ppm CO2) has no traction as does not 18 yrs. of ‘no-temp.-change’ or ‘minimal real sea-level rise.’ It’s also worthless going after things like 10.10org with their splattered children except to offer to pay their way to meet up with their kindred ISIS. We have to find things which resonate with us and ordinary folks. We are all feeling beings who think, not t’other way round.
    Looking @ ‘warmer’ sites, some of them are clearly ‘bought-and-paid-for’ but others really believe the warmista-stuff. Debate will not do it. The bedmap2 data is interesting, especially since the ‘warmers’ say, “yes, the ice in Antarctica is increasing in extent, but is thinner, so still a net loss.” We should push for another such survey in a year. We need hard, incontrovertible data. Annddd… can we put our heads together and come up with simple, obvious experimental ways to show it’s nonsense?
    At the end of ICCC9, I suggested to Joe Bast that we invite the Gores/Manns etc. to submit their stuff to a judge and we’d do the same. Ifn they decline, we just use their published works. The judge would highlight conflicting evidences [HADCRUT-goalpost-moving?] and would render 1 of 4 opinions The evidence is strongly in favor of/against cAGW or the evidence leans to or away from same. The judge would be a plain-vanilla or a self-aware, supercomputer.
    AR5 final draft p.7: “Due to a low level of scientific understanding there is low confidence in attributing the causes of the observed loss of mass from the Antarctic ice sheet since 1993.” Translation: We have no clue what’s going on down there. But certainty begins at Punta del Fuego, right? Riiight.
    The Guardian article is a great gift: We might (gently) ask them if this sentiment also applies to ‘Global Warming’ and if not, why not.

  18. rtj1211 September 28, 2014 at 7:06 am
    That’s probably print runs
    =====================
    It is. However, that’s what matters, as they have no paywall online (yet), hence the GMG’s revenues are spiralling downwards. Whilst Rusbridger’s salary spirals ever upwards.

  19. Opinionated Bloggers may be useful as indeed they are in their sphere but they are no substitute for your peers. Those that compete with you to discover first, for the same funding and academic posts. They know your strengths and weaknesses and from the semi anonymity of the peer review process will happily shoot you down should you fail in rigour. Ain’t no old boys network of back-slappers or idea buggins’ turn in my area of Science. Thankfully

    • Open review is review by everyone or anyone. Peer review is review by a small subset. There is a possibility of bias in creating the subset. Which is more likely to uncover mistakes? The challenge is to create a proper mode of organization for open review.

  20. Just to reassure you all
    In an article the other day stating that the greens should give up on black Carbon and focus on CO2 as the primary driver of global warming, I posted (as Strativarius) that RSS is on 18 years now and easily passes the ‘Santer’ test.
    5 minutes later it was removed
    I was obviously off-message

  21. It’s funny that the Guardian has been mentioned, and yes, it’s about as sad and prehistoric as it gets> it’s as though they’ve gone back thirty years to when the scam started and it’s readers have ignored all the latest facts like no warming since 1998. It’s difficult if you try telling any of their readers about the facts
    I had a three day online argument with a warmist after entering a comment on a Guardian climate article about Alaska becoming the new Florida! He sent me links to sites with all his ‘evidence’ and didn’t see a problem when I pointed out that all his websites were run by environmental activists. He thought it was stupid when I mentioned that science was meant to be neutral and stick to the facts. He then tried to get the upper hand by suggesting that his evidence was imperial as he had quoted the Royal Society in his argument. Again, he didn’t like it when I pointed out that the former chairman Lord Rees was an environmental activist. And again, he didn’t see a problem with this.
    As you’ll all know, it’s very difficult to debate with a warmist becuase they see nothing wrong with a one sided and purely environmental activist agenda.

Comments are closed.