More on Koutsoyiannis and the homogenization of temperature data – plus some comments on blog review

First, correcting an error that originated with the blog The Hockey Schtick about not giving appropriate credit. Marcel Crok writes on De staat van het klimaat

One of the basic principles of blogging is to give credit when credit is due. I always try to mention the source of my information and most blogs do this using an acronym like h/t (hat tip). This week one of my own blog articles was not referred to as the source for another blog article on The Hockey Schtick. This blog then alerted WUWT? who brought the same news quite loudly and then also Steve McIntyre picked it up and Bishop Hill.

Marcel, I always try my best to give credit where it is due, and had I known, you would most certainly have been cited. I agree, knowing of your original blog post would have saved much trouble and speculation. I’ve also left a comment with The Hockey Schtick, asking him to credit you. By way of compensation, I’ve added De staat van het klimaat to the WUWT blog roll. That said, the post did generate quite a bit of discussion, always a good thing. Marcel writes of a guest post by Koutsoyiannis: 

A blog post earlier this week about an EGU presentation of Eva Steirou (a researcher in the group of Demetris Koutsoyiannis) on temperature data homogenisation created some stir in the blogosphere after Watts Up With That? and Climate Audit paid attention to it. Koutsoyiannis has now written a guest blog to give some first reactions.

Demetris Koutsoyiannis writes:

I believe that science blogs have offered a very powerful means in scientific dialogue, which is a prerequisite of scientific progress. I have very positive personal experiences. In 2008, a poster paper in EGU, “Assessment of the reliability of climate predictions based on comparisons with historical time series”,  was widely discussed at blogs and this was very useful to improve it and produce a peer-reviewed paper, “On the credibility of climate predictions” , which again was widely discussed at blogs. In the follow up paper, “A comparison of local and aggregated climate model outputs with observed data” we incorporated replies to the critiques we have seen in lots of blog comments.

In comparison, the formal peer reviewed system, while in principle encourages post-publication discussion through formal Commentaries and Replies, was able to offer us a single Commentary for the second paper (none for the former), which also gave us the opportunity to clarify our methodology (and feel safer about it) in our reply, “Scientific dialogue on climate: is it giving black eyes or opening closed eyes? Reply to “A black eye for the Hydrological Sciences Journal” by D. Huard”.

Writing about the complaints that this was a presentation, and not a paper yet, he says:

But we plan to produce a peer-reviewed paper (unless we have made a fatal error, which we hope not) and we keep studying the topic more thoroughly. That is why we think that we are lucky to have received all these comments from the blogs. I did not have the time to read them all, let alone to assimilate them, so I will not provide replies here. From first glance I find most of them very useful, whether they are positive or negative.

But of course these are scientific disagreements and it is fine if scientists disagree. Some arguments, though, fall into other categories, such as arguments from authority or ad hominem. Well, I am familiar with such arguments within scientific transactions, formal (paper reviews) or informal (in blogs), but they are always saddening and also make it necessary to refer to personal information in order to reply.

You can read the entire guest post here, it is fascinating reading.

Koutsoyiannis makes an interesting point about blogs -vs- traditional peer review. Traditional peer review is a slow and arduous process, taking months, sometimes years, and in my opinion is a holdover from a much slower time pre-Internet and email. Usually less than a dozen people are involved in that process. Blog review of papers, presentations, and topics is like an insta-launch, where citizens and scientists alike spar in sometimes a gladiatorial style over broad issues as well as minutiae. Hundreds and often thousands of eyes and minds are brought to bear, often picking the carcass clean of errors until nothing is left.

For all its warts, science blogging has a purpose and a place in today’s world. I happen to think that the occasional errors like this one, where by accident, credit wasn’t given, actually work to improve things in the long run, because as we all know:

Source: XKCD – Duty Calls

35 thoughts on “More on Koutsoyiannis and the homogenization of temperature data – plus some comments on blog review

  1. Anthony – Good for you making things right.

    Your website ends up being crowd sourcing review. In many ways, it is superior to the legacy peer review system. The IPPC should treat blog posts on WUWT as having been reviewed and worthy of inclusion in their reports.

  2. Crowd-sourcing – whether that be in data collection such as the SurfaceStations project, data analysis such as in the many cool Zooniverse (https://www.zooniverse.org/) projects, or blog critique of research – is proving to be immensely valuable in the advancement of scientific understanding because it’s democratic. That is, it opens up participation to the talents of more than the few individuals who by chance happen to be at the right place. IOW, “On the internet nobody knows you’re a dog.” Furthermore it succeeds because of merit rather than cronyism and the rewards for most participants are intellectual rather than monetary, which dampens the tendency toward corruption.

  3. Anthony – Good for you for making things right in terms of attribution.

    Crowd review of blog posts seems to be more rigorous than the legacy peer review system. IPPC should include WUWT blog posts in their reports since they are more heavily reviewed than most of the papers included in their reports.

  4. Hi, I fully agree that it is important to the justification of science bloggin that credit is always given to the source, as a result please p[eople always include them. I know that this is difficult, tired, late at night after a full day at the office maybe, BUT it will pay dividends. It removes yet another area of critique for the “establishment” to hang on.

  5. The true riesig Fade (extra points for identifying the reference!) is that so many are wrong on the Internet!

  6. “IPPC should include WUWT blog posts in their reports”

    Thanks for the giggle! The difference is that drafts of submitted papers get revised and improved as a function of peer review. Here, things just get repeated over and over. Not quite the same thing.

  7. Anthony, thanks for the correction. The presentation of Steirou was freely available on the web, so in some way The Hockey Schtick was free to ignore my post. But then the wording changed (paper instead of presentation) and that generated a lot of noise. Anyway, it’s good that we now discuss temperature homogenisation again and I am looking forward to more constructive discussions in the future.
    Marcel

  8. Kelly, the IPPC ignores blog posts on WUWT as precisely becausse they are better reviewed and more worthy of inclusion in their reports than the hackery they need to support their agenda.

  9. Good move to correct the record on sources. Due credit when known is extremely valuable and everyone should make an effort to trace ideas back to their source.

    I think we are seeing a turning point in the scientific review process. The old formal review process has demonstrated it is (and has been) broken for some time. In theory it is a good concept, but in implementation, human frailties cause it to creep toward a corrupt process unless a strong hand is at the editorial wheel that enforces good practice and no back room manipulation on the review process.

    As crowd source blog review becomes more mainstream, I see us moving toward a combined process, where an initial presentation of the idea is posted, and quickly screened for errors by literally hundreds or thousands of reviewers. Each poking at their area of interest or expertise. Sometimes bringing to bear unique experience or exposure to obscure knowledge that often leads the original poster to sources and problems that would have gone unnoticed in a formal review by a small group of specialists. Then after that quick reasonableness check the formal paper is refined and presented for formal review by a broad range of specialties, not just the primary specialty of the author.

    We see it often here where climate is examined from the point of view of process control engineers, electrical engineers (feedback, capacitive and inductive circuits), physics, chemistry and geology specialists just to name a few specialties that come to mind from recent discussions. In doing so they bring fresh ways of looking at the problem and sometimes expose what appears to be a poorly understood process that in another specialty (such as process control — ie feed backs ) is a very well studied and understood field.

    Suddenly the original poster is opened up to areas of knowledge that he/she simply did not know existed. The depth of human knowledge today is so vast that no one person can possibly know even a small fraction of it.

    This is evidenced by “climate scientists” who dabble in computer programing and statistics and think themselves competent in those areas when they are in fact blundering amateurs, making sophomoric level mistakes.

    Unfortunately the formal review process by experts in their narrowly defined field only reinforces those faulty perceptions of confidence because the reviewers often do not themselves have journeymen level understanding of statistics or computer coding good practices, and proper code validation and testing methods.

    In time I think we will see that the “instant review” of the blogs and crowd sourcing backed up by formal review of multiple specialists such as statistics, higher mathematics, physics etc. will help to break down the current faults of the peer (pal) review system as it exists today.

    Please someone publish a paper on the frailties of computer models, and the inherent risk of undocumented bugs in all computer code, even well tested code. Model output is not data it is a method to make quick reasonableness checks on a theory or evaluate its sensitivity to initial conditions “if the model is correct”. For example aerodynamic analysis models do a good job of allowing aerodynamic designs to be developed on the computer but they are not data, that is only gathered in wind tunnel testing and flight testing to prove the model expectations were close to reality.

    Larry

  10. We have been witnessing the process of “creative destruction” in the pathways and byways of society as the digital age advances.

    Remember this?:
    By 1925, the speed of the record was becoming standardized at a nominal value of 78 revolutions per minute (rpm). . . . 78.26 rpm, being the speed of a 3600 rpm synchronous motor reduced by 46:1 gearing.

    http://en.wikipedia.org/wiki/78_rpm#78_rpm_disc_developments

    As the comments in the post mention, the write, review, publish, and wait (post-publish commentary) method of producing science literature has just about been destroyed. There are way to many print publications and articles of questionable value. Many are familiar with this, especially those in an academic setting that have been asked by your chair or dean about the number of your publications due out or accepted for publication before your next performance review.

    This current (minor) kerfuffle was easily defused by Anthony – as others have said above. Demetris K. indicates the value of the digital (web) posting and review. The old way will continue to fade. The new way is not yet clear. Still, we move on —

    Thanks to sites such as WUWT, the hosts, moderators, and commenters.
    ~~~~~~~~~~~~~~
    Note: I just took the hint and visited Marcel Crok’s site. See Anthony’s second line in this post for the link.

  11. ~Gary says:
    July 20, 2012 at 8:30 am
    “IPPC should include WUWT blog posts in their reports”
    Thanks for the giggle! The difference is that drafts of submitted papers get revised and improved as a function of peer review. Here, things just get repeated over and over. Not quite the same thing.

    —–

    Example(s), please? Did you forget the Geurgis (sp) paper, among likely many others?

  12. Interesting, because it illustrates how unsure working scientists are of the theory/philosophy/epistemology (choose your term) of science.

    Which is a recurrent theme of mine. Just because you are a working scientist, doesn’t mean you understand how science works.

    BTW, great cartoon.

  13. I enjoy reading the papers, research and comments on WUWT. I’m an accountant, not a scientist per se, but I’ve had a lot of experience in a lot of different things to get an idea when something is not quite “right”. Many times the comments on WUWT help confirm my suspicions and fine tune my BS meter. And I am one of those who have come to have no faith in the current peer review process when it comes to “climate science” research. So thank you Anthony and all you bloggers and commenters for your work, and all the entertainment you provide.

  14. Larry Ledwick (hotrod) says:”
    I think we are seeing a turning point in the scientific review process. The old formal review process has demonstrated it is (and has been) broken for some time. In theory it is a good concept, but in implementation, human frailties cause it to creep toward a corrupt process unless a strong hand is at the editorial wheel that enforces good practice and no back room manipulation on the review process.
    As crowd source blog review becomes more mainstream, I see us moving toward a combined process, where an initial presentation of the idea is posted, and quickly screened for errors by literally hundreds or thousands of reviewers. ”

    The EGU journal Atmospheric Chemistry and Physics has an open review. Open in the sense that the comments of the reviewers are posted on the web and in the sense that everyone is invited to comment. In practice the number of reviewers is a little less than “literally hundreds or thousands of reviewers”, which is why the journal also always requests two official reviews. It would be great if one could get some sort of continual review process going.

    Anthony, credit were credit is due: the cartoon is funny!

  15. I find D. Koutsoyiannis’ work and comments always cogent, interesting and very well thought through. Much like Watts’ essays and comments, except the very well thought through part is sometimes short due to the immediacy of daily blogging. (that is okay too part of the process) The way science blogging and mass review has developed over the years is both democratic and refreshing. The more formal science review process has its problems but it has its strengths too. It is my belief that we need both approaches and that when blended, in the long run, we all will benefit. Like most things in the world this blending is a work in progress and find the observing and participating in it highly stimulating.

  16. Nothing like a legion of citizen scientists….to keep the ‘pros’ on their toes….

    thanks for the Romper Room of Truth….

    where we work to correct reality, instantaneously….

  17. “There is scarcely any well informed person ,who, if he has the will, has not also the power to add something essential to the general stock of knowledge, if he will only observe regularly and methodically some particular class of facts which may most excite his attention, or which his situation may best enable him to study with effect.” Herschell – quoted in The Zoologist 1860

    See also “Withering Academia” http://www.iew.uzh.ch/wp/iewwp512.pdf

  18. @Larry
    Is this then just the difference ‘twixt open source and propriorty science?

    -apologies if a double post – sent via BB

  19. “Koutsoyiannis makes an interesting point about blogs -vs- traditional peer review. Traditional peer review is a slow and arduous process, taking months, sometimes years, and in my opinion is a holdover from a much slower time pre-Internet and email.”

    True.
    Inefficiency is orders of magnitude beyond what we can afford to tolerate.


    “Blog review of papers, presentations, and topics is like an insta-launch, where citizens and scientists alike spar in sometimes a gladiatorial style over broad issues as well as minutiae. Hundreds and often thousands of eyes and minds are brought to bear, often picking the carcass clean of errors until nothing is left.

    For all its warts, science blogging has a purpose and a place in today’s world.”

    It’s not working. There’s a fatal shortage of sensible, capable parties participating.

    One possibility is that most sensible parties steer clear of volunteering blog comments (although they may be reading). At present the “blog review” system is operating like a corrupted justice system — corrupted by coupled pools of ignorance &/or deception.

    One possible way around this is more competition — for example (no offense intended – only advancement of collective human lucid awareness intended) WUWT needs at least a handful of serious competitors with at least equal readership. I would suggest that some of these widely-read competing blogs should have different core foci & rules.

    I will now suggest what I would personally find most efficient (efficiency is a high personal value due to severe constraints on time & resources):

    1. Encourage conciseness (Pareto Principle).

    2. Core Purpose:
    Focus exclusively on exploring natural variability (using observations).

    3. Bad Apples:
    Strictly prohibit protracted, presumptively-“authoritative” swipe-quote harassment (Peter Principle, Law of Authoritative Ignorance &/or Deception). Judith Curry has the idea with one of her blog rules, which suggests saying what YOU have to say (as opposed to aiming solely to protractedly harass others ad infinitum). Ignorant &/or deceptive “critics” can be allowed to concisely say what they have to say ONCE. After that they are unacceptably wasting everyone’s time if they continue with dark negativity & criticism rather than switching to offering positive, truly constructive, possibly even inspiring, observation-based alternatives. The guilty parties here are bankrupt of exploratory vision and should be cut off from further commentary after 3 back-&-forths (to keep it simple) if they’re incapable of independently exercising appropriate restraint. Just a few bad apples ruin the tone of the whole blog. I have 2 WUWT regulars in mind with this suggestion for new, competing blogs (but I refrain from naming them).

    4. Ban mention of chaos & CO2.

  20. Any author who immediately acknowledges that there is a possibility of a fatal flaw in his work always gets my attention. There actually may be some science going on.

  21. Thanks for pointing out this blog post by Marcel Crok. However, his blog post was not the “source” of the posting on The Hockey Schtick. The “source” was a tipster who only provided this link:

    http://itia.ntua.gr/en/docinfo/1212/

    from which I wrote my own interpretation, extracted and transcribed slides and the abstract, and created my own post.

    I don’t know where the tipster found this link and therefore don’t consider this an “error” or lack of courtesy, but nonetheless give Marcel credit for having found this link to a very interesting presentation and for the additional information provided by his 2 posts.

    I am also not the source of the confusion about this paper being “peer-reviewed” as I never claimed that it was in my post. Having “presented papers” at over two dozen international scientific conferences myself, I am well aware that the purpose of meetings is to present and discuss new research findings often prior to even writing a paper and submitting it to formal peer review. Nevertheless, the scientific community still calls this “presenting a paper” even though the written version often hasn’t yet been prepared or submitted for formal peer-review.

    I, of course, agree that credit for finding interesting links should be acknowledged when known.

  22. fadingfool says:
    July 20, 2012 at 11:04 am

    @Larry
    Is this now a difference ‘twixt open source and propriority science?

    Yes if I understand your question, the open source software model is a useful point of comparison, vs the proprietary software developers where the software is only released for “beta testing” after it is fairly far along in its development cycle and often there are imbedded in the code assumptions about how to handle issues and code certain modules which lock in the developers per-conceptions of what the public want in the product.

    Often we find that the developers in that model add bells and whistles that the majority of users have no interest in and will never use, and make assumptions about how the user will interact with the software that may not mirror reality.

    The AGW community is doing the same thing. Imbedded in their models at very core levels are assumptions about what the climate is doing, and when the model output does not match reality they fiddle with the knobs to try to get the match they want but never consider the idea that perhaps some of their core assumptions are bogus.

    It is like the pride of authorship of a writer that makes him blind to faults in his novel, or equally bad the blindness of an editor to a different style of writing than what he likes.

    Larry

  23. This is an excellent thread, once again the best of WUWT at work. I agree with pretty much everything said so far. I too think blogs are really important for the future of science.

    BUT

    There is a big caveat. I have seen at least two instances here where I felt the peer-reviewing process failed signally. I remember two because they left a bad taste in my mouth which has still not completely gone, probably because the core issues have not yet been sufficiently addressed or sufficiently compensated elsewhere.

    The first failure was the review of the Sky Dragons book. What caused the failure was the disproportionate focus on the contribution of one individual who is known here as holding a rather strange singularity of interest with the sun. Not only was this contribution removed; the problem was compounded by material that came to light about this individual. In the whole process, valuable material in the rest of the book was simply drowned. In the sense of book review, Amazon manages far better IMHO.

    The second failure was the handling of Ned Nikolov and Karl Zeller. I know there were faults on all sides. But the upshot is almost certainly that WUWT lost a lot of competent readers at that point. Still, I always try to find the positive in everything, and the upshot for me was, in the end, good, no, more than that, astounding…

    …I went back to the material to check, check and check again. I obsessed with the science of Nikolov and Zeller for a long time, finally got my head round their maths, found it really worked as they said, found their assessment was correct, that the IPCC S-B equations are wrong… so next I started a wiki for Climate Science where I might get their material in a form that not only read far more easily but also was open to updating and correction… but then ran into the problem of their apparent contravention of the Second Law of Thermodynamics… so I dived into that… read up the lives of Kelvin and Maxwell, travelled up to Scotland to come as close to their spirit and legacy as I could… then I dived into the utterly amazing work of Graeff who has verified experimentally what Loschmidt said 150 years ago, challenging Clerk Maxwell’s interpretation of the Second Law… and upholding Nikolov and Zeller… I travelled to Germany to meet Graeff because I wanted to get to the bottom of his experiments and claims… omg it all holds up, this is Nobel-quality material but the universities won’t take on replication because… because… because Nobody Challenges The Second Law of Thermocynamics (except perp.mob.nutcases)… well actually Prof Sheehan has been running conferences on 2LoT and he’s interested in Graeff… so now the wiki is on hold (asleep not dead) while I “gird up my loins” to do this replication myself (with anyone else interested) that the universities should be doing but are not. Well, not so far… heck who am I to be making these claims, retired, no science degree… but it’s needed and it promises to correct something so fundamental that Climate Science has been wrong-footed right from the start…

    … ye gods. Anthony and Willis, what have you launched me into???

    :)

  24. I fully agree, the traditional peer-review model is obsolete and should be augmented by open access blog review.

    Many of the most important discoveries in science are made when there is cross-fertilisation of ideas from different disciplines. But that becomes harder as knowledge grows exponentially and subjects of study become subdivided, specialized and compartmentalized.

    Climate scientists go to conferences to meet other climate scientists. People from related disciplines will attend, of course, but experts in unrelated subjects will not.

    If scientists want make use of the Internet but still restrict review of their work to other scientists then surely they could do this on forums where the membership is restricted and everyone’s qualifications are known. However, I think they would be wrong to (only) do this.

    Everybody has some expert knowledge in something. Sometimes an insignificant piece of information from one area of expertise can be highly significant in another. Science blogs are important not only because they provide a forum for scientists but because they enable anyone and everyone to contribute. There is no way of knowing where the next insight will come from and innocent questions are often the most insightful.

    I’ll give two examples from my own experience. They are only to illustrate a point. I don’t think either is particularly significant or likely to lead to someone’s eureka moment but I have no way of knowing.

    1. Some have suggested mitigating global warming by modifying the albedo of urban environments e.g. using white (or lighter coloured) roofing materials. Many mocked this idea but I think it has merit. It reduces urban heating and the building’s requirement for air conditioning and (in some situations) heating.

    One little-known factor that effects the whole sustainability/cost-benefit analysis is that lighter coloured roofing materials also often last longer. Because of thermal stress, the manufacturers of polymer coatings for profiled metal roofing sheets typically give a 30 year guarantee for the lighter colours in their range but only 20 for the darker ones.

    http://www.cagroup.ltd.uk/docs/Technical%20Notes/TN-46%20Rev%200%20Colorcoat%20HPS200%20or%20Colorcoat%20Plastisol.pdf

    2. The urban heat island effect is well known. Some warmists choose to dismiss it. Agricultural land use also changes the albedo of the earth but not always by increasing it, as some scientists seem to suggest. I knew this long before I had heard the terms UHI or climate change as I used to hang glide. Any hang glider (pilot) will tell you that a ploughed field is a good place to catch a thermal. If you want to see a small heat island in action just look at where the birds are circling.

    http://tinyurl.com/cxn245j

    P.S. I’m a big xkcd fan too. If you haven’t seen this one, it’ll crack you up

    http://xkcd.com/556/

  25. Reply to Faux Science Slayer
    Could you tell us some of the faux science you have actually slain, O Mighty One!

    Though the mouse that roars
    Attracts the cat
    The lion ignores
    The silly prat

    So I hunt my prey
    Around the house
    And to all I say —
    I slay the mouse

  26. David Ross says:
    July 20, 2012 at 2:01 pm

    I agree with all of what you say.

    I also argue for increased albedo roofing. There are good reasons why in many hot sunny locations, houses are painted white.

    I live in a hot sunny location (Perth) and pretty much all new roofs are of reflective metal that look white from above, and which keep the house cooler in summer.

    Although the downside is that it makes houses colder in winter. It occurs to me that someone who invents a roofing material that changes from low to high albedo at around 25C or by the flick of a switch, will get very rich, and save more energy than all the world’s renewable energy projects put together.

  27. Anthony, thank you very much for this and the previous post. I was really glad to see some misunderstandings being resolved here, as well as the comments by Marcel Crock, Hockey Schtick and Victor Venema, who all contributed with their own blogs in this discussion. I thank all three–and of course all commenters (and will acknowledge them if the paper gets published).

    Victor, almost all EGU journals (not only Atmospheric Chemistry and Physics) have an open and public review process (see a list in http://www.egu.eu/publications/open-access-journals.html and follow the link “Papers in Open Discussion” in each one). In my view this is a great progress and I hope other professional societies, like AGU, will follow the example of EGU. But as you correctly point out, in practice the number of reviewers/commenters is small–as yet.

    The blogs attract comments more easily, because of the more informal style. I had some experiences where papers were in open review in an EGU journal and at the same time discussed at blogs. While the blog discussers were aware that they could post a comment at the official journal site, they did not. This I understand very well and I would say I like it, because it is more democratic and dismantles the idea that a journal is a unique space where all experts and authorities are pent.

    In conclusion, I believe that a synergistic interaction of blogs and journals provides the best service for the scientific progress.

  28. Demetris Koutsoyiannis says:
    July 21, 2012 at 3:55 am
    “Anthony, thank you very much for this and the previous post. I was really glad to see some misunderstandings being resolved here, as well as the comments by Marcel Crock, Hockey Schtick and Victor Venema, who all contributed with their own blogs in this discussion.”

    I saw on Victor’s blog some rant about the WUWT posts and skeptics overall, also justifying homogenisation with the need to correct for UHI.
    This is an alibi for the homogenisation as it is done.
    We all know the UHI effect to be real and introducing a significant difference in the temperature between the urban and natural environment, on the other side the adjustments for UHI are so minuscule or non-existent that it is a misdirection to talk about it.

  29. Bill Tuttle;
    Full marks! I MUST obtain the Decca CD re-issue of the vinyl recording I had, now unplayable. I hunger for those tunes and lyrics …

Comments are closed.