First, correcting an error that originated with the blog The Hockey Schtick about not giving appropriate credit. Marcel Crok writes on De staat van het klimaat
One of the basic principles of blogging is to give credit when credit is due. I always try to mention the source of my information and most blogs do this using an acronym like h/t (hat tip). This week one of my own blog articles was not referred to as the source for another blog article on The Hockey Schtick. This blog then alerted WUWT? who brought the same news quite loudly and then also Steve McIntyre picked it up and Bishop Hill.
Marcel, I always try my best to give credit where it is due, and had I known, you would most certainly have been cited. I agree, knowing of your original blog post would have saved much trouble and speculation. I’ve also left a comment with The Hockey Schtick, asking him to credit you. By way of compensation, I’ve added De staat van het klimaat to the WUWT blog roll. That said, the post did generate quite a bit of discussion, always a good thing. Marcel writes of a guest post by Koutsoyiannis:
A blog post earlier this week about an EGU presentation of Eva Steirou (a researcher in the group of Demetris Koutsoyiannis) on temperature data homogenisation created some stir in the blogosphere after Watts Up With That? and Climate Audit paid attention to it. Koutsoyiannis has now written a guest blog to give some first reactions.
Demetris Koutsoyiannis writes:
I believe that science blogs have offered a very powerful means in scientific dialogue, which is a prerequisite of scientific progress. I have very positive personal experiences. In 2008, a poster paper in EGU, “Assessment of the reliability of climate predictions based on comparisons with historical time series”, was widely discussed at blogs and this was very useful to improve it and produce a peer-reviewed paper, “On the credibility of climate predictions” , which again was widely discussed at blogs. In the follow up paper, “A comparison of local and aggregated climate model outputs with observed data” we incorporated replies to the critiques we have seen in lots of blog comments.
In comparison, the formal peer reviewed system, while in principle encourages post-publication discussion through formal Commentaries and Replies, was able to offer us a single Commentary for the second paper (none for the former), which also gave us the opportunity to clarify our methodology (and feel safer about it) in our reply, “Scientific dialogue on climate: is it giving black eyes or opening closed eyes? Reply to “A black eye for the Hydrological Sciences Journal” by D. Huard”.
Writing about the complaints that this was a presentation, and not a paper yet, he says:
But we plan to produce a peer-reviewed paper (unless we have made a fatal error, which we hope not) and we keep studying the topic more thoroughly. That is why we think that we are lucky to have received all these comments from the blogs. I did not have the time to read them all, let alone to assimilate them, so I will not provide replies here. From first glance I find most of them very useful, whether they are positive or negative.
…
But of course these are scientific disagreements and it is fine if scientists disagree. Some arguments, though, fall into other categories, such as arguments from authority or ad hominem. Well, I am familiar with such arguments within scientific transactions, formal (paper reviews) or informal (in blogs), but they are always saddening and also make it necessary to refer to personal information in order to reply.
You can read the entire guest post here, it is fascinating reading.
Koutsoyiannis makes an interesting point about blogs -vs- traditional peer review. Traditional peer review is a slow and arduous process, taking months, sometimes years, and in my opinion is a holdover from a much slower time pre-Internet and email. Usually less than a dozen people are involved in that process. Blog review of papers, presentations, and topics is like an insta-launch, where citizens and scientists alike spar in sometimes a gladiatorial style over broad issues as well as minutiae. Hundreds and often thousands of eyes and minds are brought to bear, often picking the carcass clean of errors until nothing is left.
For all its warts, science blogging has a purpose and a place in today’s world. I happen to think that the occasional errors like this one, where by accident, credit wasn’t given, actually work to improve things in the long run, because as we all know:
Source: XKCD – Duty Calls
Anthony – Good for you making things right.
Your website ends up being crowd sourcing review. In many ways, it is superior to the legacy peer review system. The IPPC should treat blog posts on WUWT as having been reviewed and worthy of inclusion in their reports.
Crowd-sourcing – whether that be in data collection such as the SurfaceStations project, data analysis such as in the many cool Zooniverse (https://www.zooniverse.org/) projects, or blog critique of research – is proving to be immensely valuable in the advancement of scientific understanding because it’s democratic. That is, it opens up participation to the talents of more than the few individuals who by chance happen to be at the right place. IOW, “On the internet nobody knows you’re a dog.” Furthermore it succeeds because of merit rather than cronyism and the rewards for most participants are intellectual rather than monetary, which dampens the tendency toward corruption.
Anthony – Good for you for making things right in terms of attribution.
Crowd review of blog posts seems to be more rigorous than the legacy peer review system. IPPC should include WUWT blog posts in their reports since they are more heavily reviewed than most of the papers included in their reports.
What happened to simply sending an email to signal the omission?
Hi, I fully agree that it is important to the justification of science bloggin that credit is always given to the source, as a result please p[eople always include them. I know that this is difficult, tired, late at night after a full day at the office maybe, BUT it will pay dividends. It removes yet another area of critique for the “establishment” to hang on.
The true riesig Fade (extra points for identifying the reference!) is that so many are wrong on the Internet!
“IPPC should include WUWT blog posts in their reports”
Thanks for the giggle! The difference is that drafts of submitted papers get revised and improved as a function of peer review. Here, things just get repeated over and over. Not quite the same thing.
Matt Briggs has a series on data homogenization starting here:
http://wmbriggs.com/blog/?p=1459
Anthony, thanks for the correction. The presentation of Steirou was freely available on the web, so in some way The Hockey Schtick was free to ignore my post. But then the wording changed (paper instead of presentation) and that generated a lot of noise. Anyway, it’s good that we now discuss temperature homogenisation again and I am looking forward to more constructive discussions in the future.
Marcel
Kelly, the IPPC ignores blog posts on WUWT as precisely becausse they are better reviewed and more worthy of inclusion in their reports than the hackery they need to support their agenda.
Brian H says:
July 20, 2012 at 8:26 am
The true riesig Fade (extra points for identifying the reference!) is that so many are wrong on the Internet!
http://www.threepennyopera.org/
“Die Welt ist arm, der Mensch ist schlecht. Da hat er eben leider recht.”
Good move to correct the record on sources. Due credit when known is extremely valuable and everyone should make an effort to trace ideas back to their source.
I think we are seeing a turning point in the scientific review process. The old formal review process has demonstrated it is (and has been) broken for some time. In theory it is a good concept, but in implementation, human frailties cause it to creep toward a corrupt process unless a strong hand is at the editorial wheel that enforces good practice and no back room manipulation on the review process.
As crowd source blog review becomes more mainstream, I see us moving toward a combined process, where an initial presentation of the idea is posted, and quickly screened for errors by literally hundreds or thousands of reviewers. Each poking at their area of interest or expertise. Sometimes bringing to bear unique experience or exposure to obscure knowledge that often leads the original poster to sources and problems that would have gone unnoticed in a formal review by a small group of specialists. Then after that quick reasonableness check the formal paper is refined and presented for formal review by a broad range of specialties, not just the primary specialty of the author.
We see it often here where climate is examined from the point of view of process control engineers, electrical engineers (feedback, capacitive and inductive circuits), physics, chemistry and geology specialists just to name a few specialties that come to mind from recent discussions. In doing so they bring fresh ways of looking at the problem and sometimes expose what appears to be a poorly understood process that in another specialty (such as process control — ie feed backs ) is a very well studied and understood field.
Suddenly the original poster is opened up to areas of knowledge that he/she simply did not know existed. The depth of human knowledge today is so vast that no one person can possibly know even a small fraction of it.
This is evidenced by “climate scientists” who dabble in computer programing and statistics and think themselves competent in those areas when they are in fact blundering amateurs, making sophomoric level mistakes.
Unfortunately the formal review process by experts in their narrowly defined field only reinforces those faulty perceptions of confidence because the reviewers often do not themselves have journeymen level understanding of statistics or computer coding good practices, and proper code validation and testing methods.
In time I think we will see that the “instant review” of the blogs and crowd sourcing backed up by formal review of multiple specialists such as statistics, higher mathematics, physics etc. will help to break down the current faults of the peer (pal) review system as it exists today.
Please someone publish a paper on the frailties of computer models, and the inherent risk of undocumented bugs in all computer code, even well tested code. Model output is not data it is a method to make quick reasonableness checks on a theory or evaluate its sensitivity to initial conditions “if the model is correct”. For example aerodynamic analysis models do a good job of allowing aerodynamic designs to be developed on the computer but they are not data, that is only gathered in wind tunnel testing and flight testing to prove the model expectations were close to reality.
Larry
We have been witnessing the process of “creative destruction” in the pathways and byways of society as the digital age advances.
Remember this?:
“ By 1925, the speed of the record was becoming standardized at a nominal value of 78 revolutions per minute (rpm). . . . 78.26 rpm, being the speed of a 3600 rpm synchronous motor reduced by 46:1 gearing. ”
http://en.wikipedia.org/wiki/78_rpm#78_rpm_disc_developments
As the comments in the post mention, the write, review, publish, and wait (post-publish commentary) method of producing science literature has just about been destroyed. There are way to many print publications and articles of questionable value. Many are familiar with this, especially those in an academic setting that have been asked by your chair or dean about the number of your publications due out or accepted for publication before your next performance review.
This current (minor) kerfuffle was easily defused by Anthony – as others have said above. Demetris K. indicates the value of the digital (web) posting and review. The old way will continue to fade. The new way is not yet clear. Still, we move on —
Thanks to sites such as WUWT, the hosts, moderators, and commenters.
~~~~~~~~~~~~~~
Note: I just took the hint and visited Marcel Crok’s site. See Anthony’s second line in this post for the link.
~Gary says:
July 20, 2012 at 8:30 am
“IPPC should include WUWT blog posts in their reports”
Thanks for the giggle! The difference is that drafts of submitted papers get revised and improved as a function of peer review. Here, things just get repeated over and over. Not quite the same thing.
—–
Example(s), please? Did you forget the Geurgis (sp) paper, among likely many others?
Interesting, because it illustrates how unsure working scientists are of the theory/philosophy/epistemology (choose your term) of science.
Which is a recurrent theme of mine. Just because you are a working scientist, doesn’t mean you understand how science works.
BTW, great cartoon.
Long term persistence;
Take the hint and visit Crok.
Koutso-Venema.
=========
I enjoy reading the papers, research and comments on WUWT. I’m an accountant, not a scientist per se, but I’ve had a lot of experience in a lot of different things to get an idea when something is not quite “right”. Many times the comments on WUWT help confirm my suspicions and fine tune my BS meter. And I am one of those who have come to have no faith in the current peer review process when it comes to “climate science” research. So thank you Anthony and all you bloggers and commenters for your work, and all the entertainment you provide.
Larry Ledwick (hotrod) says:”
I think we are seeing a turning point in the scientific review process. The old formal review process has demonstrated it is (and has been) broken for some time. In theory it is a good concept, but in implementation, human frailties cause it to creep toward a corrupt process unless a strong hand is at the editorial wheel that enforces good practice and no back room manipulation on the review process.
As crowd source blog review becomes more mainstream, I see us moving toward a combined process, where an initial presentation of the idea is posted, and quickly screened for errors by literally hundreds or thousands of reviewers. ”
The EGU journal Atmospheric Chemistry and Physics has an open review. Open in the sense that the comments of the reviewers are posted on the web and in the sense that everyone is invited to comment. In practice the number of reviewers is a little less than “literally hundreds or thousands of reviewers”, which is why the journal also always requests two official reviews. It would be great if one could get some sort of continual review process going.
Anthony, credit were credit is due: the cartoon is funny!
I find D. Koutsoyiannis’ work and comments always cogent, interesting and very well thought through. Much like Watts’ essays and comments, except the very well thought through part is sometimes short due to the immediacy of daily blogging. (that is okay too part of the process) The way science blogging and mass review has developed over the years is both democratic and refreshing. The more formal science review process has its problems but it has its strengths too. It is my belief that we need both approaches and that when blended, in the long run, we all will benefit. Like most things in the world this blending is a work in progress and find the observing and participating in it highly stimulating.
Nothing like a legion of citizen scientists….to keep the ‘pros’ on their toes….
thanks for the Romper Room of Truth….
where we work to correct reality, instantaneously….
“There is scarcely any well informed person ,who, if he has the will, has not also the power to add something essential to the general stock of knowledge, if he will only observe regularly and methodically some particular class of facts which may most excite his attention, or which his situation may best enable him to study with effect.” Herschell – quoted in The Zoologist 1860
See also “Withering Academia” http://www.iew.uzh.ch/wp/iewwp512.pdf
@Larry Cullen
Is this now a difference ‘twixt open source and propriority science?
@Larry Cullen
Is this then just the difference ‘twixt open source and propriorty science?
-apologies if a double post – sent via BB
“Koutsoyiannis makes an interesting point about blogs -vs- traditional peer review. Traditional peer review is a slow and arduous process, taking months, sometimes years, and in my opinion is a holdover from a much slower time pre-Internet and email.”
True.
Inefficiency is orders of magnitude beyond what we can afford to tolerate.
–
“Blog review of papers, presentations, and topics is like an insta-launch, where citizens and scientists alike spar in sometimes a gladiatorial style over broad issues as well as minutiae. Hundreds and often thousands of eyes and minds are brought to bear, often picking the carcass clean of errors until nothing is left.
For all its warts, science blogging has a purpose and a place in today’s world.”
It’s not working. There’s a fatal shortage of sensible, capable parties participating.
One possibility is that most sensible parties steer clear of volunteering blog comments (although they may be reading). At present the “blog review” system is operating like a corrupted justice system — corrupted by coupled pools of ignorance &/or deception.
One possible way around this is more competition — for example (no offense intended – only advancement of collective human lucid awareness intended) WUWT needs at least a handful of serious competitors with at least equal readership. I would suggest that some of these widely-read competing blogs should have different core foci & rules.
I will now suggest what I would personally find most efficient (efficiency is a high personal value due to severe constraints on time & resources):
1. Encourage conciseness (Pareto Principle).
2. Core Purpose:
Focus exclusively on exploring natural variability (using observations).
3. Bad Apples:
Strictly prohibit protracted, presumptively-“authoritative” swipe-quote harassment (Peter Principle, Law of Authoritative Ignorance &/or Deception). Judith Curry has the idea with one of her blog rules, which suggests saying what YOU have to say (as opposed to aiming solely to protractedly harass others ad infinitum). Ignorant &/or deceptive “critics” can be allowed to concisely say what they have to say ONCE. After that they are unacceptably wasting everyone’s time if they continue with dark negativity & criticism rather than switching to offering positive, truly constructive, possibly even inspiring, observation-based alternatives. The guilty parties here are bankrupt of exploratory vision and should be cut off from further commentary after 3 back-&-forths (to keep it simple) if they’re incapable of independently exercising appropriate restraint. Just a few bad apples ruin the tone of the whole blog. I have 2 WUWT regulars in mind with this suggestion for new, competing blogs (but I refrain from naming them).
4. Ban mention of chaos & CO2.
Any author who immediately acknowledges that there is a possibility of a fatal flaw in his work always gets my attention. There actually may be some science going on.