A Taxonomy of Science Blogs

Guest Post by Willis Eschenbach

Over at Lucia Liljegren’s most interesting site, “Rank Exploits”, she has another fascinating post, as is often the case. I busted out laughing at the post title, which is “HotWhopper Sou Doesn’t Read WUWT”. Given how often the lady in the Batcave over at Hotwhopper writes about Watts Up With That (WUWT) in general, and how much time she wastes trying to bite my ankles, I found this ludicrous. Lucia’s post was in reference to a curious network analysis by Paige Jarreau, which purports to show that Watts Up With That is hardly read by anyone in the field, and that the most-read climate blog is RealClimate. This is Ms. Jarreau’s description of her work:

What would it look like if you asked 600+ science bloggers to list up to three science blogs, other than their own, that they read on a regular basis, and then visually mapped the resulting data?

And here are the Jarreau results:

Figure 1. The section of the Jarreau results covering climate blogs. Lines show links, people who blog at one site and say that they read another site. Size of the names indicates the numbers of people listing that blog among the three that they read most. Click to enlarge.

Now, you can see WUWT in yellow over on the left, almost totally isolated from the other groups of blogs, in tiny type, with only a few links to it. This raises an interesting question, which is—how could Ms. Jarreau’s results be so far from reality? I say “far from reality” because by just about any measure, Watts Up With That is at the center of the climate blogosphere. Whether you look at total page views, “bounce rate”, page views per visitor, daily time on site, Alexa rating, you name the metric, WUWT comes out an order of magnitude ahead of any other climate blog. For example, Alexa rates WUWT as the 20,839th most popular blog worldwide … while RealClimate is an order of magnitude lower down, at 217,939th among all blogs.

Not only that, but WUWT is read by people on both sides of the climate divide, as evidenced by the number of AGW-supporting individuals and sites who comment on the WUWT posts, both at their blogs, on Twitter, and to their co-workers. The AGW supporters may only read it to see what the opposition is up to … but given their steady rate of responses, HotWhopper Sou and others read it regularly. Now to be fair, the results of Ms. Jarreau are preliminary … but still, how did her analysis get it so wrong? I’d say three things contributed to the skewed results. First, people don’t always tell the truth. Ms. HotWhopper is the obvious poster child for this. From the topics of her posts it’s obvious that she spends a whole lot of time reading WUWT … but she didn’t list it. I suspect that for some people, it’s a guilty pleasure, but that if asked, they’d say the equivalent of “I only read Playboy for the articles” …

The second reason for the skewed results is the way that news of the survey was passed around. It doesn’t appear that there was sufficient effort given to ensuring that the questionnaire was widely distributed. A better method might have been to write up a description and invitation to participate in the study, and to ask the various blogs to use it as a guest post. In any case, more thought about how to select participants is definitely indicated. Third, and in my opinion most important, there appears to have been no definition of terms, particularly as to what constitutes a “science site”. A large number of people in Lucias thread said well, HotWhopper Sou didn’t list WUWT because it’s not a science site … and according to them, the evidence for their claim that WUWT is not a science site is that often WUWT publishes studies which later turn out to be incorrect in some way, and sometimes are totally mistaken and wrong. To me, this reflects a profound misunderstanding of what makes a science site. The problem is that there are different kinds of “science sites”.

I’ll use mostly climate science sites as examples, as I’m familiar with them. One kind of science site is just a news aggregation site. The best example of this kind is Climate Depot. It just puts up links to stories about the climate with little commentary. Another kind of science site generally restricts itself to discussions of peer-reviewed science, but gives some commentary on each link. “It’s Not Rocket Science” or the Scientific American blog are examples of this category. Another kind of science site mostly deals with the original work of an individual author. Climate Audit and Isaac Held’s blog are examples of this kind of site. Then we have sites such as Lucia Liljegren’s site, or Judith Curry’s site, which reflect the individual interests of the author but which range widely over a number of subjects. Finally, we have Watts Up With That (WUWT). What makes WUWT unusual is that it is not a site that publicly discusses peer-reviewed science documents. Instead, it is a site for the public peer-review of science documents, including original work done by guest authors such as myself, and also studies which have been peer-reviewed by one of the journals.

This is a very different animal. To start with, just as happens with the secret peer review which is the usual format for the journals, not all of the papers that are reviewed will pass muster. Of course the journals don’t publish anything that doesn’t pass peer-review, they are hidden from view. But for public peer review such as goes on at WUWT, everything is visible, good, bad, and ugly. So when people complain that there are misguided or incorrect scientific claims posted at WUWT … well, doh. That’s an unavoidable part of the public peer-review game. Some of the pieces won’t make the cut. Not only that, but it is an extremely important part of the game. Knowing not only which scientific claims are wrong, but exactly why they are wrong, is perhaps more important than knowing which scientific claims are right. So yes, there is some very sketchy science that sometimes gets published and publicly peer-reviewed at WUWT … and almost invariably, it gets shot full of holes in short order. This makes WUWT more of a scientific site, not less of one. You don’t see that kind of thing happening at say RealClimate (RC) for a simple reason—such comments are invisibly and ruthlessly censored.

And that is why the Jarreau claim that RC is at the center of anything scientific is a joke. Science doesn’t censor scientific comments, and RC does censor scientific comments. You do the math. (Of course, all sites censor comments that violate blog policy, such as those that are vulgar or insulting, or wildly off-topic … but RC censors polite, on-topic, clearly scientific objections to their posts. No bueno.) As a frequent guest author, to me this pointing out of bad science is one of the most important aspects of WUWT—any mistake that I make will be identified in very short order. This has saved me immense amounts of wasted effort following blind trails … but some foolish folks think that my occasionally publishing claims that eventually turn out to be erroneous reduces the scientific value or nature of WUWT.

Nothing could be further from the truth.

Identifying errors and falsifying claims is central to science, and the only way to do so is to first publish the claims that later turn out to be wrong. As a result, Anthony has to undertake a continuous delicate balancing act. He doesn’t want to publish things that are obviously pseudo-science, but then he doesn’t want to exclude things that might be right … plus sometimes he wants to publish things that he knows are wrong simply so that their errors can be publicly identified. Does he make mistakes in the choices at times? Of course. It’s a tough job, and it is a job that no one individual could possibly be qualified to do, for a simple reason—nobody is as smart as the collective wisdom of the crowd. There’s no way to guess what errors a thousand readers might find in a piece that you or I might think is flawless. And there’s also no way to identify the odd and curious scientific claim that in a few years might be “settled science”.

So Anthony has to pick and choose, and not every choice is right … so what? Since the public falsification of bad science is essential to scientific progress, I find the idea that WUWT is not a science site because it sometimes posts shaky claims to be very parochial and short-sited. Private secret peer-review has obviously failed. In fact, many of the ridiculously bad “science” claims discussed at WUWT are peer-reviewed studies published in the most prestigious journals … but nobody can get that kind of nonsense past the kind of public peer review which is exemplified by WUWT. There are too many smart, insightful, capable people commenting on the posts for much to slip by …

So yes, WUWT does publish some obviously bad science, including obviously bad peer-reviewed science. But what some people fail to understand is, public falsification is the heart of science … and the only way to do that is to start by publishing and discussing that science, whether it is “good” or “bad”, and whether or not it’s already been peer-reviewed. In any case, those three reasons are why I think that the Jarreau results are so out of touch with reality.

My best to all, my great appreciation to Anthony for his tireless work, and my thanks to Lucia. On my planet her posts are almost always fascinating, as are the comments that they engender. And finally, my thanks to all the readers, lurkers, and commenters for your continued efforts to move the game forwards. w.

PS—The usual request: if you disagree with someone, please QUOTE THEIR EXACT WORDS THAT YOU THINK ARE WRONG. This allows everyone to be clear about exactly what it is that you are objecting to.

Addendum by Anthony:

I thank Willis for his analysis and for his kind words. It should be noted that as far as I know, I have never been contacted by Jarreau to ask to participate in the survey. Shades of Cook and Lewandowsky’s methodology where you get your desired result by selecting your sample beforehand. (i.e. only ask the people that are in your circle) – Anthony