Further Down the "Bore Hole"

A look at comment deletion at RealClimate compared with WUWT

Guest post submitted by Ian Rons

Regular readers will doubtless be familiar, either at first- or second-hand, with the enthusiasm with which moderators at RealClimate.org seem to reject comments from AGW sceptics. Ecotretas’ recent story on Realclimate censorship (re-posted here) piqued my interest, since in addition to the usual tones of indignation, it suggested a method of estimating the RealClimate comment deletion rate by looking at the comment IDs (as revealed by WordPress’s use of the HTML (attribute), and counting the number of these IDs which are missing from the sequence.

Being at a loose end, I took up the cudgels and wrote a script using PHP and cURL which took about an hour to mine every available page on realclimate.org by accessing the page using its WordPress “post_id” value (from http://…/?p=1 to http://…/?p=8092, as at (14th July), extracting comment IDs with a simple regular expression and doing a bit of maths on the result. Naturally, the source code and the various output files are available upon request.

The figures are rather high, though doubtless some missing comments will have been spam.

However, at least in recent times, the RC site has employed the “re-CAPTCHA”service, which (unlike the Akismet service used by WUWT) does not create a new comment ID if the comment is rejected for being spam, so for instance the 56% of comments missing during June 2011 seems likely to be an accurate figure, unless some other explanation can be found.

A possible explanation might be the existence of a large number of comments on the site by an inner circle of users hidden on special-access pages, but I find it hard to believe this could account for a large proportion of (e.g.) the 933 comments which are missing in June 2011. Similarly, the apparent surge of deletions beginning July 2007 may also be truly reflective of events, since it has been suggested in comments here that it “coincided” with RC’s attack on the surface stations project. However it has also been pointed out that such interpretations are impossible to verify using this method.

Overall, there were 78,639 missing comment IDs, out of a total of 210,595, or 37.3%. As for the RC page known as “The Bore Hole”, which started on 6th January 2011 (the date being evident from the post_id of 6013 and the moderator’s response to the first comment, although they have since re-dated it 6th December 2004 for some reason), the comments on that page are of course counted here as “published” comments, however they are small in number (404) when compared with the number of comments which seem to go missing even after that date in January (5,000). At the risk of mixing metaphors, “The Bore Hole” could perhaps be regarded as something of a fig-leaf.

I ran a similar scan on WUWT (also a WordPress site) on the 14th July, extracting data from http://…/?p=1 to http://…/?p=43440. However, analysing WUWT with this method presented several problems that aren’t applicable to the RC site:

  • Some earlier comment IDs are out of chronological order (stemming, it seems, from the import from TypePad to WordPress in October 2007), so figures for early months are impossible to calculate. During this period there seems to have been some infilling of comment IDs (probably due to the TypePad import not setting the “auto_increment” values in the database properly), which would affect the overall total; however, the numbers involved (whilst impossible to calculate precisely) are probably at most in the very low hundreds.
  • WUWT has always used WordPress’s Akismet spam-filtering, which creates new comment IDs before marking them as spam. Anthony provided me with a screenshot showing the total volume of spam which had been deleted as of early on the 15th July to be 55,097. This can be adjusted down to 55,085 for the period covered by my data to late on the 14th July 2011.
  • The Tips & Notes page encourages comments from readers which are not intended to remain permanently on the site, so they are to be regarded as “legitimate” deletions. Anthony provided me with records of the numbers of Tips & Notes comments posted (then eventually deleted) for the period 24th March to 10th July 2011 (3,220), on which I based an estimate of 22,215 “legitimately deleted” comments for the period 23rd June 2009 (when the T&N page was created) to 14th July 2011 inclusive.

Overall, WUWT has 75,989 missing comments IDs, out of a total of 700,115 submitted comments (10.9%). Subtracting the above figures for Akismet and Tips & Notes gives us a problem, since it’s a negative figure: -1,311. I think this is most likely due to an over-estimation of the number of comments posted on the Tips & Notes page, combined with perhaps a few hundred from the infilling problem mentioned above. However, the combined additions from these two sources of error would have to be in excess of 8,000 to raise the number of deletions to 1% of the total submitted, which I think very unlikely.

Putting it another way, and assuming a total of 200 “infilled” comment IDs (a high estimate, in my opinion), I would have to have over-estimated the volume of Tips & Notes comments by some 58% to reach a 1% deletion rate. I therefore see no reason to doubt the claims made on behalf of WUWT that the deletion rate is less than 1%. In fact it may be considerably lower. It is, however, noteworthy that December 2007 and January 2008 show high deletion rates, with another bump during Sep-Oct 2008:

In summary, whilst there remains some uncertainty especially regarding the RC data, the data does tend to support the anecdotal evidence concerning RC’s tendentious comment moderation practices. It also tends to support (or at least does not contradict) WUWT’s claims of a <1% comment deletion record.

For reference, here are the monthly totals which I used for the graphs. This table excludes incomplete months and some early WUWT months as noted:

RealClimate Watts Up With That?
Month Missing Submitted Missing (%) Missing Submitted Missing (%)
Jan 2005 86 524 16.4%
Feb 2005 111 383 29%
Mar 2005 53 286 18.5%
Apr 2005 96 294 32.7%
May 2005 47 305 15.4%
Jun 2005 119 482 24.7%
Jul 2005 524 826 63.4%
Aug 2005 255 474 53.8%
Sep 2005 112 527 21.3%
Oct 2005 99 664 14.9%
Nov 2005 67 654 10.2%
Dec 2005 544 1150 47.3%
Jan 2006 277 944 29.3%
Feb 2006 306 1236 24.8%
Mar 2006 390 1292 30.2%
Apr 2006 660 2130 31%
May 2006 580 1477 39.3%
Jun 2006 174 995 17.5%
Jul 2006 142 1252 11.3%
Aug 2006 888 2123 41.8%
Sep 2006 253 1005 25.2%
Oct 2006 340 1055 32.2%
Nov 2006 114 1290 8.8%
Dec 2006 62 876 7.1%
Jan 2007 203 1791 11.3%
Feb 2007 223 2282 9.8%
Mar 2007 343 3107 11%
Apr 2007 160 1960 8.2%
May 2007 213 2271 9.4%
Jun 2007 188 2055 9.1%
Jul 2007 4061 5724 70.9%
Aug 2007 7171 9511 75.4%
Sep 2007 4140 5499 75.3%
Oct 2007 4561 7091 64.3%
Nov 2007 6064 8226 73.7% 108 476 22.7%
Dec 2007 4184 6073 68.9% 547 869 62.9%
Jan 2008 493 1938 25.4% 497 1217 40.8%
Feb 2008 452 1656 27.3% 536 2027 26.4%
Mar 2008 332 1444 23% 776 3212 24.2%
Apr 2008 854 2222 38.4% 396 3023 13.1%
May 2008 1159 3050 38% 465 3192 14.6%
Jun 2008 880 2526 34.8% 586 5781 10.1%
Jul 2008 1156 3086 37.5% 751 6651 11.3%
Aug 2008 922 2733 33.7% 514 6775 7.6%
Sep 2008 873 2827 30.9% 1596 9174 17.4%
Oct 2008 692 1892 36.6% 1918 8936 21.5%
Nov 2008 1466 3026 48.4% 931 7012 13.3%
Dec 2008 1089 3127 34.8% 436 7599 5.7%
Jan 2009 1063 3269 32.5% 508 11357 4.5%
Feb 2009 834 2587 32.2% 1053 12586 8.4%
Mar 2009 1232 3260 37.8% 857 16186 5.3%
Apr 2009 1635 4369 37.4% 662 16291 4.1%
May 2009 2037 4361 46.7% 641 14217 4.5%
Jun 2009 808 3183 25.4% 1236 13525 9.1%
Jul 2009 646 3664 17.6% 1561 14722 10.6%
Aug 2009 384 2341 16.4% 1606 13619 11.8%
Sep 2009 337 1657 20.3% 1802 15389 11.7%
Oct 2009 722 3699 19.5% 2187 19746 11.1%
Nov 2009 1518 5745 26.4% 2945 25712 11.5%
Dec 2009 981 6401 15.3% 4339 36716 11.8%
Jan 2010 728 5349 13.6% 2250 26840 8.4%
Feb 2010 966 6020 16% 2267 26640 8.5%
Mar 2010 873 5066 17.2% 2349 26051 9%
Apr 2010 883 4227 20.9% 2312 23259 9.9%
May 2010 966 3425 28.2% 2877 20174 14.3%
Jun 2010 983 2915 33.7% 2295 19584 11.7%
Jul 2010 1613 3808 42.4% 2789 23840 11.7%
Aug 2010 772 2324 33.2% 3211 27241 11.8%
Sep 2010 770 2072 37.2% 3414 24257 14.1%
Oct 2010 681 2267 30% 2547 24362 10.5%
Nov 2010 824 2698 30.5% 2667 20508 13%
Dec 2010 1942 3744 51.9% 1983 22411 8.8%
Jan 2011 685 2794 24.5% 2716 24451 11.1%
Feb 2011 963 2901 33.2% 2243 22524 10%
Mar 2011 1077 2326 46.3% 2371 23480 10.1%
Apr 2011 684 1674 40.9% 2124 17466 12.2%
May 2011 738 1679 44% 2457 20028 12.3%
Jun 2011 933 1677 55.6% 2544 20682 12.3%

====================================================================

UPDATE:

Thanks to Ian. It should be noted that I had no influence of any kind on his analysis, other than providing the input data he requested. It is published exactly as he presented it to me, with only some small edits for formatting, with no content changes.

I thought this might be a good time to show something I encountered personally on June 7th, 2009 at RC. Gavin posted up a thread asking for ideas about the blog.

http://www.realclimate.org/index.php/archives/2009/06/groundhog-day-2/

His central question to readers was:

“What is it that you feel needs more explaining?”

I decided I’d offer my suggestion. Big mistake. Here’s a series of screen caps I made illustrating the central systemic bias that RC has, even for basic and germane topics.

It starts out like this when my first suggestion was not published:

It never appeared, so I thought I’d try an experiment. Using my wife’s computer (on the same DSL circuit, same IP address) I decided I’d submit an upbeat generic comment that didn’t offer any sort of challenge to RC using a new email account to see if it was an automation problem related to IP or my name/email address, or if it was simply that RC does not like challenges to their position:

And amazingly, it went right through. So I knew I was not being blocked by IP address or name/keyword, as you can see below, it was approved:

So, I tried again, again on the same home network, my PC this time:

And here it is awaiting moderation:

Nope, it was consigned to the ether:

A few comments later, we can see who is moderating, Gavin himself, note the inline response:

I decided to send a polite email inquiring about my missing comments:

And of course, I never received a response.

So there you have it, even when they ASK for ideas, ones that come from skeptics are apparently deleted; real open debate from a Real Climate scientist, Dr. Gavin Schmidt of NASA GISS.

Update#2 Ric Werme asks in comments:

The next question is “Why do people even bother posting comments at RC?” I’ve found it easier to not go there at all, so I don’t get tempted to add a comment.

Apparently, other than dhogaza and a few hangers on, not many do:

Except for search engine hits, WUWT beats RC in every measure. See for yourself here:

http://www.alexa.com/siteinfo/wattsupwiththat.com+realclimate.org#trafficstats

0 0 votes
Article Rating

Discover more from Watts Up With That?

Subscribe to get the latest posts sent to your email.

162 Comments
Inline Feedbacks
View all comments
July 23, 2011 6:57 am

I have had the same experiences. Nice post.

DirkH
July 23, 2011 7:25 am

Here’s Futurism Now on the web.
http://www.futurismnow.com/
Looks like fun:
“The Gulf of Mexico waters are dying, and not so slowly any more. Will life even exist in it in 20 years?”

John Whitman
July 23, 2011 8:16 am

My journey to WUWT in late 2008 involved the Climate Debate Daily (CDD) site. When first interested in the climate discussion I would go to CDD several times daily and taste the pro and con of the climate debate. Over time I became more interested in the con (skeptic) argument.
Of the skeptic sites, WUWT just looked more open and friendly. So, it became my default skeptic site.
I have commented rarely at RC. It is so rare that I do not really recall being deleted but probably I was. I do recall less than friendly reception. : )
Overall, I think RC has been pivotal in some major cultural shifts against the CAGW/IPCC focused pseudo-science. Without RC site policies and behavior actively alienating independent thinkers, I think there would be far fewer skeptics. So, for that, I thank RC.
John

kadaka (KD Knoebel)
July 23, 2011 9:13 am

Rabe said on July 23, 2011 at 3:52 am:

kadaka, you are bad, ROFL, just bad.

But not as bad as Michael Jackson, since he’s rotting. Currently he does have lower personal carbon emissions so in that way I’m badder than him, however his estate does generate a considerably huge amount of carbon pollution, far more than me. To compensate and be not as bad overall, I’m just going to have to leave behind a much smaller estate than Michael Jackson did. It’ll be difficult, but I think I can swing it.

Editor
July 23, 2011 9:14 am

Hans Moleman says:
July 22, 2011 at 5:59 pm

@Anthony
As for people taking me more seriously when using a single handle, I don’t see how they would’ve even known that I had posted in other articles under different names if you hadn’t brought it up here in order to deflect from the actual issues I raised.

I automatically derate any comment posted under a pseudonym, though I acknowledge some people have very good reasons for needing one.
At this point, I’m not taking you seriously until you do use your real name.
I’ve used mine on the Internet and ARPAnet since 1972 (sorry about that Email thing, we had no idea…) and have had no negative consequences.

Lady in Red
July 23, 2011 9:25 am

Ya know, John Whitman: I was thinking the same thing about RC — and being grateful that, while
I was straining to follow the tree ring problem on CA and the math on The Air Vent and…. RC was
so unfriendly and defensive, my radar, twitching antennae, warned me: there’s no there there.
So, raise a glass of cheer to RC.
In a similar vein, I enjoyed Keith Kloor’s Collide-a-Scape for a bit, also: a good hash about politics or climate or Sarah Palin over wine and dinner is fun! So, let’s go virtual with the fun! And, although Keith seems like a very nice person, I’m sure, the strain to identify *something* about which to argue — to no end whatsoever — brought me back down to life, to my dirty dishes and the extent to which, arguing about the angels on the head of a pin, day-in/day-out isn’t healthy for me. smile….. Better to spend the time parsing Judith Curry’s thinking.
…..Lady in Red

edbarbar
July 23, 2011 9:55 am

After being told what a great website realclimate.org is, I went there and made some points about some book about climate deniers. The post went through, but the crowd rose up in indignant answer, and the moderator never allowed a single response back.
That post generated a lot of responses, but not a single rebuttal allowed. Somehow, that post added to the convictions and the self-righteous indignation of a crowd of people who are full of group think. Amazing to me.

July 23, 2011 10:24 am

Ric Werme says:
July 22, 2011 at 10:42 am
James Sexton says:
July 22, 2011 at 9:36 am
The next question is “Why do people even bother posting comments at RC?” I’ve found it easier to not go there at all, so I don’t get tempted to add a comment.
=================================================================
The one and only reason to go over there a comment….. it to goad. 🙂 Its been almost a year since I commented over there. Some readers may recall the buzz RC had over a graph of M&W10, stating it verified the hockey stick. I went over there and suggested they were misinterpreting what the study stated. In some cases, they will allow these comments to be posted…… because they feel they can adequately respond to them. Mine we posted and responded to, and then ignored. I bet Gavin wishes he had listened instead of being embarrassed in the rejoinder. 🙂
Was much lulz for me in that one!!!

John Whitman
July 23, 2011 10:33 am

Lady in Red says:
July 23, 2011 at 9:25 am
””So, raise a glass of cheer to RC.””
——————
Lady in Red,
I agree with you that RC has caused many independents to take a more skeptical position. So I also will join you in raising a glass of cheer to RC. To RC!
NOTE: RC? What could they do, anyway, with most independent thinkers?
John

July 23, 2011 10:44 am

OK S. says:
July 22, 2011 at 12:46 pm
James Sexton says July 22, 2011 at 11:09 am:
“None can love freedom but good men;…….
As for the Milton quote, it’s sad that the misapplication of the First Amendment is used by teachers to restrict American school children from studying the works of some of the greatest thinkers of Western Civilization. ……..
===================================================================
While much credit to my exposure to many great works of literature must be extended to my father, (I had read Gibbon’s “Rise and Fall….” by a very young age….) much of the credit has to be extended to an old school marmish English instructor which refused to be told what works of literature should be taught to her students. Milton’s “The Tenure of Kings and Magistrates” was one of countless exposures to great works that I can attribute to her. It wasn’t that she instructed on so many, it was that she cultivated an insatiable appetite to read and to understand. Absence any para-normal powers, we must read to achieve understanding.
Thanks for the trip down memory lane.
James

Editor
July 23, 2011 10:49 am

kadaka (KD Knoebel) says:
July 23, 2011 at 9:13 am

To compensate and be not as bad overall, I’m just going to have to leave behind a much smaller estate than Michael Jackson did. It’ll be difficult, but I think I can swing it.

Social Security reform should take care of that, I think.
Hmm, the Fed’s 12 T$ debt ought to hold the promise significant C footprint shrinkage. 🙂

Editor
July 23, 2011 11:01 am

Barry Woods says:
July 23, 2011 at 2:05 am

A very good friend directed me to the Realclimate blog, when the climateate story broke. I had never been interested in or aware of C Audit, Watts Up, B Hill, etc before the 20th Novemner 2009.

If I can twist your comment into a different thread…. (Ok, there’s no doubt I can, whether it gets past the moderators is, well, not a problem!)
Traffic here increased quite a bit after Climategate. I expected it to fall back in the weeks afterward, but it never really did.
So, questions to you readers who joined then:
> Did you know about WUWT before Climategate?
> How has your understanding of global warming changed since then?
> Why are you still here? (And a very belated welcome to you all!)

Dennis Wingo
July 23, 2011 11:30 am

I think that I see a hockey stick in that first graph.
🙂

Lady in Red
July 23, 2011 11:58 am

I knew of the “climate issue” prior to Climategate but after the death of my scientist husband dubbed myself too scientifically illiterate to handle the nuances of the respective arguments.
I stumbled across a London Telegraph piece complaining about UEA refusing to give data to Steve McIntyre. I was amazed, confused, wrote to an oceanographer friend and asked if it were true: No, no, NO! he said. But, but BUT! said I.
CA was much too dense for me at the beginning. RC was nasty from the start. (How dumb was I? I actually believed I might help negotiate a truce between CA and RC, that the differences were misunderstandings. Yesh. I did think that.) When Climategate happened, I thought all of “climate science” would chant an enormous mea culpa. And science would go on. Nope. Took me a bit to find Jeff Id and the Blackboard and Bishop Hill.
I stay with WUWT and the rest because you’re smart and fun. I loved the poke in the eye to Eric Steig by Jeff Id, Ryan, et al at the beginning of the year. I can’t imagine “scientists” with slower thicker brains than the well-trained but poorly educated “climate science” monkeys. On the other hand, you’re agile and facile and, of course, you will “win” in the long term, but it’s the fun of watching the checkers game unfold that keeps me hanging around now.
Anthony Watts and Steve McIntyre (and probably several others I haven’t followed closely enough) are awe-inspiring. If Gavin and Michael Mann weren’t so arrogant and pompous, it would be sad to observe them getting bludgeoned. But, they are. And, thus, it is fun.
Plus, long after the fat cat retirement of the mediocre bureaucrats who bluster and sputter and defend the indefensible, I’ve watched and “touched” a bit of science history. This is one for the history books. And, I was there! Smile……
Ah, also: one person at a time, I try to introduce some to the sites (and RC and Climate Progress are good counterpoints for thinking folk. (Poor Joe Romm! He’s not as dumb as the rest of them, I think, and I can only hope that he is getting big big big bucks for his intellectual prostitution. He has to go to reunions at MIT, I imagine, and all the old classmates must avert their eyes, or slap his back and move on quickly.)
Oh well. Chuckle. As the old joke goes: “We’ve established what you are. Now, we are just haggling over the price.” …..Lady in Red

July 23, 2011 12:04 pm

Hans Moleman says:
July 22, 2011 at 5:59 pm
“It’s too bad, since it’s a prime example of the things I’ve been railing against today: jumping to conclusions when you have much uncertainty……..”
====================================================
Our friend Hans has taken to writing comedy!!
Hans, read the comments here. The data presented only reinforces personal experience. Here’s the thing. Anyone who has engaged in the climate discussion on the blogosphere, for any considerable length of time has ventured over to RC. As Smokey has pointed out, your criticisms would be valid if these were singular and rare events. The data shows this isn’t the case. But, more damning is the shared experience with many of the commentators here.
Now, contrast that with your experience here. You may not like the opinions here, you may not like the information presented, but your voice is heard. You are allowed to state your piece, even when you violate stated site policy!!! While I’m sure none is expected, you owe Anthony some gratitude. For he has given you great latitude which is much more than would be offered by the people you so earnestly are defending. He has been much more gracious than I would have been. I would have allowed your posts only for purposes of ridicule and scorn. Imagine, someone embracing censorship while purporting pretending to be an advocate of science.
Hans, you serve us well. My thanks.
James Sexton

KenB
July 23, 2011 12:13 pm

I too, am of the opinion that RC has been the tipping point, away from AGW to a more sceptical position. In a quest to explore the intricacies of Climate Science I visited many “science blogs” read books, observed comments, and also the demonstrated knowledge of those that frequented the sites.
When the climate tide turned, I was interested to see and hear what many scientists had to say on the topic of why and how, they came to question the consensus and science. The almost universal theme for change came about when they used or tried to use their own expert knowledge (as a scientist) to correct a scientific fact on RC! The treatment, the scorn, the denial of a right of reply or, even to question why, exposed RC as both a megaphone for their own views, and little to do with the advancement of knowledge or science.
My observations of that site tends to confirm that view, though I did feel they made an effort at one time to allow more discourse, but the attack dog (mad dog?) attitude of some of their prized regulars put me off the place permanently. So I guess they have in their own way contributed to a closer scrutiny of the science of climate, by scientists!
So they have been useful if only for that!

dp
July 23, 2011 3:47 pm

Lars P says:
July 22, 2011 at 12:25 pm
dp says:
July 22, 2011 at 11:14 am
“This is a crazy survey. ”
dp from your comment I have the feeling you did not encounter the experience of trying to communicate, having valid arguments and being simply rubbed away by somebody wanting to have only his truth standing.

Actually I’m an IT professional who has been in the business far longer than the WWW has existed, and currently host lots of WordPress sites and no, I’m not looking for more business as I have too much to do already. I have been involved in data mining and business intelligence for years, have seen thousands of pages of web log reports, and have analyzed what they mean till I’m bored to death with it. There are and are not things you can infer from URLs, changes to URLs, and sequences of URLs. Particularly databased URLs such as WP provides. The database that runs behind the web server code is frequently on a server different than the one the web pages are served from. In good environments there are parallel systems for reliability and performance. In many cases it is needed to do hardware refreshes or software refreshes, or to migrate date to a different technology. It also happens that data merges need to happen across servers, or it may be that it is needed to segment sections of a web site to less expensive hardware. Any one of these maintenance operations can affect the URL that is generated by the database behind the web server running WP.
More over, it is not possible for me or anyone on the outside to know if the change is related to a maintenance activity or to intentional, and perhaps intentionally misleading human activity. The uncertainty is very high without knowing more about how and where the databases are for all the WordPress segments.
I host WP pages and actually have three blogs of my own that are built up of three different WP instances and databases, but so tightly integrated nobody would know. Or care. It would be silly for anyone to presume they can use URLs from those pages forensically. The web server logs would be required as well as the URL list to establish that fact.
None of this considers site policy that can and will impact web content and history, including purged content history. And we have seen in a recent post here that policy may be applied arbitrarily, depending upon who is affected by that policy.
To your point about having been shut out – yes, in fact I have been summarily booted from Tamino’s site every time I’ve posted there. I do that as a kind of sport, actually, because some people are just more pleasant when they are very annoyed and I am responsible for it. The posts I create are reasonable, of course, but definitely not what he is interested in including among his disinformation.
So I think RC is guilty of many of the things suggested in the OP here, but that the chosen proof falls short of being proof. And while I can think what I wish, without proof I can conclude nothing. Climate is like that, too.

Ian Rons
July 23, 2011 11:18 pm

dp,
You say you’re an IT professional with vast experience in all the relevant areas (OK, what’s your name and where have you been employed?), yet you are effectively arguing that one might have to alter the contents of a WP database — actually the primary key of the (main) posts table — when migrating across hardware or software (you mean database software, presumably). Remind me not to use your company for webhosting.
You also suggest that RealClimate might be composed of several different WordPress installations, all cunningly hidden behind webserver URL rewrites, or something. Are you suggesting, for instance, that all the posts from 2008 onwards might be hosted on a totally different WordPress installation? Well, I didn’t think of that, but now that you mention it, why wouldn’t you want to do that? It’s just too boring to add posts to one’s existing WP installation like a normal person. Imagine the joy of having three different login pages for staff, etc.!
However, I’ve scanned the entire site using Xenu, and on the basis of that I am confident that there is no WordPress-defying weirdness going on behind the scenes, and that there are no pages with comments on that can’t be accessed using the URLs http://…/?p=x, where x=1 to 8092.

None of this considers site policy that can and will impact web content and history, including purged content history.

If pages have been deleted then this would have some effect on the stats, though there would have to be a lot of deleted pages all replete with comments to have any significant effect. I have found no internal links on RC that point to any pages now deleted, except the following:
http://www.realclimate.org/index.php/archives/2005/01/draft-do-not-post/
which can reasonably be discounted, I think. This doesn’t mean that no pages have been deleted, just that I can’t find internal evidence of any.
For reference, there are 14 URLs which appear to be deleted pages (i.e., they give 404 errors), but in fact they have merely been renamed:
http://www.realclimate.org/index.php/archives/2004/12/non-significant-trend-in-cosmic-galactic-rays-and-climate-change/
http://www.realclimate.org/index.php/archives/2006/06/national-%20academies-synthesis-report/
http://www.realclimate.org/index.php/archives/2006/06/noaa-apres-moi-le-deluge/
http://www.realclimate.org/index.php/archives/2006/10/con-All%E8gre-ma-non-troppo/
http://www.realclimate.org/index.php/archives/2007/10/cosmic-rays-don%E2%80%99t-die-so-easily/
http://www.realclimate.org/index.php/archives/2007/10/cosmic-rays-don%e2%80%99t-die-so-easily/
http://www.realclimate.org/index.php/archives/2007/12/les-chevaliers-de-l%E2%80%99ordre-de-la-terre-plate-part-ii-courtillots-geomagnetic-excursion/
http://www.realclimate.org/index.php/archives/2007/12/les-chevaliers-de-l%e2%80%99ordre-de-la-terre-plate-part-ii-courtillots-geomagnetic-excursion/
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%E2%80%93dispatch-2/
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%E2%80%93dispatch-3/:
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%E2%80%93dispatch-6/:
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%e2%80%93dispatch-3/:
http://www.realclimate.org/index.php/archives/2010/08/expert-credibility-in-climate-change-%E2%80%93-responses-to-comments/
So far as anecdotal evidence goes, RealClimate does not (so far as I am aware) have a record of deleting posts. Comments, but not posts. I would imagine a number of people here check out RC on a regular basis: have any posts been deleted in the last month, for instance? Or in every month that I’ve plotted on the graph? No, whilst it’s a reasonable comment to make, I don’t think post deletions are really problematic. A quick estimate suggests about 5 deleted posts — if full of comments — would equate to about 1% of total deletions. In general, website owners really don’t like doing that sort of thing, and as I say I don’t see any actual evidence that RealClimate has done so.

Brian H
July 24, 2011 8:28 am

The article link to the Alexa stats now returns:

“No data”
and
“We don’t have enough data to display the traffic metrics for wattsupwiththat.com”

And no graphs.

aeroguy48
July 24, 2011 8:46 am

De-fund the NASA GISS now! Then magically Gavin Schmidt is gone, then in 2012 when Joe Bastardi says el-nino will be gone and the conservatives take office-kicking out all the commies in the highest levels of OUR government then we can expose the fraud.

Brian H
July 24, 2011 8:51 am

Forming my own query gave full stats and graphs. Some very interesting stuff. Currently, about 50% of WUTW visitors have just 1 pageview, but it now surpasses 90% for RC.
And here is an fascinatiing list of rank by country:
New Zealand 839
Australia 1,811
Finland 2,747
Canada 4,434
Dominican
Republic 5,019
United States 7,055
Denmark 7,284
UK 9,238
Japan 12,613
Sweden 14,981
Spain 22,432
Germany 26,076
Italy 29,908
Netherlands 42,046
Pakistan 43,645
India 100,056
Note the absence of China. But what about Taiwan, etc?
Interesting that it is much higher in Canada than the US.

Brian H
July 24, 2011 8:55 am

Typoz: “a fascinating” not “an fascinatiing”.

dp
July 24, 2011 10:35 am

Ian Rons says:
bla bla bla
I said URLs are not enough evidence. I’ve said merging data across systems can produce surprises. Mixing data across time and systems can produce conflicts. A blog db is not like a billing db where consistency is worth the expense to preserve it. I’ve said we don’t know enough to say with certainty, based on URL information, what the truth is. And you don’t, either.
I’ve also said I agree with the conclusions of the OP but don’t agree the evidence is adequate to prove it. And I’m right about that.

dp
July 24, 2011 11:28 am

Ian – lets avoid future ad homs and discuss what we know and infer what is realistic, and then see what scenarios are possible. And to be honest, this is tedious and has nothing what so ever to do with science.
Real Climate appears to be a WordPress blog hosted by Web Faction – a hosting company. We learn this simply using the DNS lookup tool:

bash-3.2$ nslookup realclimate.org
Server: 192.168.1.13
Address: 192.168.1.13#53
Non-authoritative answer:
Name: realclimate.org
Address: 174.133.50.216
bash-3.2$ nslookup 174.133.50.216
Server: 192.168.1.13
Address: 192.168.1.13#53
Non-authoritative answer:
216.50.133.174.in-addr.arpa name = web83.webfaction.com.
Authoritative answers can be found from:
50.133.174.in-addr.arpa nameserver = ns1.theplanet.com.
50.133.174.in-addr.arpa nameserver = ns2.theplanet.com.
ns1.theplanet.com internet address = 207.218.247.135
ns2.theplanet.com internet address = 207.218.223.162

So we see that realclimate.org shares an ip (at this time) with web83.webfaction.com. What can we learn about webfaction.com? How about using whois:

Query: webfaction.com
Registry: whois.tucows.com
Results:
Registrant:
Swarma Limited
Office 404, Albany House, 324 Regent Street
London, W1B 3HH
GB
Domain name: WEBFACTION.COM
Administrative Contact:
Admin, DNS hostmaster@webfaction.com
Office 404, Albany House, 324 Regent Street
London, W1B 3HH
GB
+44.8458620318
Technical Contact:
Admin, DNS hostmaster@webfaction.com
Office 404, Albany House, 324 Regent Street
London, W1B 3HH
GB
+44.8458620318

We can use traceroute and other tools to attempt to locate the physical site but that is particularly important.
And what do we see when we visit http://www.webfaction.com/ ?
A rather typical hosting service. Now we know there is a hosting service, and a web farm. We can’t see what the database is hosted on or how many data base servers they make available. We can infer the hosts use a common methodology to improve reliability and performance including BigIP, clustered back end systems, virtual machines to keep costs down. We can infer that to improve their bottom line they offer maintenance, backup, and OS patching, and other infrastructure services. Like me they may provide first line support when the WordPress user krumples the installation. Like me they may handle migration of data from MySQL 3.x to 5.x and deal with all the attendant data dictionary impacts something like that may involve. Like me they may prefer to do this pro-actively on a schedule so they don’t have to do it as an out of sequence restoration because the end user hasn’t the skill or interest. It makes our AR sing and it is a good all around business strategy.
We may also infer that the end user accepts good service and a reduced price if unimportant metrics become modified in the process.
But we have no way to know the truth behind what we infer – we know they are plausible scenarios. And here’s another plausible scenario – the customer requested, or directly performed the operation of fudging dates to create a false impression.
Many of us suspect they are guilty of fudging data. It would be nice if we had a higher level of certainty.

Ian Rons
July 24, 2011 3:26 pm

dp,
Differences between MySQL <4/4.0 and 4.1/5.xxx are irrelevant, and it's totally spurious to raise such issues. FUD.