A look at comment deletion at RealClimate compared with WUWT
Guest post submitted by Ian Rons
Regular readers will doubtless be familiar, either at first- or second-hand, with the enthusiasm with which moderators at RealClimate.org seem to reject comments from AGW sceptics. Ecotretas’ recent story on Realclimate censorship (re-posted here) piqued my interest, since in addition to the usual tones of indignation, it suggested a method of estimating the RealClimate comment deletion rate by looking at the comment IDs (as revealed by WordPress’s use of the HTML (attribute), and counting the number of these IDs which are missing from the sequence.
Being at a loose end, I took up the cudgels and wrote a script using PHP and cURL which took about an hour to mine every available page on realclimate.org by accessing the page using its WordPress “post_id” value (from http://…/?p=1 to http://…/?p=8092, as at (14th July), extracting comment IDs with a simple regular expression and doing a bit of maths on the result. Naturally, the source code and the various output files are available upon request.
The figures are rather high, though doubtless some missing comments will have been spam.
However, at least in recent times, the RC site has employed the “re-CAPTCHA”service, which (unlike the Akismet service used by WUWT) does not create a new comment ID if the comment is rejected for being spam, so for instance the 56% of comments missing during June 2011 seems likely to be an accurate figure, unless some other explanation can be found.
A possible explanation might be the existence of a large number of comments on the site by an inner circle of users hidden on special-access pages, but I find it hard to believe this could account for a large proportion of (e.g.) the 933 comments which are missing in June 2011. Similarly, the apparent surge of deletions beginning July 2007 may also be truly reflective of events, since it has been suggested in comments here that it “coincided” with RC’s attack on the surface stations project. However it has also been pointed out that such interpretations are impossible to verify using this method.
Overall, there were 78,639 missing comment IDs, out of a total of 210,595, or 37.3%. As for the RC page known as “The Bore Hole”, which started on 6th January 2011 (the date being evident from the post_id of 6013 and the moderator’s response to the first comment, although they have since re-dated it 6th December 2004 for some reason), the comments on that page are of course counted here as “published” comments, however they are small in number (404) when compared with the number of comments which seem to go missing even after that date in January (5,000). At the risk of mixing metaphors, “The Bore Hole” could perhaps be regarded as something of a fig-leaf.
I ran a similar scan on WUWT (also a WordPress site) on the 14th July, extracting data from http://…/?p=1 to http://…/?p=43440. However, analysing WUWT with this method presented several problems that aren’t applicable to the RC site:
- Some earlier comment IDs are out of chronological order (stemming, it seems, from the import from TypePad to WordPress in October 2007), so figures for early months are impossible to calculate. During this period there seems to have been some infilling of comment IDs (probably due to the TypePad import not setting the “auto_increment” values in the database properly), which would affect the overall total; however, the numbers involved (whilst impossible to calculate precisely) are probably at most in the very low hundreds.
- WUWT has always used WordPress’s Akismet spam-filtering, which creates new comment IDs before marking them as spam. Anthony provided me with a screenshot showing the total volume of spam which had been deleted as of early on the 15th July to be 55,097. This can be adjusted down to 55,085 for the period covered by my data to late on the 14th July 2011.
- The Tips & Notes page encourages comments from readers which are not intended to remain permanently on the site, so they are to be regarded as “legitimate” deletions. Anthony provided me with records of the numbers of Tips & Notes comments posted (then eventually deleted) for the period 24th March to 10th July 2011 (3,220), on which I based an estimate of 22,215 “legitimately deleted” comments for the period 23rd June 2009 (when the T&N page was created) to 14th July 2011 inclusive.
Overall, WUWT has 75,989 missing comments IDs, out of a total of 700,115 submitted comments (10.9%). Subtracting the above figures for Akismet and Tips & Notes gives us a problem, since it’s a negative figure: -1,311. I think this is most likely due to an over-estimation of the number of comments posted on the Tips & Notes page, combined with perhaps a few hundred from the infilling problem mentioned above. However, the combined additions from these two sources of error would have to be in excess of 8,000 to raise the number of deletions to 1% of the total submitted, which I think very unlikely.
Putting it another way, and assuming a total of 200 “infilled” comment IDs (a high estimate, in my opinion), I would have to have over-estimated the volume of Tips & Notes comments by some 58% to reach a 1% deletion rate. I therefore see no reason to doubt the claims made on behalf of WUWT that the deletion rate is less than 1%. In fact it may be considerably lower. It is, however, noteworthy that December 2007 and January 2008 show high deletion rates, with another bump during Sep-Oct 2008:
In summary, whilst there remains some uncertainty especially regarding the RC data, the data does tend to support the anecdotal evidence concerning RC’s tendentious comment moderation practices. It also tends to support (or at least does not contradict) WUWT’s claims of a <1% comment deletion record.
For reference, here are the monthly totals which I used for the graphs. This table excludes incomplete months and some early WUWT months as noted:
| RealClimate | Watts Up With That? | |||||
|---|---|---|---|---|---|---|
| Month | Missing | Submitted | Missing (%) | Missing | Submitted | Missing (%) |
| Jan 2005 | 86 | 524 | 16.4% | |||
| Feb 2005 | 111 | 383 | 29% | |||
| Mar 2005 | 53 | 286 | 18.5% | |||
| Apr 2005 | 96 | 294 | 32.7% | |||
| May 2005 | 47 | 305 | 15.4% | |||
| Jun 2005 | 119 | 482 | 24.7% | |||
| Jul 2005 | 524 | 826 | 63.4% | |||
| Aug 2005 | 255 | 474 | 53.8% | |||
| Sep 2005 | 112 | 527 | 21.3% | |||
| Oct 2005 | 99 | 664 | 14.9% | |||
| Nov 2005 | 67 | 654 | 10.2% | |||
| Dec 2005 | 544 | 1150 | 47.3% | |||
| Jan 2006 | 277 | 944 | 29.3% | |||
| Feb 2006 | 306 | 1236 | 24.8% | |||
| Mar 2006 | 390 | 1292 | 30.2% | |||
| Apr 2006 | 660 | 2130 | 31% | |||
| May 2006 | 580 | 1477 | 39.3% | |||
| Jun 2006 | 174 | 995 | 17.5% | |||
| Jul 2006 | 142 | 1252 | 11.3% | |||
| Aug 2006 | 888 | 2123 | 41.8% | |||
| Sep 2006 | 253 | 1005 | 25.2% | |||
| Oct 2006 | 340 | 1055 | 32.2% | |||
| Nov 2006 | 114 | 1290 | 8.8% | |||
| Dec 2006 | 62 | 876 | 7.1% | |||
| Jan 2007 | 203 | 1791 | 11.3% | |||
| Feb 2007 | 223 | 2282 | 9.8% | |||
| Mar 2007 | 343 | 3107 | 11% | |||
| Apr 2007 | 160 | 1960 | 8.2% | |||
| May 2007 | 213 | 2271 | 9.4% | |||
| Jun 2007 | 188 | 2055 | 9.1% | |||
| Jul 2007 | 4061 | 5724 | 70.9% | |||
| Aug 2007 | 7171 | 9511 | 75.4% | |||
| Sep 2007 | 4140 | 5499 | 75.3% | |||
| Oct 2007 | 4561 | 7091 | 64.3% | |||
| Nov 2007 | 6064 | 8226 | 73.7% | 108 | 476 | 22.7% |
| Dec 2007 | 4184 | 6073 | 68.9% | 547 | 869 | 62.9% |
| Jan 2008 | 493 | 1938 | 25.4% | 497 | 1217 | 40.8% |
| Feb 2008 | 452 | 1656 | 27.3% | 536 | 2027 | 26.4% |
| Mar 2008 | 332 | 1444 | 23% | 776 | 3212 | 24.2% |
| Apr 2008 | 854 | 2222 | 38.4% | 396 | 3023 | 13.1% |
| May 2008 | 1159 | 3050 | 38% | 465 | 3192 | 14.6% |
| Jun 2008 | 880 | 2526 | 34.8% | 586 | 5781 | 10.1% |
| Jul 2008 | 1156 | 3086 | 37.5% | 751 | 6651 | 11.3% |
| Aug 2008 | 922 | 2733 | 33.7% | 514 | 6775 | 7.6% |
| Sep 2008 | 873 | 2827 | 30.9% | 1596 | 9174 | 17.4% |
| Oct 2008 | 692 | 1892 | 36.6% | 1918 | 8936 | 21.5% |
| Nov 2008 | 1466 | 3026 | 48.4% | 931 | 7012 | 13.3% |
| Dec 2008 | 1089 | 3127 | 34.8% | 436 | 7599 | 5.7% |
| Jan 2009 | 1063 | 3269 | 32.5% | 508 | 11357 | 4.5% |
| Feb 2009 | 834 | 2587 | 32.2% | 1053 | 12586 | 8.4% |
| Mar 2009 | 1232 | 3260 | 37.8% | 857 | 16186 | 5.3% |
| Apr 2009 | 1635 | 4369 | 37.4% | 662 | 16291 | 4.1% |
| May 2009 | 2037 | 4361 | 46.7% | 641 | 14217 | 4.5% |
| Jun 2009 | 808 | 3183 | 25.4% | 1236 | 13525 | 9.1% |
| Jul 2009 | 646 | 3664 | 17.6% | 1561 | 14722 | 10.6% |
| Aug 2009 | 384 | 2341 | 16.4% | 1606 | 13619 | 11.8% |
| Sep 2009 | 337 | 1657 | 20.3% | 1802 | 15389 | 11.7% |
| Oct 2009 | 722 | 3699 | 19.5% | 2187 | 19746 | 11.1% |
| Nov 2009 | 1518 | 5745 | 26.4% | 2945 | 25712 | 11.5% |
| Dec 2009 | 981 | 6401 | 15.3% | 4339 | 36716 | 11.8% |
| Jan 2010 | 728 | 5349 | 13.6% | 2250 | 26840 | 8.4% |
| Feb 2010 | 966 | 6020 | 16% | 2267 | 26640 | 8.5% |
| Mar 2010 | 873 | 5066 | 17.2% | 2349 | 26051 | 9% |
| Apr 2010 | 883 | 4227 | 20.9% | 2312 | 23259 | 9.9% |
| May 2010 | 966 | 3425 | 28.2% | 2877 | 20174 | 14.3% |
| Jun 2010 | 983 | 2915 | 33.7% | 2295 | 19584 | 11.7% |
| Jul 2010 | 1613 | 3808 | 42.4% | 2789 | 23840 | 11.7% |
| Aug 2010 | 772 | 2324 | 33.2% | 3211 | 27241 | 11.8% |
| Sep 2010 | 770 | 2072 | 37.2% | 3414 | 24257 | 14.1% |
| Oct 2010 | 681 | 2267 | 30% | 2547 | 24362 | 10.5% |
| Nov 2010 | 824 | 2698 | 30.5% | 2667 | 20508 | 13% |
| Dec 2010 | 1942 | 3744 | 51.9% | 1983 | 22411 | 8.8% |
| Jan 2011 | 685 | 2794 | 24.5% | 2716 | 24451 | 11.1% |
| Feb 2011 | 963 | 2901 | 33.2% | 2243 | 22524 | 10% |
| Mar 2011 | 1077 | 2326 | 46.3% | 2371 | 23480 | 10.1% |
| Apr 2011 | 684 | 1674 | 40.9% | 2124 | 17466 | 12.2% |
| May 2011 | 738 | 1679 | 44% | 2457 | 20028 | 12.3% |
| Jun 2011 | 933 | 1677 | 55.6% | 2544 | 20682 | 12.3% |
====================================================================
UPDATE:
Thanks to Ian. It should be noted that I had no influence of any kind on his analysis, other than providing the input data he requested. It is published exactly as he presented it to me, with only some small edits for formatting, with no content changes.
I thought this might be a good time to show something I encountered personally on June 7th, 2009 at RC. Gavin posted up a thread asking for ideas about the blog.
http://www.realclimate.org/index.php/archives/2009/06/groundhog-day-2/
His central question to readers was:
“What is it that you feel needs more explaining?”
I decided I’d offer my suggestion. Big mistake. Here’s a series of screen caps I made illustrating the central systemic bias that RC has, even for basic and germane topics.
It starts out like this when my first suggestion was not published:
It never appeared, so I thought I’d try an experiment. Using my wife’s computer (on the same DSL circuit, same IP address) I decided I’d submit an upbeat generic comment that didn’t offer any sort of challenge to RC using a new email account to see if it was an automation problem related to IP or my name/email address, or if it was simply that RC does not like challenges to their position:
And amazingly, it went right through. So I knew I was not being blocked by IP address or name/keyword, as you can see below, it was approved:
So, I tried again, again on the same home network, my PC this time:
And here it is awaiting moderation:
Nope, it was consigned to the ether:
A few comments later, we can see who is moderating, Gavin himself, note the inline response:
I decided to send a polite email inquiring about my missing comments:
And of course, I never received a response.
So there you have it, even when they ASK for ideas, ones that come from skeptics are apparently deleted; real open debate from a Real Climate scientist, Dr. Gavin Schmidt of NASA GISS.
Update#2 Ric Werme asks in comments:
The next question is “Why do people even bother posting comments at RC?” I’ve found it easier to not go there at all, so I don’t get tempted to add a comment.
Apparently, other than dhogaza and a few hangers on, not many do:
Except for search engine hits, WUWT beats RC in every measure. See for yourself here:
http://www.alexa.com/siteinfo/wattsupwiththat.com+realclimate.org#trafficstats
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.











I have had the same experiences. Nice post.
Here’s Futurism Now on the web.
http://www.futurismnow.com/
Looks like fun:
“The Gulf of Mexico waters are dying, and not so slowly any more. Will life even exist in it in 20 years?”
My journey to WUWT in late 2008 involved the Climate Debate Daily (CDD) site. When first interested in the climate discussion I would go to CDD several times daily and taste the pro and con of the climate debate. Over time I became more interested in the con (skeptic) argument.
Of the skeptic sites, WUWT just looked more open and friendly. So, it became my default skeptic site.
I have commented rarely at RC. It is so rare that I do not really recall being deleted but probably I was. I do recall less than friendly reception. : )
Overall, I think RC has been pivotal in some major cultural shifts against the CAGW/IPCC focused pseudo-science. Without RC site policies and behavior actively alienating independent thinkers, I think there would be far fewer skeptics. So, for that, I thank RC.
John
Rabe said on July 23, 2011 at 3:52 am:
But not as bad as Michael Jackson, since he’s rotting. Currently he does have lower personal carbon emissions so in that way I’m badder than him, however his estate does generate a considerably huge amount of carbon pollution, far more than me. To compensate and be not as bad overall, I’m just going to have to leave behind a much smaller estate than Michael Jackson did. It’ll be difficult, but I think I can swing it.
Hans Moleman says:
July 22, 2011 at 5:59 pm
I automatically derate any comment posted under a pseudonym, though I acknowledge some people have very good reasons for needing one.
At this point, I’m not taking you seriously until you do use your real name.
I’ve used mine on the Internet and ARPAnet since 1972 (sorry about that Email thing, we had no idea…) and have had no negative consequences.
Ya know, John Whitman: I was thinking the same thing about RC — and being grateful that, while
I was straining to follow the tree ring problem on CA and the math on The Air Vent and…. RC was
so unfriendly and defensive, my radar, twitching antennae, warned me: there’s no there there.
So, raise a glass of cheer to RC.
In a similar vein, I enjoyed Keith Kloor’s Collide-a-Scape for a bit, also: a good hash about politics or climate or Sarah Palin over wine and dinner is fun! So, let’s go virtual with the fun! And, although Keith seems like a very nice person, I’m sure, the strain to identify *something* about which to argue — to no end whatsoever — brought me back down to life, to my dirty dishes and the extent to which, arguing about the angels on the head of a pin, day-in/day-out isn’t healthy for me. smile….. Better to spend the time parsing Judith Curry’s thinking.
…..Lady in Red
After being told what a great website realclimate.org is, I went there and made some points about some book about climate deniers. The post went through, but the crowd rose up in indignant answer, and the moderator never allowed a single response back.
That post generated a lot of responses, but not a single rebuttal allowed. Somehow, that post added to the convictions and the self-righteous indignation of a crowd of people who are full of group think. Amazing to me.
Ric Werme says:
July 22, 2011 at 10:42 am
James Sexton says:
July 22, 2011 at 9:36 am
The next question is “Why do people even bother posting comments at RC?” I’ve found it easier to not go there at all, so I don’t get tempted to add a comment.
=================================================================
The one and only reason to go over there a comment….. it to goad. 🙂 Its been almost a year since I commented over there. Some readers may recall the buzz RC had over a graph of M&W10, stating it verified the hockey stick. I went over there and suggested they were misinterpreting what the study stated. In some cases, they will allow these comments to be posted…… because they feel they can adequately respond to them. Mine we posted and responded to, and then ignored. I bet Gavin wishes he had listened instead of being embarrassed in the rejoinder. 🙂
Was much lulz for me in that one!!!
Lady in Red says:
July 23, 2011 at 9:25 am
””So, raise a glass of cheer to RC.””
——————
Lady in Red,
I agree with you that RC has caused many independents to take a more skeptical position. So I also will join you in raising a glass of cheer to RC. To RC!
NOTE: RC? What could they do, anyway, with most independent thinkers?
John
OK S. says:
July 22, 2011 at 12:46 pm
James Sexton says @ur momisugly July 22, 2011 at 11:09 am:
“None can love freedom but good men;…….
As for the Milton quote, it’s sad that the misapplication of the First Amendment is used by teachers to restrict American school children from studying the works of some of the greatest thinkers of Western Civilization. ……..
===================================================================
While much credit to my exposure to many great works of literature must be extended to my father, (I had read Gibbon’s “Rise and Fall….” by a very young age….) much of the credit has to be extended to an old school marmish English instructor which refused to be told what works of literature should be taught to her students. Milton’s “The Tenure of Kings and Magistrates” was one of countless exposures to great works that I can attribute to her. It wasn’t that she instructed on so many, it was that she cultivated an insatiable appetite to read and to understand. Absence any para-normal powers, we must read to achieve understanding.
Thanks for the trip down memory lane.
James
kadaka (KD Knoebel) says:
July 23, 2011 at 9:13 am
Social Security reform should take care of that, I think.
Hmm, the Fed’s 12 T$ debt ought to hold the promise significant C footprint shrinkage. 🙂
Barry Woods says:
July 23, 2011 at 2:05 am
If I can twist your comment into a different thread…. (Ok, there’s no doubt I can, whether it gets past the moderators is, well, not a problem!)
Traffic here increased quite a bit after Climategate. I expected it to fall back in the weeks afterward, but it never really did.
So, questions to you readers who joined then:
> Did you know about WUWT before Climategate?
> How has your understanding of global warming changed since then?
> Why are you still here? (And a very belated welcome to you all!)
I think that I see a hockey stick in that first graph.
🙂
I knew of the “climate issue” prior to Climategate but after the death of my scientist husband dubbed myself too scientifically illiterate to handle the nuances of the respective arguments.
I stumbled across a London Telegraph piece complaining about UEA refusing to give data to Steve McIntyre. I was amazed, confused, wrote to an oceanographer friend and asked if it were true: No, no, NO! he said. But, but BUT! said I.
CA was much too dense for me at the beginning. RC was nasty from the start. (How dumb was I? I actually believed I might help negotiate a truce between CA and RC, that the differences were misunderstandings. Yesh. I did think that.) When Climategate happened, I thought all of “climate science” would chant an enormous mea culpa. And science would go on. Nope. Took me a bit to find Jeff Id and the Blackboard and Bishop Hill.
I stay with WUWT and the rest because you’re smart and fun. I loved the poke in the eye to Eric Steig by Jeff Id, Ryan, et al at the beginning of the year. I can’t imagine “scientists” with slower thicker brains than the well-trained but poorly educated “climate science” monkeys. On the other hand, you’re agile and facile and, of course, you will “win” in the long term, but it’s the fun of watching the checkers game unfold that keeps me hanging around now.
Anthony Watts and Steve McIntyre (and probably several others I haven’t followed closely enough) are awe-inspiring. If Gavin and Michael Mann weren’t so arrogant and pompous, it would be sad to observe them getting bludgeoned. But, they are. And, thus, it is fun.
Plus, long after the fat cat retirement of the mediocre bureaucrats who bluster and sputter and defend the indefensible, I’ve watched and “touched” a bit of science history. This is one for the history books. And, I was there! Smile……
Ah, also: one person at a time, I try to introduce some to the sites (and RC and Climate Progress are good counterpoints for thinking folk. (Poor Joe Romm! He’s not as dumb as the rest of them, I think, and I can only hope that he is getting big big big bucks for his intellectual prostitution. He has to go to reunions at MIT, I imagine, and all the old classmates must avert their eyes, or slap his back and move on quickly.)
Oh well. Chuckle. As the old joke goes: “We’ve established what you are. Now, we are just haggling over the price.” …..Lady in Red
Hans Moleman says:
July 22, 2011 at 5:59 pm
“It’s too bad, since it’s a prime example of the things I’ve been railing against today: jumping to conclusions when you have much uncertainty……..”
====================================================
Our friend Hans has taken to writing comedy!!
Hans, read the comments here. The data presented only reinforces personal experience. Here’s the thing. Anyone who has engaged in the climate discussion on the blogosphere, for any considerable length of time has ventured over to RC. As Smokey has pointed out, your criticisms would be valid if these were singular and rare events. The data shows this isn’t the case. But, more damning is the shared experience with many of the commentators here.
Now, contrast that with your experience here. You may not like the opinions here, you may not like the information presented, but your voice is heard. You are allowed to state your piece, even when you violate stated site policy!!! While I’m sure none is expected, you owe Anthony some gratitude. For he has given you great latitude which is much more than would be offered by the people you so earnestly are defending. He has been much more gracious than I would have been. I would have allowed your posts only for purposes of ridicule and scorn. Imagine, someone embracing censorship while
purportingpretending to be an advocate of science.Hans, you serve us well. My thanks.
James Sexton
I too, am of the opinion that RC has been the tipping point, away from AGW to a more sceptical position. In a quest to explore the intricacies of Climate Science I visited many “science blogs” read books, observed comments, and also the demonstrated knowledge of those that frequented the sites.
When the climate tide turned, I was interested to see and hear what many scientists had to say on the topic of why and how, they came to question the consensus and science. The almost universal theme for change came about when they used or tried to use their own expert knowledge (as a scientist) to correct a scientific fact on RC! The treatment, the scorn, the denial of a right of reply or, even to question why, exposed RC as both a megaphone for their own views, and little to do with the advancement of knowledge or science.
My observations of that site tends to confirm that view, though I did feel they made an effort at one time to allow more discourse, but the attack dog (mad dog?) attitude of some of their prized regulars put me off the place permanently. So I guess they have in their own way contributed to a closer scrutiny of the science of climate, by scientists!
So they have been useful if only for that!
Actually I’m an IT professional who has been in the business far longer than the WWW has existed, and currently host lots of WordPress sites and no, I’m not looking for more business as I have too much to do already. I have been involved in data mining and business intelligence for years, have seen thousands of pages of web log reports, and have analyzed what they mean till I’m bored to death with it. There are and are not things you can infer from URLs, changes to URLs, and sequences of URLs. Particularly databased URLs such as WP provides. The database that runs behind the web server code is frequently on a server different than the one the web pages are served from. In good environments there are parallel systems for reliability and performance. In many cases it is needed to do hardware refreshes or software refreshes, or to migrate date to a different technology. It also happens that data merges need to happen across servers, or it may be that it is needed to segment sections of a web site to less expensive hardware. Any one of these maintenance operations can affect the URL that is generated by the database behind the web server running WP.
More over, it is not possible for me or anyone on the outside to know if the change is related to a maintenance activity or to intentional, and perhaps intentionally misleading human activity. The uncertainty is very high without knowing more about how and where the databases are for all the WordPress segments.
I host WP pages and actually have three blogs of my own that are built up of three different WP instances and databases, but so tightly integrated nobody would know. Or care. It would be silly for anyone to presume they can use URLs from those pages forensically. The web server logs would be required as well as the URL list to establish that fact.
None of this considers site policy that can and will impact web content and history, including purged content history. And we have seen in a recent post here that policy may be applied arbitrarily, depending upon who is affected by that policy.
To your point about having been shut out – yes, in fact I have been summarily booted from Tamino’s site every time I’ve posted there. I do that as a kind of sport, actually, because some people are just more pleasant when they are very annoyed and I am responsible for it. The posts I create are reasonable, of course, but definitely not what he is interested in including among his disinformation.
So I think RC is guilty of many of the things suggested in the OP here, but that the chosen proof falls short of being proof. And while I can think what I wish, without proof I can conclude nothing. Climate is like that, too.
dp,
You say you’re an IT professional with vast experience in all the relevant areas (OK, what’s your name and where have you been employed?), yet you are effectively arguing that one might have to alter the contents of a WP database — actually the primary key of the (main) posts table — when migrating across hardware or software (you mean database software, presumably). Remind me not to use your company for webhosting.
You also suggest that RealClimate might be composed of several different WordPress installations, all cunningly hidden behind webserver URL rewrites, or something. Are you suggesting, for instance, that all the posts from 2008 onwards might be hosted on a totally different WordPress installation? Well, I didn’t think of that, but now that you mention it, why wouldn’t you want to do that? It’s just too boring to add posts to one’s existing WP installation like a normal person. Imagine the joy of having three different login pages for staff, etc.!
However, I’ve scanned the entire site using Xenu, and on the basis of that I am confident that there is no WordPress-defying weirdness going on behind the scenes, and that there are no pages with comments on that can’t be accessed using the URLs http://…/?p=x, where x=1 to 8092.
If pages have been deleted then this would have some effect on the stats, though there would have to be a lot of deleted pages all replete with comments to have any significant effect. I have found no internal links on RC that point to any pages now deleted, except the following:
http://www.realclimate.org/index.php/archives/2005/01/draft-do-not-post/
which can reasonably be discounted, I think. This doesn’t mean that no pages have been deleted, just that I can’t find internal evidence of any.
For reference, there are 14 URLs which appear to be deleted pages (i.e., they give 404 errors), but in fact they have merely been renamed:
http://www.realclimate.org/index.php/archives/2004/12/non-significant-trend-in-cosmic-galactic-rays-and-climate-change/
http://www.realclimate.org/index.php/archives/2006/06/national-%20academies-synthesis-report/
http://www.realclimate.org/index.php/archives/2006/06/noaa-apres-moi-le-deluge/
http://www.realclimate.org/index.php/archives/2006/10/con-All%E8gre-ma-non-troppo/
http://www.realclimate.org/index.php/archives/2007/10/cosmic-rays-don%E2%80%99t-die-so-easily/
http://www.realclimate.org/index.php/archives/2007/10/cosmic-rays-don%e2%80%99t-die-so-easily/
http://www.realclimate.org/index.php/archives/2007/12/les-chevaliers-de-l%E2%80%99ordre-de-la-terre-plate-part-ii-courtillots-geomagnetic-excursion/
http://www.realclimate.org/index.php/archives/2007/12/les-chevaliers-de-l%e2%80%99ordre-de-la-terre-plate-part-ii-courtillots-geomagnetic-excursion/
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%E2%80%93dispatch-2/
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%E2%80%93dispatch-3/:
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%E2%80%93dispatch-6/:
http://www.realclimate.org/index.php/archives/2007/12/live-almost-from-agu%e2%80%93dispatch-3/:
http://www.realclimate.org/index.php/archives/2010/08/expert-credibility-in-climate-change-%E2%80%93-responses-to-comments/
So far as anecdotal evidence goes, RealClimate does not (so far as I am aware) have a record of deleting posts. Comments, but not posts. I would imagine a number of people here check out RC on a regular basis: have any posts been deleted in the last month, for instance? Or in every month that I’ve plotted on the graph? No, whilst it’s a reasonable comment to make, I don’t think post deletions are really problematic. A quick estimate suggests about 5 deleted posts — if full of comments — would equate to about 1% of total deletions. In general, website owners really don’t like doing that sort of thing, and as I say I don’t see any actual evidence that RealClimate has done so.
The article link to the Alexa stats now returns:
And no graphs.
De-fund the NASA GISS now! Then magically Gavin Schmidt is gone, then in 2012 when Joe Bastardi says el-nino will be gone and the conservatives take office-kicking out all the commies in the highest levels of OUR government then we can expose the fraud.
Forming my own query gave full stats and graphs. Some very interesting stuff. Currently, about 50% of WUTW visitors have just 1 pageview, but it now surpasses 90% for RC.
And here is an fascinatiing list of rank by country:
New Zealand 839
Australia 1,811
Finland 2,747
Canada 4,434
Dominican
Republic 5,019
United States 7,055
Denmark 7,284
UK 9,238
Japan 12,613
Sweden 14,981
Spain 22,432
Germany 26,076
Italy 29,908
Netherlands 42,046
Pakistan 43,645
India 100,056
Note the absence of China. But what about Taiwan, etc?
Interesting that it is much higher in Canada than the US.
Typoz: “a fascinating” not “an fascinatiing”.
Ian Rons says:
bla bla bla
I said URLs are not enough evidence. I’ve said merging data across systems can produce surprises. Mixing data across time and systems can produce conflicts. A blog db is not like a billing db where consistency is worth the expense to preserve it. I’ve said we don’t know enough to say with certainty, based on URL information, what the truth is. And you don’t, either.
I’ve also said I agree with the conclusions of the OP but don’t agree the evidence is adequate to prove it. And I’m right about that.
Ian – lets avoid future ad homs and discuss what we know and infer what is realistic, and then see what scenarios are possible. And to be honest, this is tedious and has nothing what so ever to do with science.
Real Climate appears to be a WordPress blog hosted by Web Faction – a hosting company. We learn this simply using the DNS lookup tool:
So we see that realclimate.org shares an ip (at this time) with web83.webfaction.com. What can we learn about webfaction.com? How about using whois:
We can use traceroute and other tools to attempt to locate the physical site but that is particularly important.
And what do we see when we visit http://www.webfaction.com/ ?
A rather typical hosting service. Now we know there is a hosting service, and a web farm. We can’t see what the database is hosted on or how many data base servers they make available. We can infer the hosts use a common methodology to improve reliability and performance including BigIP, clustered back end systems, virtual machines to keep costs down. We can infer that to improve their bottom line they offer maintenance, backup, and OS patching, and other infrastructure services. Like me they may provide first line support when the WordPress user krumples the installation. Like me they may handle migration of data from MySQL 3.x to 5.x and deal with all the attendant data dictionary impacts something like that may involve. Like me they may prefer to do this pro-actively on a schedule so they don’t have to do it as an out of sequence restoration because the end user hasn’t the skill or interest. It makes our AR sing and it is a good all around business strategy.
We may also infer that the end user accepts good service and a reduced price if unimportant metrics become modified in the process.
But we have no way to know the truth behind what we infer – we know they are plausible scenarios. And here’s another plausible scenario – the customer requested, or directly performed the operation of fudging dates to create a false impression.
Many of us suspect they are guilty of fudging data. It would be nice if we had a higher level of certainty.
dp,
Differences between MySQL <4/4.0 and 4.1/5.xxx are irrelevant, and it's totally spurious to raise such issues. FUD.