New Climate Audit Server Migration Completed

If some of you have noticed I’ve been a bit absent from WUWT it is because I’ve been working the last two weeks with Mr. Pete and JohnA to get the climateaudit.org server migrated to a new home.

We wanted off a single server box and onto wordpress.com where they have a distributed cloud computing system, gobs of optical pipe bandwidth, automated backup, automated code updates, and 24/7 support. I wish to personally thank Nick and Lloyd of WordPress VIP support who went above an beyond to manage the huge migration of files that spanned all the way back to 2004. WordPress.com has a great staff and a great product.

When Climategate broke, so did the climateaudit single box server. It was more traffic than any one server could bear. The datapipe to the box at our CoLo was also not sufficient to handle the load. I setup camirror.wordpress.com for Steve to use in the interim while the CA IT team figured out how to make this migration happen. There were lots of hitches. Some are still unresolved but the bulk of it is there.

Today, about 5PM PST 90% of the migration has been completed. Steve McIntyre made this announcement:

We have a new and hopefully final home. It has been a very big and complicated job moving climateaudit.org with its nearly 2000 threads, over 200,000 comments, plus images, data, scripts to wordpress.com, which has large enough facilities to prevent the chronic server problems that plagued climateaudit as a standalone server. camirror.wordpress.com was set up to deal with overthrow while our new “home” was being prepared and has been transferred as well.

What substantially complicated the move was our desire to preserve the very large number of links to existing climateaudit images and threads (which I’m told create a subterranean draw and which we didn’t want to disconnect.) Anthony, Mr Pete and John A all contributed to the move, which was accomplished during a period when I was otherwise occupied so to speak.  I’m very grateful to them.  WordPress.com went several extra yards to accomodate the move and I’m very grateful to their staff as well.  We think the old addresses are all pointing to the new addresses – let us know if you have any problems.

A couple of caveats before you rush off to view it:

1) There may still be an occasional broken link or missing file, we’ll get to them all as time permits. Don’t overload Steve with comments about these.

2) DNS propogation may not have completed yet for your portion of the web. So if you visit www.climateaudit.org and you get nothing don’t freak out. The IP address/DNS server changes will eventually propagate to you.

3) Can’t wait? You can try forcing a DNS refresh if you own a PC with Windows by doing the following.

Open a command prompt and type the command in yellow below and press enter.

If you have a Mac, I have no idea how to do this, maybe somebody who owns one can leave a comment.

So within 24 hours or so, almost everybody should be able to get to www.climateaudit.org

Also, beyond the above, and maybe cycling your router/modem, I can’t suggest how to solve DNS propagation delays at your location, so just wait it out if none of that works.

We all look forward to Steve’s next big thing. Now CA can handle most anything we throw at it and Steve can drive CA “car” without having to carry a set of tools and a small army of mechanics jammed in the trunk.

Advertisements

70 thoughts on “New Climate Audit Server Migration Completed

  1. Cheers, Steve!
    Nice to see the new site…the formatting looks a little wonky on my screen, I think because I have non-default font settings (I have to…I’m visually impaired), but otherwise…everything seems to be working just fine.

  2. You guys did a great job. DNS propagation is still sporadic at my ISP, and yes I did flush my resolver cache, before I even read your instructions.

  3. Excellent work to all of you and thanks for keeping the subterranean views of stalactites and stalagmites of climate debate alive and kicking! [:)]
    Oh, Steve you were great on CNN by the way! Looked like you had more to say at the end, what was it that you wanted to say when they said by and cut off your feed?

  4. Kudos to you, Anthony, for all your generous efforts. Climate Audit’s new bandwidth capabilities will serve the public well in the post-climategate era. Who knows what sudden climate change controversies will explode in 2010! I look forward to being reliably informed by WUWT & CA in the years to come. Cheers!

  5. To flush the DNS cache on Mac OS X Leopard, open Terminal and type:
    dscacheutil -flushcache
    Thanks for all who helped get CA back up and running.

  6. Thanx for the clue of the DNSflush.
    Didn’t make it fly, but probably will soon.
    At least i have something to try besides mindlessly trying the same old thing over and over.
    any idea about the actual IP address?
    TIA
    TL

  7. I am unable to connect to the new Climate Audit site. Neither of the links work for me in FireFox or IE. I have tried all evening.

  8. Thank you for your hard work, Anthony. As an “investor” in the last Climate Audit upgrade, perhaps you can tell me and the other contributors what will happen to that hardware going forward?
    REPLY: That hardware becomes a new repository for code, data, scripts, and misc files that can’t be stored on wordpress.com It will be setup as data.climateaudit.org or climateaudit.info at the CoLo – Anthony

  9. Actually, I was just thinking how the tendency of Climateaudit to “break down” and “be overrun” with hits was a great marketing tool. Those episodes did three things:
    Created a sense of drama that elevated the importance of whatever was being posted. Other blogs would say, “CA is overloaded,” creating more of a stampede to get over there and see what all the fuss was about.
    Created scarcity – people who checked and couldn’t get in would keep coming back. Something that’s hard to get seems more valuable.
    Established McIntyre’s credentials as an everyday citizen auditor, unfunded by interests that could afford a big fancy server.
    For whatever that’s worth.

  10. Climateaudit.org just vanished from the internet. Gone. I get a Comcast page with “Sorry, we can’t find “www.climateaudit.org”. We suggest that you check the spelling of the web address or search above.” I take it this is a hiccup?

  11. A little tip but if you have your DNS folks cut down the TTL (time to live) on your record ahead of time if you know you’re going to be moving something, you can cut down significantly on that 24 hours to propogate
    Congrats on the move to all involved!

  12. I doubt this was free. Can we help with the cost?
    REPLY: Steve’s tip jar isn’t working just yet, perhaps later. Thank you for the offer. – Anthony

  13. Heads up …The old climateaudit.com does not redirect to the new one.
    The new one doesn’t let me past the man page, anything I click on goes to nothing.

    REPLY:
    heads up ….it’s climateaudit.org

  14. sorry for the ot post, but did I just read correctly Al Gore saying the climaegate email is all over 10 years old? Three times?
    “A: To paraphrase Shakespeare, it’s sound and fury signifying nothing. I haven’t read all the e-mails, but the most recent one is more than 10 years old.”
    He has a shockingly bad grip on all this situation, as well as the science, I’m actually quite surprised. He isn’t doing his cause any favors spewing nonsense like this. (The north polar ice cap was the same size for 3 million years? Wow.)
    http://www.slate.com/id/2237789/

  15. I connected nicely.
    Well done. I know that kind of platform transition can be incredibly painful and detailed work. Doing it while under machine crushing load and staying live? Priceless…
    Now that you’re back, take a look at the current weather. Wonderful things are happening. De-icing planes at San Jose, snow in Sacramento, Chicago intraday temps dropping like a rock, blizzard warnings in N. Arizona, lake effect snow in N.Y. (get ready for the “usual” Buffalo story 😉 Seattle Wa. did not get above freezing as the high today, and some places are having HIGHS that are “below zero F”. The Weather Channel is calling it “amazing arctic cold” and there was a report that IOWA (the whole thing) might be shut down soon from the expected cold. Oh, but I-5 has finally reopened…
    AND on 810 KGO here in the SF area I heard the News Readers discussing the bitter cold in Napa (where they had been for some reason) and doing a poll among the staff to find if ANYONE had ever experienced colder. (I *think* they said 18F low at night, but I wasn’t listening very closely..) One guy said he thought there might have been ONE day back in the ’70s that was colder, but he wasn’t sure.
    So it ought to be a “Weatherman’s Candy Store” 😎
    Have fun, do another “Weather is not Climate” and stock up on Firewood!

  16. Oh, and while it’s too late to be any help now:
    When doing a transition like this, call up your ISP and ask that they push a new Time To Live for your DNS entry (the old one) and set it to something like 1 hour. This means any cache will update after no more than one hour. This is done on unit of your prior expiration in advance. ( So if TTL was 10 days, you do it 11 days before the expected move).
    After the move, you push a new, longer, TTL with the new DNS entry.
    That way your total DNS cache “issue” time is the short TTL interval. I’ve done as short as 8 min TTL (though the ISP complained… we did it in two steps which made them happier. Push 1 day 10 in advance, then push 8 minutes the day before the move).
    I know, “Now you tell me”…
    If you ever want a free experienced P.M. on this kind of stuff, I’m often available.
    Oh, and Steve, the CNN piece was priceless. That little smile when you knew you “had him” on a point was enough for me to do a little jig… Folks catch those small things and know who’s got the truth and who’s faking it.

  17. Great job by all. My ISP is still a little out of it and sometimes can’t find CA, but a few hits on refresh and they get it right. In no time everything should be easier.
    As for E.M.Smith (23:05:47), here in Buffalo we’re facing a mix of snow, sleet, and freezing rain overnight, turning to rain by morning and throughout the day. Some storm. By sunset tomorrow, it’ll change back to snow and, depending on where you live here, we’ll get anywhere from 2 to 8 inches of global warming. But they’re not too sure of where it’ll all fall – depends on the wind direction and speed (too much wind doesn’t allow for an orderly formation of lake effect snow bands).
    As for the hype of snow in Buffalo, well, it’s a bit oversold. There’s also lots of the stuff to go around – Rochester, Syracuse, and especially Watertown (to the lee of Lake Ontario – they really get hit there more than Buffalo).

  18. The IP is a ’round robin’ for the cluster and it routes you to a given site image based on the name used when you connect. No name? You get the wordpress image:

    [chiefio@tubularbells analysis]$ host climateaudit.org
    climateaudit.org. has address 76.74.255.123
    climateaudit.org. has address 72.233.2.58
    climateaudit.org. has address 72.233.2.59
    climateaudit.org. has address 74.200.243.251
    climateaudit.org. has address 74.200.243.253
    climateaudit.org. has address 76.74.254.123
    

    But if you put the IP in your browser, you get the WORDPRESS image. It does, however, have a search function so that might give you a climateaudit connection as wordpress ought to know that they are the target IP… or maybe not. Worth a shot.
    You could also just, for a while, point to a different DNS server. Mine are:
    [chiefio@tubularbells analysis]$ cat /etc/resolv.conf
    nameserver 4.2.2.4
    nameserver 76.14.0.8
    nameserver 69.111.95.106
    And I get the new IP so they’ve got it.
    And there are lots of open access DNS servers. See:
    http://www.dnsserverlist.org/
    for details. But there ought to be lots of them with the updated info. You are not bound to use the DNS server of your ISP. I don’t (they are slower than the ones I chose from the online speed testing tool.) Then again, I have a caching server in my living room (the list here) that all the other stuff points at, so most of the time most of my DNS lookups don’t leave the house. Yeah, it’s “a geek thing”…

  19. The Copenhagen Climate Challenge is good but the letter could have been improved with a little editing, I think; for instance:
    … [those who advocate AGW] have the obligation to convincingly demonstrate that recent climate change is not of mostly natural origin and, if we do nothing, catastrophic change will ensue. To date, this they have utterly failed to do so.
    A better version, I suggest, would be:

    … who have the obligation to demonstrate convincingly that recent climate change is not mostly of natural origin and that, if we do nothing, catastrophic change will ensue. They have utterly failed to do this.

    Picky, I know. By the way, what do you call someone who always takes things literally? A thief.

  20. debreuil (22:13:02)
    Delicious, Al should have looked a bit further!
    Out, out, brief candle!
    Life’s but a walking shadow, a poor player
    That struts and frets his hour upon the stage
    And then is heard no more. It is a tale
    Told by an idiot, full of sound and fury
    Signifying nothing.

  21. As of 12:25 am PT, unable to link to new site, even after DNS cache flush. Tried both CA.org and CA.wordpress.org. Got an IE “Cannot Display page” msg. Will try again tomorrow. Thanks to all of you who work on this.

  22. I’m getting the new .org site OK from the UK – MUCH quicker than before!
    Well done – moving a site from one host to another is never easy. The internet is arguably our best weapon and it needs to be working to maximum effect…
    Take the new http://www.copenhagenclimatechallenge.org/ initiative – last night there were 8 people online – this morning there were 175 people looking at it…

  23. I got in a few hours ago by hitting my toolbar bookmark (climateaudit.org). I did a search and was toodling around some pages when suddenly everything I clicked on popped a 404.
    Been trying off and on since but no go.
    Just tried climateaudit.wordpress.com and my status bar keeps flashing ‘sending’ and ‘waiting’ for climateaudit.org. 😮
    Did something get mixed up? seems like there’s some infinite looping going on?

  24. Seems that the wordpress link is redirecting to the climateaudit.org site which is no longer working.
    Or its being suppressed by Gore 😉

  25. climateaudit.wordpress.com
    http://www.climateaudit.org
    Neither of the above links take me to the new site (nor indeed, anywhere). I get these messages from Google search and Bing
    Safari can’t open the page “http://climateaudit.org/” because Safari can’t find the server “climateaudit.org”
    and
    Sorry! We could not find http://www.climateaudit.org
    Same when I click on the image here.

  26. P.S. Mine is a Mac.
    Suggestions, please?
    [Reply: try restarting. I have a Mac and the link comes up with no problem. ~dbs, moderator.]

  27. Re: Syl (01:59:21)
    It’s working for me. If you’ve done the DNS cache clearing thing, it may still be your ISP’s caches having a stale record. DNS records don’t propogate! They’re pulled rather than pushed, which is why it’s important to lower the TTL as E.M.Smith says. If your ISP’s DNS resolvers still have the old IP details from the original DNS, they’ll stay there till the TTL expires.
    If you’re using old XP, that has/had a really annoying feature described here-
    http://support.microsoft.com/kb/318803

    If you do not want negative responses to be cached, set the MaxNegativeCacheTtl registry setting to 0.

    Which means if there’s a temporary glitch in DNS, XP will remember it for default 15m and throw you an instant 404 every time you try to reconnect. If you’re comfortable with editing registry, it’s worth setting that to zero as you’ll rarely, if ever want to cache a negative response.

  28. I’ve tried restarting, doesn’t work, and I don’t understand all the other stuff posted. I guess I’ll just wait.

  29. I just saw the User’s Manual for the Vaisala WXT520, a “non-meterological” weather station. It has no moving parts. I wonder if it is time to update all the thousands of stations everywhere. I will be getting a quote for a project I have coming up. I’ll let you know.

  30. Can’t Steve get the old server back online as a mirror until we can all get onto the new one? Sort of reverse of what happened before ?
    Andy

  31. AndyW,
    I’m sure it would take longer to reconfigure it than it will to simply wait until the new site has propagated.

  32. I just pulled up my favorites directory, clicked on CA, and there it was, new WordPress site and all. Thanks for all the dog work, Anthony, and thanks for all the content, Steve. Way to go, guys!

  33. @Wag
    “Established McIntyre’s credentials as an everyday citizen auditor, unfunded by interests that could afford a big fancy server.
    For whatever that’s worth.”
    As an opinion from you worth noting? Nothing at all. The big money is right behind Gore and the alarmists. After all, ETS schemes will make the energy companies a very great deal of profit.

  34. Home page doesn’t work for me, but the individual article pages come up (like the one from link http://climateaudit.org/2009/12/09/8155/ Once on the site, clicking link to home page results in “site unavailable” error again.
    There could be something wrong on the server side with the main index file.
    Russian hackers ????? (Just kidding)

  35. Great work Anthony. I haven’t tried to get on the new Climate Audit site yet – patience everyone – all good things…
    I’ll be visiting a few tip jars over this festive season – if Steve’s isn’t up and running by then, I’ll drop a bit extra in your’s so you can buy him a very well deserved beer.
    2009 has been a good very good year – here’s to 2010.

  36. Wasn’t working for me but I added a manual entry to my hosts file and it’s fine now.
    Under c:\windows\system32\drivers\etc is a hosts file you can open with notepad. If you add an entry exactly as the line below to the bottom of the file and save it, it will temporarily fix the problem.
    76.74.255.123 http://www.climateaudit.org
    Note, this makes your pc bypass DNS for that site, so remember to remove the entry in a few days otherwise if the ip address for the site changes again you will still keep trying to use the above forever.

  37. Looks like a DNS issue.
    It should resolve itself in a day or so.
    It worked on my home PC, but here at work I get the DNS error.

  38. For those of you who are having (or had) problems accessing the new site, you should think about switching DNS providers. Your ISP provides one for free, but they are usually slow and neglected. Accessing a single web page can trigger several DNS queries, so using a slow DNS can be a huge bottleneck for an otherwise fast internet connection. Plus, you can have connectivity issues like reported above.
    I switched to OpenDNS and have never looked back. Google just launched a similar service. There is also UltraDNS, which I know nothing about. They are all free of charge. Here’s a program you can use to test the various alternate services:
    http://lifehacker.com/5420931/namebench-helps-you-find-the-fastest-dns-server-for-your-computer
    Switching isn’t very difficult. If you are using a router at your home (Linksys, Netgear, or such), you can change the setting on that box, and it will use the new alternate DNS for every computer connected to it in your home. I know the instructions and the help forums are very good at OpenDNS. Don’t know about the other alternatives.
    Don’t go changing the DNS at your work computers, though… at least without checking with the IT people. Large corporate networks use DNS to control access to things, and you’ll lose connectivity for all sorts of services (printing, internal websites, etc) if you change those settings.

  39. I just used Cocktail to clear my Internet cache (on my MacBook Pro running Snow Leopard), and no joy getting to ClimateAudit.org. Adding ‘www’ doesn’t help.
    /Mr Lynn

  40. CA server is now broke, or the DNS is updating, but can’t access it.
    REPLY: The server is fine.

  41. [snip – For the last several days you have been thread bombing WUWT with posts duplicated across several threads in the span of a couple of minutes. Yesterday it was the political link. They have been moved to the trash. Stop this behavior or you will be banned – Anthony]

  42. Atomic Hairdryer
    Thanks for info.
    I had already gotten in once with no problems, everything went south after that. But whatever/whereever the glitch was, it seems to be fine now.

  43. Richard Black has just posted a report on the BBC Website http://news.bbc.co.uk/1/hi/sci/tech/8405025.stm
    “UK Met Office warns carbon emissions must peak by 2020”
    As he provides an e-mail address I have just sent him a message with the following contents:
    “Richard Black
    I really do think that Vicky Pope should spend her time going through the HAD/CRU temperature time series to check for “man made global warming” in the actual so-called “enhancements” they and particularly CRU may have added to make the present warmer and/or the past cooler. I am not saying that this has happened. I am simply saying that the “hacked” Fortran code may indicate that such adjustments have been made. Surely you also know about the accusations that NASA/Giss may have been “adjusted”. The various Mann/Briffa hockey sticks have been shown to be, well not perhaps the best scientific efforts to eliminate MWP. There is also an accusation that New Zealand’s temperature records have seen “adjustments”.
    So I – as a non scientist – read all these stories now emanating from COP15 and then look at the various indices on the Internet.
    So we are subject to “run-away warming”. Looking at http://www.drroyspencer.com/latest-global-temperatures/ I will simply claim that there has been no warming this Century. It may be the warmest “on record” (their records) per Met Office and NASA but that is only because they start in 1880 and there is an accusation that efforts have been made to reduce the peak temperatures in the 1930s. Hundreds of “peer reviewed” research lend credence to the claim that it was warmer globally 1000 years ago. The main point is that temperatures have stopped rising this century when CO2 continues to rise.
    So the heat is in the oceans. The Maldives and Tuvalu will disappear within the next couple of decades etc. Apart from Mr Moerner’s assurances to the president of the Maldives http://www.spectator.co.uk/coffeehouse/5595813/why-the-maldives-arent-sinking.thtml there is no evidence of any heat build-op in the oceans per the ARGO site http://www.argo.ucsd.edu/global_change_analysis.html They actually state “Domingues et al (2008) and Levitus et al (2009) have recently estimated the multi-decadal upper ocean heat content using best-known corrections to systematic errors in the fall rate of expendable bathythermographs (Wijffels et al, 2008). For the upper 700m, the increase in heat content was 16 x 1022 J since 1961. This is consistent with the comparison by Roemmich and Gilson (2009) of Argo data with the global temperature time-series of Levitus et al (2005), finding a warming of the 0 – 2000 m ocean by 0.06°C since the (pre-XBT) early 1960’s.”
    Recent information on ocean sea levels also kind of support Mr Moerner’s contention about the safety of the Maldives http://sealevel.colorado.edu/current/sl_noib_global.jpg Despite “runaway global warming” sea levels have stopped rising since 2006. Normally that would signify no hearing of ocean (lack of expansion) and/or no smelting of land based ice in Antarctica and Greenland.
    Well, if you look at http://nsidc.org/data/seaice_index/images/daily_images/S_timeseries.png you can see that at least the sea ice surrounding Antarctica is above the 1979-2000 average. All the talk is about west Antarctica peninsula. They forget about the vast east Antarctica, where the ice is. A couple of years ago during the 2007 summer minimum predictions from the likes of Gore and Hansen indicated that the Arctic summer ice will be gone in five years. They are now talking about 2030. Convenient because their predictions will have been forgotten. Here is the reason why it is now 2030 http://www.ijis.iarc.uaf.edu/en/home/seaice_extent.htm The summer sea ice recovered in 2008 and even more so in 2009 and it is doing fine.
    So this century has been relatively stable temperature wise. There is no heat build-up in the oceans, sea levels have stopped rising and N/S sea ice is doing not badly. Meantime CO2 is continuing to rise http://www.esrl.noaa.gov/gmd/ccgg/trends/
    I am sure that you also know well that their climate computer models have so far struggled to predict reality. So why should they be any better at it now?
    Please understand that people like me are not “deniers”, “flat earthers” and whatever other expressions are hurled at us because we just look at the official data and cannot get it to connect with all the “alarmist” propaganda that is “emitting” from Copenhagen now. Throwing such accusations about makes me suspicious that they lack real arguments. So I – like many others – are still to be convinced. Swear words will not do. We need convincing arguments based on facts.
    You and your colleagues are paid for by people like me and we expect realistic and fact based reporting. Hope you are enjoying Copenhagen. I lived there over 40 years ago. Watch out as there are predictions that cold weather is on its way from Sweden. ”
    It won’t have any effect but at least he should get the message that some of the BBC license payers are tired of the endless stream of “propaganda”. If they could provide real scientific evidence of “man made global warming” I would convert from being a “sceptic” to “believer” in next to no time.

Comments are closed.