NSIDC fixes their Arctic Sea Ice graphing problem

As I mentioned yesterday, NSIDC had an oops moment, but with the help of skeptical blogs, was notified of the problem and responded timely and appropriately. They posted this update today:

Update, April 19, 2012: The nine-day trailing average climatology on the daily data graph has been changed to a five-day trailing average, to be consistent with the five-day trailing average for the daily data.

I verified their correction for the climatology was accurate with a new overlay, combining the unaffected graph NSIDC’s Dr. Julienne Stroeve sent me from their internal server storage Tuesday night with the corrected one published on the web today:

http://nsidc.org/data/seaice_index/images/daily_images/N_stddev_timeseries.png

The extent data and the climatology now match, whereas yesterday they did not. The x-axis offset is to be expected given that we are comparing graphs with the temporal data offset due to a trailing average they implemented.

This is what the same comparison looked like yesterday, for graphs made on the same day:

So, problem solved.

Unfortunately, somebody jumped to a conclusion and has already had to issue a correction.

April19: NSIDC graph still appears wrong

now reads:

Correction : April 19 – NSIDC Graph Now Lines Up With April 16

The maxim “haste makes waste” seems appropriate.

The way NSIDC’s Dr. Walt Meier and Dr. Julienne Stroeve handled this should be an example to other agencies that don’t bother to even respond to skeptics.

0 0 votes
Article Rating
86 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
April 19, 2012 9:20 am

NSIDC should take the opportunity to switch to center point averaging. Given the changes this week, switch to center weight immediately.

Ian Hoder
April 19, 2012 9:21 am

Sorry to be completely skeptical on this, but do you really believe they would have corrected their “error” unless it wasn’t pointed out to them by skeptics? 9 times out of 10 (a number I pulled out of my arse but I bet is close) the “adjustments” favour the CAGW theory.
REPLY: If not by us, eventually it would have been caught. In this particular case, I see no nefarious motives, just simple miscommunications. The graphs match, the wiggle problem is solved, and if the extent line data is reflecting a situation where it will cross the median line, we’ll see it. – Anthony

Anything is possible
April 19, 2012 9:23 am

Posted this on the previous ice blog, but I think it bears repeating now it has been superceded….
For those of you calling for NSIDC to use a 30-year average : I took the opportunity to raise this issue on Steve’s blog yesterday, and Julienne responded that they were planning to do just that later in the year.
Let’s hope that implementation goes more smoothly than this one…….

Steven Kopits
April 19, 2012 9:25 am

That standard deviation calculation is way, way wrong. You cannot go from 2 std down back to near average in two months. This would be like going from the bottom of the NBA to second best team in the two months. Doesn’t happen. I don’t know how they calculated that std, but I can tell you with simple visual inspection that they did the math wrong.

Scottish Sceptic
April 19, 2012 9:29 am

Ok, I have to admit it, they’ve done a good job. Well done.

Rhys Jaggar
April 19, 2012 9:32 am

Out of interest, does anyone know the longest period of sea ice volume being continually within the statistically ‘normal’ range since the Millennium? If it’s right now, maybe that’s an indication that a slow but steady recovery may be underway?

April 19, 2012 9:35 am

Which internet gossip blog reports the exact history and status of Watts/Goddard relationship?

GlynnMhor
April 19, 2012 9:38 am

It’s still going to be wrong to use what they call a ‘trailing average’, since the output of the averaging operator (1,1,1,1,1) is going to be artificially shifted ahead by half of the operator length.
An operator outputs at its centre point, not at its ends.

Ed_B
April 19, 2012 9:40 am

Its not ice, it is our knowledge of CO2 that is the issue. From Tallblokes workshop:

Sorry to dissapoint everyone, but it looks like we can all go home. We did not cause the big rise in CO2.

Editor
April 19, 2012 9:52 am

Steven Kopits says:
April 19, 2012 at 9:25 am

That standard deviation calculation is way, way wrong. You cannot go from 2 std down back to near average in two months. This would be like going from the bottom of the NBA to second best team in the two months. Doesn’t happen. I don’t know how they calculated that std, but I can tell you with simple visual inspection that they did the math wrong.

I assume they computed the the std the conventional way taking all the extents for a particular day and doing the math on it. There’s no indication the data is at all useful for short term limits as you suggest in you NBA example. To continue that analogy, a team can go from the bottom of the NBA to second best in one year.
What you are expecting would be an interesting display, perhaps you can work on generating it.

Casper
April 19, 2012 9:54 am

Nice shot, guys!
I can imagine what kind of tricks are applied in the statistical data analysis. Just keep an eye on those cheaters!

John F. Hultquist
April 19, 2012 9:59 am

Works for me. Thanks Anthony and Drs. Walt and Julianne.
And a “climatology” update is coming this year. Great.

April 19, 2012 10:00 am

Dear Anthony,
Excuse me, but I see plenty of nefariousness in the graph.
Why else would they hide the actual data and instead post phony “confidence” intervals around a known data set. There is no estimating needed. The facts (the data) are known with 100% “confidence”. I cannot see any reason for the gray zone except to artificially (and nefariously) shrink the actual range of the actual measurements.
Are some years “outliers”? Why does the government (aka Big Brother) need to hide the actual facts? Why vanilla-ize what really happened?
Please forgive me if I hurt somebody’s feelings because I want the actual real data, that I paid through my taxes to have collected. The NSIDC is not the CIA. National security will not be jeopardized. Are the big bad gummit scientists having a pout because some lowlife upstart dared to ask for the ACTUAL DATA!!!
And what’s with the 2007 line? Why is that year the ONLY year graphed? Is there something special about 2007, something so ALARMING such that the paternalistic head-patter gummit mucky mucks need me to see that year and none of the others?
Nefarious, nefarious, nefarious.
Pretty obvious to me that Big Brother is putting out propaganda masquerading as science. Not welcome. Dirty tricksters should be shown the door.

April 19, 2012 10:01 am

Reblogged this on Climate Ponderings.

kbray in california
April 19, 2012 10:03 am

I will have a profound sense of relief when those lines cross.
To me it represents the Arctic ice cycle as an oscillation as opposed to a straight line to a hot hell.
It plunges an ice dagger into the heart of CAGW zombieism.
I will pop a cork when it happens.
The ’12 ice appears way more robust that ’07.
A good sign.
However, I am dismayed by the recent flurry of Global Warming “propaganda” that has been
released recently. It seems like “the last gasp” when something is dying. I hope so.
I can’t wait to tell my warmy friends when “Arctic ice is above normal”.
Regardless of which way it goes, the truth will out.
…but I’m rootin’ for the ice.

April 19, 2012 10:08 am

Anthony, I don’t know if you or Steve McKintyre have noticed it, but it is very unlikely indeed that the standard deviation for sea ice is correct for late March. It necks in to (eyeballed) half of its extent in early March or mid-April even though the mean is smooth across the two domains. Either there is some extremely odd climate factor that causes a crossover between two essentially independent mechanisms that are responsible for the larger variance before or after or, more reasonably, the standard deviation should be at its fattest all the way across this region. This, by the way, would keep late March in 2007 from being a 5 or 6 sigma event (which is absurd). If one smooths the sd the way it almost certainly should be, March 2007 would be a less than 3 sigma, maybe a 1 in 50 to 1 in a 100 shot (consistent with the whole year).
Oh, and if one adds in the 2007 data, there is no way in hell that the sd is right for any moderate number of sample years. An outlier like this simply bumps the sd to accommodate for few samples (and suggests if anything else a possible bimodal distribution so that the central limit theorem hasn’t yet kicked in, although outliers do happen).
So either the data for late March is “miraculously” tightly clustered compared to expectations from the time immediately before or after or there is something funky about either the data or the way it is being processed into a sigma here. At the very least, something that demands explanation in terms of climate observations over this period, where the ice melts (a tiny bit) consistently where before and after it is melting and growing much more variably.
As they say, “What’s Up With That?”_tm…;-)
rgb

Rob L
April 19, 2012 10:25 am

NSIDC have done a good job here; listening to criticism, and fixing the problem quickly and openly in a way that we don’t see from the usual AGW suspects. As such I don’t think they deserve any rebuke.
Moving to a 30 year baseline later in the year will certainly screw up the narrative as the confidence intervals will widen and the averages will drop so that relatively low sea ice levels won’t appear anywhere near as impressive as they do with the old baseline.

April 19, 2012 10:26 am

Sure right … oops, we got caught.

kim
April 19, 2012 10:35 am

Sumpin’ funky about the last two months.
H/t Bill Illis.
========

April 19, 2012 10:36 am

Regarding the 1979-2000 average, NSIDC uses it for both Arctic and Antarctic graphs. If 1982-2011 were used, current Arctic extent would probably be above the average, but Antarctic extent would probably be less above the average. I doubt it would make any difference to True Believers and, by definition, objective observers don’t care.
That NSIDC uses 2007 for Arctic reference and 2011 for Antarctic reference is inconsistent. On the one hand, 2007 is a revered year for warmistas but, on the other hand, 2007 has made every succeeding year look like one of cooling.

April 19, 2012 10:45 am

“Ed_B says: April 19, 2012 at 9:40 am
Its not ice, it is our knowledge of CO2 that is the issue. From Tallblokes workshop
http://www.youtube.com/watch?feature=player_embedded&v=YrI03ts–9I
Thanks for the HT – All I can say is WOW. Everyone needs to watch this presentation.

Sammus
April 19, 2012 10:50 am

A government agency tried to improve their product, had some internal misscomuniction and growing pains. They then openly admitted this and promptly made corrections. How sad that this is considered so remarkable.

Stacey
April 19, 2012 10:50 am

The maxim “haste makes waste” seems appropriate.
Dear Anthony
Unlike you to be churlish, at least Mr Goddard corrected his error and he did spot the change in the first place so credit where credit is due. 🙂
Another maxim by a great American “The only man who never makes mistakes is the man who never does anything”
Thank you for all your great work and keeping this blog alive and lively.
S
REPLY: In the original post I gave him full credit for spotting it first, but he also published first, and used a title that said “data tampering” suggesting malfeasance where there was simply a boneheaded mistake brought on by poor communications at NSIDC. He published that without having asked NSIDC a single question, that responsibility fell to me. Steve then compounded his first error by rushing out an opinion on the correction which he had to retract today.
So yes, in this case, “haste makes waste” – Anthony

Kelvin Vaughan
April 19, 2012 11:01 am

The big question now is will it? wont it?

April 19, 2012 11:08 am

Do NOT attribute to arravice or malevolence that which may more simply explained by ignorance, stupidity or laziness. Anonomous…
However, I’d add, “Hey, a calculational mistake not immediately caught…as there are a limited number of reviewers!” (Which I think is the case here.)

John Blake
April 19, 2012 11:10 am

Kudos for simply displaying base-data with integrity? In climate-scientology terms, O Wow. Now we’d like to see RGBatduke’s cogent concerns addressed.

April 19, 2012 11:13 am

Funny how this ‘opps’ problem just happened to result in the line going back farther from the mean… I don’t think this just happened on its own.

April 19, 2012 11:14 am

Honest to Pete, why have “confidence” intervals at all? Why not reveal the actual data?
All you experts out there, maybe you can explain to me why the data should be hidden and an arbitrary gray zone plotted instead?
Does it give you a warm fuzzy to NOT see the data? Do you prefer pablum to steak?

April 19, 2012 11:21 am

Hi all,
Yes, just to confirm, we are looking to switch to a 30-year climatology. This is more complicated because it doesn’t only affect our daily time series plot, but all of our images. Also, the timing of it is a bit uncertain. We haven’t changed the climo before, so not sure of any issues that may be in the code. Also, we want to do it at non-sensitive time (e.g., near the minimum). We’re tentatively planning to use 1981-2010. We now have final (non NRT) data through 2010, and this is consistent with the NOAA normals. If folks have other suggestions, we’d be happy to hear them and take them under consideration – post a reply here.
The other thing in the previous post that got lot of attention were my brief comments on trust in climate science. I can’t expound on that here, but maybe I’ll try to expand on that sometime when I have time.
Thanks.
Walt Meier
NSIDC

Peter
April 19, 2012 11:23 am

Might it be remarkable that in a single winter the ice extent from rather low to median.

Ian E
April 19, 2012 11:27 am

Sure points up the need for transparency! [Are you there, Mann?]

D. J. Hawkins
April 19, 2012 11:32 am

Mike Dubrasich says:
April 19, 2012 at 10:00 am
Dear Anthony,
Excuse me, but I see plenty of nefariousness in the graph.

Pretty obvious to me that Big Brother is putting out propaganda masquerading as science. Not welcome. Dirty tricksters should be shown the door.

This is the sort of rant that is truly pointless. You can go to NSIDC’s web site and download the data for yourself, for crying out loud. Do anything you want with it, plot it any way you please.
“Hide” the actual data? If you tried to plot every year on the graph it would be a jumble of lines. Try it yourself, if you’re not too lazy.
Why 2007? Seriously, you don’t know why? Try year of minimum extent.
Paranoid, paranoid, paranoid.

Scottish Sceptic
April 19, 2012 11:35 am

Walt Meier says: April 19, 2012 at 11:21 am
Hi all,
Yes, just to confirm, we are looking to switch to a 30-year climatology.

May I suggest that you just go ahead as soon as you are ready and run the old and the new in tandem.
Alternatively, you could cheat … just be nice to us sceptics and we’ll all roll over on our backs and purr.
But seriously. Just do the right thing, do it well and whilst we might grumble, … we don’t bite.

Tenuk
April 19, 2012 11:37 am

In the mean-time, Arctic-roos now updated, showing 2012 above the 1979-2006 average for both extent and area!!!
http://arctic-roos.org/observations/satellite-data/sea-ice/ice-area-and-extent-in-arctic. It will be interesting to see if the higher NSIDC average (based 1979 -2000) also crosses in the next few days. It is puzzling how temperature up in the Arctic is all over the place…
http://ocean.dmi.dk/arctic/meant80n.uk.php
Perhaps our quiet sun is starting to have an effect?

April 19, 2012 11:37 am

I cannot help but get a chuckle out of how the ‘pigbook’ came
out today talking about government waste. It is from the
‘Citizens Against Government Waste.” CAGW

Latitude
April 19, 2012 11:45 am

…call me cynical
Julienne’s very first post….on Steve’s blog…..she said he’s right…obviously they didn’t know about it……Steve caught it……
===========================================
Julienne Stroeve says:
April 18, 2012 at 7:41 pm
Steve is correct that the switching to a trailing mean resulted in a mis-match in the timing between the climatology and the daily extent since different averaging periods were used (this date mis-match wasn’t a problem in the centered means). We’re working to fix the problem. However, Steve should note that switching to a 5-day trailing climatology mean actually puts the current extent farther away from the climatology.

Peter
April 19, 2012 11:49 am

Steven Kopits: “That standard deviation calculation is way, way wrong. You cannot go from 2 std down back to near average in two months. This would be like going from the bottom of the NBA to second best team in the two months. Doesn’t happen. I don’t know how they calculated that std, but I can tell you with simple visual inspection that they did the math wrong.”
I think the 2 standard deviations is measured on the 1979-2000 data. So essentially the chart is showing that the sea ice extent for 2007 and 2012 has been outside of (i.e., less than) 2 standard deviations of the mean for 1979-2000. You can’t say by “simple visual inspection” of a chart that “they did the math wrong”. If you think you are smarter than the scientists at NSIDC, then take the 1979-2000 data, calculate a mean and standard deviations for each day and plot it on a chart. But then, I’m sure you won’t be doing that. It’s much easier for you to simply dismiss data that tells you something you don’t want to believe. It’s human nature to just say “that can’t be true” rather than face the facts. It’s also a form of cowardice.

April 19, 2012 11:55 am

NSIDC fixes their Arctic Sea Ice graphing…

Blue Sky
April 19, 2012 11:58 am

Thanks Mr Meier for taking time out of your schedule to explain the “oops” and to confirm future plans.

EW-3
April 19, 2012 11:58 am

What year was it that the Gore and his mob said we’d have an ice free north pole ?
2013 sounds familiar. Anyone know ?

Latitude
April 19, 2012 12:02 pm

Walt Meier says:
April 19, 2012 at 11:21 am
Also, the timing of it is a bit uncertain. We haven’t changed the climo before, so not sure of any issues that may be in the code. Also, we want to do it at non-sensitive time (e.g., near the minimum)
==============================
You have a mean streak………….rotfl
….save yourself, do it sometime in the middle

NZ Willy
April 19, 2012 12:04 pm

SG may have spotted it 1st, but I claim credit for having spotted it 0th in my screed 3 days prior on “Tips & Notes” on the software-model-caused interrelationship of Arctic & Antarctic ice volumes — of which this is an example, as Antarctic anomalies have hopped around in reverse to Arctic during this episode. 🙂

See - owe to Rich
April 19, 2012 12:24 pm

Mike Dubrasich seems to be relatively new, and a little angry, but he is asking for information, so someone should be kind (and patronizing, impossible to avoid). He said at
April 19, 2012 at 11:14 am
“Honest to Pete, why have “confidence” intervals at all? Why not reveal the actual data?
All you experts out there, maybe you can explain to me why the data should be hidden and an arbitrary gray zone plotted instead?”
Well, the gray zone simply surrounds a line in the middle which _is_ the mean data, so look at that and ignore the gray if you want. The gray is somewhat interesting because when the curve stays below it for a long time, as has happened a lot recently, it says (without explanation) “something out of the ordinary is going on here”. That something may be an oceanic warm phase, or it may be CO2, or it may be the sun, and we’re trying to work it all out.
Regarding Mike’s earlier question about 2007, it is displayed because it was the biggest Arctic melt in the recent record, so worth comparing with. Warmists think we are going to go under that curve a lot in the next decade, and sceptics say “really? give me a 3-1 bet on it and I’ll take you on”.
I hope this helps, Rich.

Joseph Bastardi
April 19, 2012 12:27 pm

about the retraction.. let me say this. He forced the issue and was right about everything leading up to it, so we judge him by the last mistake? Cmon. The fact is that NSIDC does a great job, but because of some of the other things going on in this debate one has to be skeptical. I take them at their word, but I have made mistakes with them too. it doesnt help that if my memory serves me correct , the idea of the ice cap death spiral came out of there several years ago, and one of the years mentioned was by 2012. So you cant blame people that look at this for being a bit hesitant at times to trust but no verify.
The fact that the people at the top DID NOT EVEN KNOW WHEN THIS WOULD BE IMPLEMENTED is troubling. If they waited till mid summer it would seem better. I would like to trust, but again given all that is going on in other areas, why now, and why without any message before hand that this was in the works.
As for the retraction, I believe the person has shown he is a good and honorable man. That he is judged by his one mistake, when we see people getting away with things one after another, seems kind of odd to me.
Personally after an honest mistake I made over confusion I stay away from it. Joe Romm went out of his mind, so I watch the Norsex site for my ice fix.
Trust, but verify

Mycroft
April 19, 2012 12:31 pm

If the ice keeps recovering like this, i can see Anthony adding Gore’s and others iceless Arctic to the Climate Fails Page …… LOL

April 19, 2012 12:41 pm

“If you tried to plot every year on the graph it would be a jumble of lines.”
Yes … its hard to tell a narrative if people know how different the data is from year to year.
Better if the “public” think the average line is actual data.

April 19, 2012 12:52 pm

Statistical calculation quibbles aside, I think that the way that this “Ooops” has been handled by Dr. Walt Meier, Dr. Julianne Stroeve and Anthony is an object lesson in how such matters ought to be resolved.
It is called ‘talking with each other’. When lines of communication are not obliterated by political or other considerations simple truths emerge. I, for one, am most cynical when it comes to matters concerning ‘official’ statistics relating to matters climatological but the manner, style and speed in which Dr. Meier responded to this deserves a good deal of respect. Dr. Walt Meier and Dr. Julianne Stroeve, thank you so very much for being so good at your public relations. Would that so many of your colleagues were as open and responsive.

Mike Smith
April 19, 2012 12:56 pm

Like making sausages, the process of research and presenting the results frequently involves more than a little messiness. This was clearly no exception. But, at the end of the day, the system worked perfectly. Some inevitable errors and ambiguities were resolved to the satisfaction of all involved, in an incredibly short period of time. Heck, it was a day or so!
Congratulations to Anthony, and Drs Meier and Stroeve. Keep up the wonderful work!

philincalifornia
April 19, 2012 12:59 pm

EW-3 says:
April 19, 2012 at 11:58 am
What year was it that the Gore and his mob said we’d have an ice free north pole ?
2013 sounds familiar. Anyone know ?
++++++++++++++++++++
The one I remember clearly (although the video has been deleted at Youtube) was at a visit to a German museum in the second week of December, 2008 – 5 years to an ice-free Arctic, so yes 2013.
Also, anyone know what’s going on with the NORSEX plot? It hasn’t been updated since the 5th. I’ve seen this question asked at least a couple of times but haven’t seen an answer. If I missed it, could someone point me to the answer please.

April 19, 2012 1:02 pm

Al Gore`s models predicted there was a 75% chance there would be an ice-free Arctic by the summer of 2014.

philincalifornia
April 19, 2012 1:16 pm

Aaaaah, I see that NORSEX has now been updated to today, and it is above the 1979 – 2006 average.

Eric Webb
April 19, 2012 1:20 pm

I think the NSIDC freaked out seeing the Arctic Sea Ice approaching average, so they figured they could make an adjustment under the table, but that failed. I guarantee you no such change would be made if sea ice was near 2007 levels, I find it more than mere coincidence how sea ice almost crosses the average 1979-2000 line, but they suddenly decide to make changes to the average, thus making the current data appear warmer. I don’t understand why they wont use the other 10 years of available data, through 2010, even though it has large swings, I don’t see why not, but wait, it would make the new data appear cooler and less alarming, so they don’t want to do that do they? 🙂

Eric Webb
April 19, 2012 1:24 pm

Anyone see the NORSEX ice site which, unlike the NSIDC uses a 1979-2006 average, and currently it is almost an entire standard deviation ABOVE AVERAGE?

Mac the Knife
April 19, 2012 1:27 pm

Looks to me like this ‘peer review’ thingy works… as long as the ‘peers’ include sharp eyed citizens with no conflict of interest or hidden agenda!
We are inside the ‘1 sigma’ range for the 1979 – 2000 average for ice coverage at this time of year. Situation ‘normal’ in the Arctic ocean. How is the ice coverage in the Bering Sea doing? Is it still at a ‘max condiiton’? Is there any historical measure of daily/weekly/monthly ice coverage for that area?

Jeef
April 19, 2012 2:13 pm

Guys asking about confidence intervals – there’s a gap in satellite cover that has to be infilled using a computer model. Next question please!

April 19, 2012 2:29 pm

They should be proud of real data- no averages please. Whatever qualities it contains should be plotted out.

See - owe to Rich
April 19, 2012 2:33 pm

Before you all get excited about Arctic ice returning to normal, bear in mind that 2 years ago it was also at a good level in April but fairly low by September. The summer melt will obviously be led from the Western side, and the NE passage might open again, but the NW passage must be more doubtful.
Rich.

Michael D Smith
April 19, 2012 2:46 pm

Anthony, her name is Julienne…
REPLY: Yep, I know another person who spells it the way I wrote it. Fixed – A

stan stendera
April 19, 2012 2:59 pm

NSIDC deserves the standing and fluttering applause they are getting from my birdfeeder. THEY PROVIDED THE RAW DATA!!!! Would that the ‘team’ would??????

Dave
April 19, 2012 3:21 pm

I think people wouldn’t have jumped on NSIDC as hard for the initial error were it not for the continued hysterical ramblings of a certain Mark Serreze. I understand he’s clearly passionate about what he does but he does immense damage to the credibility of the organisation he works for so that in the end simple mistakes are seen as something a little more sinister.
Its just a shame that his shenanigans counter the excellent work Walt does in engaging on this site.

April 19, 2012 3:28 pm

You can’t say by “simple visual inspection” of a chart that “they did the math wrong”.
Actually, sometimes you can. Suppose you have 30 samples. 29 of them are anything you like — say 0 (putting the mean of the distribution very close to 0). The 30th is 1. The standard deviation is then around \sqrt{\frac{1}{29}} \approx 0.2 The 1 in this case is roughly 5 standard deviations above the mean, and is pretty close to being the maximum possible value it can theoretically have, since we’ve got zero variance on 29 out of 30 samples (and since obviously everything I just stated is scale independent in a certain sense). To put it another way, you cannot observe a data point in a sample of size N=30 that is ten standard deviations away from the mean — there are sanity limits that scale with N. With only 30 samples, how can you state with any confidence at all that the probability of obtaining 1 from the central limit theorem is 0.00000001 or whatever erf(10) is? That’s nonsense.
Given a roughly 30 year base, by visual inspection the standard deviation around the end of March is around 5 standard deviations, which means either that a bloody miracle happened (given the variability on both sides and the size of the mean, it means that the sea ice extent has been almost exactly the same for every year but 2007 in late March) or it means that there is a simple error somewhere that somebody should look for, and look hard, until they are certain that a miracle happened. And then they should try to understand the miracle. In the meantime, the graph above is moderately unbelievable, and would be so even if the data is exactly the way it has to be to make it correct.
Perhaps you might study some math and statistics before concluding that simple inspection cannot often pick up errors. Actually, it is one of the best, and simplest ways of doing so — when a curve doesn’t make sense somewhere, it is often time to go back to the data and/or algorithm used to produce the curve.
This is the case even if the curve turns out to be true and correct! “Unusual” results are often precisely the points were interesting discoveries are made.
rgb

April 19, 2012 3:32 pm

Guys asking about confidence intervals – there’s a gap in satellite cover that has to be infilled using a computer model. Next question please!
This may well be true, but then the obvious point is that they’ve set the sd wrong in the infilling model. It needs to interpolate the sd on the adjacent sides of the end of March, not shrink it to “unbelievable”.
rgb

Bruce of Newcastle
April 19, 2012 4:07 pm

Well done to NSIDC!
But I hope they understand that on the back of very recent radical changes to sea level datasets (which make it look worse – in flagrant disagreement with tide gauge datasets) and to the temperature record (HadCRUT 4 again further cools the past) we are just slightly suspicious.
When all of this data is transparent, maybe we could go back to a small degree of trust. But trust is sorely lacking at the moment.
Perhaps NSIDC might, for example, let a contract to Climate Audit to audit climate data? Corporations are required to be audited. Could it be so bad to have external auditors catch these issues rather than force unpaid scientists and engineers such as Steve McIntyre to do so?

Eric Webb
April 19, 2012 6:21 pm

After this incident like this, I think it’s safe to say we’ve all lost a substantial amount of trust in NSIDC, seems like they’re certainly on the pro AGW side of things.

Bill Illis
April 19, 2012 6:25 pm

Here is my estimate of the daily sea ice extent going back to 1972 (when there actually was satellite images available and the NASA Team developed daily sea ice extent estimates from these using an algorithm which is mostly the same as that used today).
I’m matching up 4 different datasets here (including the Cryosphere Today ice area data) so there might be some problems but having worked with this for a few years now, I think it is pretty accurate.
2012 (in Red) is closing in on the 1972 to 2011 average ice extent.
And I’ve also included a line (Purple) showing the type of change that would be required to have an ice-free Arctic on a typical minimum day. Huge changes are required in both the March sea ice extent maximum and then the amount that melts each day compared to the average to get us anywhere near to an ice-free Arctic. Just so you know, 2011 followed the average melt per day almost to the “T” so having a melt rate 20% higher than average each day from March to September is probably impossible. The ice-free Arctic people are relying on “non-physical” models.
http://img818.imageshack.us/img818/9855/dailyseiapril182012.png

April 19, 2012 6:45 pm

It was good of Dr. Meier to participate here, but if he wants to know why so many view government-sponsored climate science with cynicism, he has only to connect the dots between Hansen, Gleick, Serreze of his own organization, and today’s hateful rant by Zwick in Forbes magazine.

April 19, 2012 7:11 pm

This has been a pleasant interlude. People are actually behaving in a more or less civil manner. Congratulations to all concerned.
We need both the people who stir us up and those who calm us down. Complacancy is a vice, but so is hysteria. We humans are not the most stable items on earth, and I actually think we are at our best when we move between two views. That’s why we have two eyes. I myself become very compacent, when I go to bed, and each morning the alarm stirs me up. I feel the better for having been complacent for a while, but to stay in bed usually means I am ill.
By the way, concerning: “Tenuk says:
April 19, 2012 at 11:37 am……
…….It is puzzling how temperature up in the Arctic is all over the place…
http://ocean.dmi.dk/arctic/meant80n.uk.php …..”
Hopefully I can tempt Joe Bastardi to add to my reflections on the swings in arctic temperatures. As I recall it was he who suggested that, when it gets windy up there, the temperature rises. When it is calm, the temperature falls.
When I thought about this, it occurred to me that higher temperatures likely create more ice. Why? Because of the wind. It is not due to the wind-chill, but rather because winds shove the ice around, and not only piles up pressure ridges, but create open water in the form of “leads.” These swiftly freeze over, because even when the wind has temperatures ten degrees above normal, we are still talking thirty degrees below zero. This open water swiftly becomes several feet thick, whereas if the same area was not exposed and was instead insulated by as much as nine feet of new ice, only an inch or two would have been added on the very bottom.
In other words, though a windy winter may show temperatures ten degrees above normal, it is constantly piling up pressure ridges and exposing leads of open water which freeze over, and far more ice will be produced.
Which brings me to the touchy subject of “ice volume.” You are likely to hear a lot of Alarmists bring up the subject of “volume,” when both “extent” and “area” fail to alarm anyone.
As far as I can tell, “volume” is not determined by counting pixels, but rather by a bouncing radar beam from a satellite. I have my doubts about whether radar can pick up pressure ridges, especially the smaller ones, which you can’t see even in the best enlargements from outer space, for a small one may only be three feet tall and ten feet across, though it extends for miles. From outer space it looks thinner than a spider-mite’s web.
However most of an iceburg is under water. If a pressure ridge sticks up three feet it sticks down twenty-seven feet. If it towers, (like the really big pressure ridges,) twenty feet, then it has roots that go down one hundred eighty feet.
These pressure ridges are a jumble of ice mashed together, amounting to a large volume in a small area. They are made up of lots of small plates, ice as thin as a few inches thick, but as much as nine feet thick, only glued together by snow and whatever else holds chunks together when it is thirty below. When it warms they fall apart and revert to a bunch of plates floating in water, and this messes with our minds. Why?
Well, let us suppose an area a mile wide got crunched together into a big pressure ridge thirty yards across. When that ridge melts, it will revert to a mass of small chunks a mile wide. That simply does not compute, to our common sense. How can an area thirty yards across abruptly become a mile wide? Especially due to MELTING!
I think that is what happened with the ice, in 2006. In the noontime-dark of mid-winter, 2006’s extent was of the lowest in recent years, however it included so many pressure ridges that, in September, 2006’s extent was of the highest, in recent years.
I am betting a nickel, (and no more,) that the same thing has occurred this past winter. As the ice melts it will turn out that the “temperature up in the Arctic is all over the place” you mentioned meant a lot of wind was piling up lots of pressure ridges, and as these pressure ridges fall aprt they will make mincemeat of our sensible calculations, based on the sensible idea ice is of the same thickness all over the place.
In any case, roughly two thirds will have melted by next September, and September’s extent will have roughly tripled by next March.
I’m willing to bt a whole dollar on that.

Editor
April 19, 2012 7:59 pm

Walt Meier says: April 19, 2012 at 11:21 am
If folks have other suggestions, we’d be happy to hear them and take them under consideration – post a reply here.
Hello Walt
What I would really like to see from NSIDC is Global Sea Ice Extent charts. I have looked around your site and not been able to find them, i.e.:
http://nsidc.org/data/seaice_index/
http://nsidc.org/data/seaice_index/images/
Cryosphere Today offers a Global Sea Ice Area chart;
http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/global.daily.ice.area.withtrend.jpg
which offers helpful perspective and generates significant public interest and traffic.
I decided to plot the Global Sea Ice Extent March Monthly Anomalies to see for myself, so I grabbed the data from here;
ftp://sidads.colorado.edu/DATASETS/NOAA/G02135/Mar/N_03_area.txt
ftp://sidads.colorado.edu/DATASETS/NOAA/G02135/Mar/S_03_area.txt
and plotted the Northern and Southern Sea Ice Extent March Anomalies;
http://i42.tinypic.com/2wocarb.png
http://i39.tinypic.com/2uo2gyv.png
in order to validate the data and methodology. Here are NSIDC’s originals for comparative reference:
http://nsidc.org/data/seaice_index/images/n_plot_hires.png
http://nsidc.org/data/seaice_index/images/s_plot_hires.png
I then merged the two data sets and created a chart for Global Sea Ice Extent March Anomalies:
http://i40.tinypic.com/20f21wo.png
Based on my calculations, the slopes of the trendlines are as follows:
Northern Sea Ice Extent Anomalies March 2012: Slope = -2.5% per decade
Southern Sea Ice Extent Anomalies March 2012: Slope = 3.2% per decade
Global Sea Ice Extent Anomalies March 2012: Slope = -1.4% per decade
Would it be possible to have Global Sea Ice Extent charts added to NSIDC’s site?
Thanks for your consideration
JTF

April 19, 2012 11:45 pm

D. J. Hawkins says:You can go to NSIDC’s web site and download the data for yourself, for crying out loud. Do anything you want with it, plot it any way you please.
Nope. Not there. Besides the obvious point: if you want me to do NSIDC’s job, then please send me their paychecks.
If you tried to plot every year on the graph it would be a jumble of lines. Try it yourself, if you’re not too lazy.
Oh I see, they don’t know how to make a graph. It’s incompetence, not deliberate scientific fraud. Well, DJ, please see Bill Illis’ graph below. See, it CAN BE DONE, just not by government functionaries.
See – owe to Rich says:…we’re trying to work it all out. … 2007 is displayed because it was the biggest Arctic melt in the recent record, so worth comparing with.
Well, Rich, “we” aren’t going to work anything out without the data, with phony gray areas, and with emphasis on one extreme year. Where is the maximum year? Is that of no interest? How does the (truncated, missing) data as displayed tell “us” (or anybody) anything about long-term change?
Bill Illis says:Here is my estimate of the daily sea ice extent going back to 1972 (when there actually was satellite images available and the NASA Team developed daily sea ice extent estimates from these using an algorithm which is mostly the same as that used today).
I’m matching up 4 different datasets here (including the Cryosphere Today ice area data)…

http://img818.imageshack.us/img818/9855/dailyseiapril182012.png
There you go. It CAN be done, and it HAS been done, not by NSIDC but by Bill Illis, Skeptic Extraodinaire.
Dear Walt,
Please send Bill your paycheck.

D. J. Hawkins
April 20, 2012 12:52 am

rgbatduke
I’m not sure I understand your issue regarding the SD’s. I see the baseline is 1979-2000, not the entire record to date. 2007 isn’t even in this data set so obviously won’t “broaden” the SD boundaries. If there has been a general decline in the ice extent over the instrumental record, wouldn’t one expect that later years would eventually fall beyond the 2 sigma boundary if the base period remains the same? It would be a question of “when” not “if”, depending only on the rate of decline.

AlexS
April 20, 2012 1:19 am

“Walt Meier says: April 19, 2012 at 11:21 am
If folks have other suggestions, we’d be happy to hear them and take them under consideration – post a reply here.”
Very well. This is one:
The future data needs also to be compiled in the current method for decades to verify the values with new method. It would be important to verify the error levels between methodology changes.

John from CA
April 20, 2012 4:15 am

AlexS says:
April 20, 2012 at 1:19 am
===========
I agree, noting the error rates and basis of observation (15% as opposed to 30%) is helpful.
It would also be great if they would pick-up some of the better features from The Cryosphere Today.
Latest Regional Sea Ice Coverage and Anomalies for 14 areas is a great approach. It shows exactly where sea ice isn’t forming. Showing the Complete Annual Satellite Record is also a great way to objectively communicate the sea ice record. Both are available on The Cryosphere Today.
Northern Hemisphere Sea Ice Area — Complete Annual Satellite Record
http://arctic.atmos.uiuc.edu/cryosphere/arctic.sea.ice.interactive.html
Latest Regional Sea Ice Coverage and Anomalies
http://arctic.atmos.uiuc.edu/cryosphere/

P. Solar
April 20, 2012 5:07 am

Now all we need is for NCDC to correct their spurious Mannian padded “9 pt binomial filter” : http://www.ncdc.noaa.gov/temp-and-precip/time-series/index.php?parameter=tmp&month=3&year=2012&filter=1&state=110&div=0
Like Met office Hadley , they will probably only feel the need to correct this misrepresentation when it produces a deceptive down swing.
I emailed them 3 days ago, no sign of even an acknowledgment so far.
This is a classic case of padding of the filter window to get the filtered result to go beyond the end of the filter. It’s pretty clear here that they are repeating the final March temperature over fully half the window , thus producing an unprecedented rise that does not even exist.

scotchman1
April 20, 2012 5:26 am

To Rhys Jaggar, 19 April, 9.32 am: I have been monitoring this graph for years. The Arctic ice briefly moved into the 2 std region for a few days some years ago but the recent trending is without precedent in recent years. We may be seeing a step change just like the one that lowered ice levels and caused all the GW hysteria. Could we now all draw this new situation to the attention of journalists on magazines like Time who still write about the Arctic melting?

John from CA
April 20, 2012 5:41 am

Dr. Meier,
Thanks for the opportunity to comment about the NSIDC site.
The addition of the Ice Lights section is great and most of the articles are reasonably balanced. I have one constructive criticism that extends to all sites and the IPCC reports. Observations made about aspects of the climate system fail to properly reflect the climate system as a whole. The top-line approach fails to properly reflect the current state of scientific understanding related to climate forecasting.
Arctic Sea Ice or to be more accurate Northern Hemisphere Sea Ice Area and Extent is influenced by a number of interrelated forces; Salinity, Oceanic Oscillations, Atmospheric Oscillations and Wind Direction, Milankovitch cycles, Solar activity during a portion of the year, etc.
In my opinion, the NSIDC Arctic sea ice 101 page needs to be rewritten to properly reflect Sea Ice changes in relation to the current scientific understanding of the climate system with an emphasis on changes in relation to Natural cycles. A tall request, I’ve never seen it done properly on any site, but it would be a remarkable tool for education.
Thanks for considering the request.
Icelights: Your Burning Questions About Ice & Climate
Arctic sea ice 101
Excerpts:
Arctic sea ice is declining at an increasing rate in all months of the year, with a stronger decline in summer months. Researchers who study climate and sea ice expect that at some point, the Arctic Ocean will lose its ice cover completely in late summer. A variety of evidence suggests that Arctic sea ice is declining because of climate warming resulting from increased greenhouse gases in the atmosphere. Because greenhouse gases persist and are expected to increase, scientists see no reversal to the downward trend in ice extent.
Despite year-to-year variations, satellite data show a decline of more than 11 percent per decade in September ice extent, since the satellite record started in 1979. Before 1979, the data are less comprehensive, but shipping records and other evidence show that the ice extent has been in a continued state of decline for at least the last hundred years. Climate models predicted that Arctic sea ice loss would accompany warming temperatures in the Arctic. But the ice loss has happened faster than any models predicted, and researchers now expect that the Arctic Ocean will be ice-free in summer well before the end of the century.

thelastdemocrat
April 20, 2012 6:27 am


The current and average lines are getting pretty darn close…Will NSIDC let the streams cross? That would violate GhostBusters Rule #1: Don’t Let the Streams Cross!

G. Karst
April 20, 2012 8:33 am

Increasing ice sheets are a catastrophe for the Earth’s biomass. Please refrain from referring to growing ice as a recovery. Ice sheets are massive and any decreasing trend is a recovery from our ice age infliction. Increasing ice is a relapse to our hypothermia condition… afflicting us since we came down from the trees. One day it could be fatal. GK

April 20, 2012 10:17 am

I’m not sure I understand your issue regarding the SD’s. I see the baseline is 1979-2000, not the entire record to date. 2007 isn’t even in this data set so obviously won’t “broaden” the SD boundaries. If there has been a general decline in the ice extent over the instrumental record, wouldn’t one expect that later years would eventually fall beyond the 2 sigma boundary if the base period remains the same? It would be a question of “when” not “if”, depending only on the rate of decline.
Sure, but this is simply cheating with statistics, especially since 1979-2000 is too few years for the standard deviation to mean much anyway. However, my objection can be formulated in perfectly reasonable terms even excluding 2007 from the data. 2007 is smoothly similar to the mean curve representing the average of the baseline years (where one has to ask — why bother plotting 2007 in particular on this curve when one has the data in hand and where adding in 2007 data would both increase the sd and increase the reliability of the sd and mean — surely a desirable and necessary thing to do for it to have any meaning in terms of the central limit theorem, presuming that one hasn’t cherrypicked a start date and end date so that it emphasizes some point one wishes to make). Given the noise, it trends along an average of around 3 sds off of the mean for the early part of the cycle, then pulls in to be 2 sds off by summer. Except for late March, where it continues perfectly smoothly to nearly parallel the mean, but the standard deviation for the mean necks in to make the March result 5 to 6 standard deviations off.
This necking in is a clear signal that the standard deviation is nearly meaningless in the curve up above. To some extent this is to be expected with such a small baseline, but this is where excluding the subsequent data is criminal. Anybody who thinks that arctic sea ice can in any way be represented by a 20 year baseline average raise your hand. What, no takers?
Of course not. We know that its natural variability on a century timescale is far larger than this.

D. J. Hawkins
April 20, 2012 10:44 am

Mike Dubrasich says:
April 19, 2012 at 11:45 pm
D. J. Hawkins says:You can go to NSIDC’s web site and download the data for yourself, for crying out loud. Do anything you want with it, plot it any way you please.
Nope. Not there. Besides the obvious point: if you want me to do NSIDC’s job, then please send me their paychecks.
If you tried to plot every year on the graph it would be a jumble of lines. Try it yourself, if you’re not too lazy.
Oh I see, they don’t know how to make a graph. It’s incompetence, not deliberate scientific fraud. Well, DJ, please see Bill Illis’ graph below. See, it CAN BE DONE, just not by government functionaries…

You really ought to make at least the minimal effort to look for the data before making a claim. Start at http://nsidc.org/data/docs/noaa/g02135_seaice_index/ and note the following, since I’m sure you won’t make the effort: “Data are stored in ASCII text, Portable Network Graphics (PNG), Keyhole Markup Language (KML), and Geospatial Vector Data format and are available from November 1978 to present via FTP or the Sea Ice Index Web site.”
“They” don’t know how to make a graph? The graph you requested, of all the years, would be incomprehensible due to data overlap at any reasonable scale. Note that Bill’s doesn’t even do what you demanded and it’s pretty busy visually. You want a graph? Get off your duff and make it yourself.
You are the kind of lazy twit that gives skeptics a bad name.

Curt
April 20, 2012 1:12 pm

Dear Dr. Meier:
You say, “If folks have other suggestions, we’d be happy to hear them and take them under consideration – post a reply here.”
I deal with these types of averaging/filtering calculation all the time in my professional capacity in feedback control systems (and I am writing this while taking a break from dealing with one such issue), and I believe you are taking a big step backwards in switching from centered to trailing averages. Especially for after-the-fact reporting, a centered average/filtered value is far superior to a trailing average, both for trends and for measurement noise. If the value you report for today, April 20, is a 5-day trailing average, you are really reporting the best estimate for April 18, not for today. This has all sorts of potential for confusion down the line.
I realize that if you want to report “real-time” data every day with an N-point centered average, the last (N-1)/2 points are necessarily tentative and subject to change. But it would be far better to show these with dotted lines and/or in a different color to emphasize their tentativeness. People will be using these plots years from now, and they will be using shifted data if you keep using the trailing average. This is simply not correct.

Daniel
April 20, 2012 1:34 pm

kbray in california says:
It plunges an ice dagger into the heart of CAGW zombieism.
I hate to bring this up, but a dagger to the heart won’t kill a zombie…
…now, back to the discussion.

Svend Ferdinandsen
April 20, 2012 5:27 pm

I wonder why the alarm was not sounded when the “change” appeared. Credit to NSIDC that they investigated fast, gave an explanation and corrected the fail.
It seems anyway that the ice extent is close to mean value, and the alarming messages of the Arctic ice are very silent at the moment. Wonder why?

Steve from Rockwood
April 20, 2012 5:35 pm

Walt Meier says:
April 19, 2012 at 11:21 am

[snip]
If folks have other suggestions, we’d be happy to hear them and take them under consideration – post a reply here.

Walt, averages mean nothing when we don’t know the >30 year variation in Arctic ice extent as the records are too short. Why not show the years with the highest and lowest summer extent along with the current year, indicate the years on the graph and let people argue from there?
Thanks for posting and thanks to WUWT for showing class in handling an honest mistake.

NZ Willy
April 20, 2012 10:36 pm

I agree with Curt, centered was better. A different color could be used for the 2 most recent days to show that they are in wiggle-land. Future users will not like the 2-days-offset graphs. Someday, it will be changed back — an issue has been created, not solved.

Pamela Gray
April 21, 2012 5:31 am

Regardless of the statistical changes, what is far more interesting is just how long the blue line will go towards black “normal” during this melt season. I see three scenarios here. 1. We have a drifting satellite, 2. We have ice up the yin yang, or 3. We have a very hard wind spreading the ice WAAYYYY out thin and it will flush-melt like an ice cream bar in the hands of a toddler.

Robin Edwards
April 21, 2012 9:01 am

Can’t find a trace of my posting of yesterday 🙁 Was I too numerical or technical? Or perhaps I was nit-picking! I’d like to know so that I can adjust things in future. Anyway, a pity really. I tried to put the case for being explicit about what people really understand by “standard deviation”, and to ask for a full definition of what is understood by the “SD” phrase in the current context.
Robin