In the prior thread I raised a question of why there was a large downward jump in sea ice extent on the graph presented by NSIDC’s Artic Sea Ice News page. The image below was the reason, dozens of people called my attention to it in emails and comments overnight because in the space of a weekend, a million-plus square kilometers of Arctic sea ice went missing. Note the blue line.
Click for larger image
When I checked NSIDC’s web site this morning, about 8:30 AM PST (9:30AM MST Mountain time in Boulder where NSIDC is located) the image was still up. A half hour later it remained. I checked all around the NSIDC web site for any notice, including the links they provide for the data issues.
Learn about update delays, which occasionally occur in near-real-time data. Read about the data.
Finding nothing, and knowing that it was now 10AM in Boulder, which should have been plenty of time to post some sort of notice, I decided to write a quick post about it, which was published at 9:10AM PST (10:10MST) and drove to work.
The corrected image (with the million square kilometers of sea ice restored) appeared on the NSIDC web site just shy of 3 hours later, about noon PST or 1 PM MST.
Click for larger image
About the same time this comment was posted on WUWT by NSIDC’s chief research scientist, Dr. Walt Meier:
Anthony,
We’re looking into it. For the moment, we’ve removed the data from the timeseries plot.
You need to remember that this is near real-time data and there can be data dropouts and bad data due to satellite issues. While the processing is automatic, the QC is partly manual. Thus errors do happen from time to time and one shouldn’t draw any dramatic conclusions from recent data.
I’m not sure why you think things like this are worth blogging about. Data is not perfect, especially near real-time data. That’s not news.
Walt Meier
Research Scientist
NSIDC
ps – FYI, the JAXA data is from a different sensor, so it is not consistent with our data, but it provides a good independent check. If the JAXA data does not show a dramatic change while the NSIDC data does (or vice versa), then it’s likely an issue of missing data or bad data.
First let me say that I have quite a bit of respect for Dr. Meier. He has previously been quite accessible and gracious in providing answers, and even a guest post here. But I was a bit puzzled by his statement “I’m not sure why you think things like this are worth blogging about…. That’s not news“
First let us consider a recent event. The BBC ran really badly researched video report just a couple of days ago where the reporter obviously didn’t know the difference between positive and negative feedbacks in the climate. I wrote about it. The video is now gone. Now I ask this question; if nobody speaks up about these things, would the video still be there misinforming everyone? Probably.
The point I’m making here is that in my experience, most reporters know so little about science that they usually can’t tell the difference between real and erroneous science. Most reporters don’t have that background. I say this from experience, because having worked in TV news for 25 years, I was always the “go to guy” for questions about science and engineering that the reporters couldn’t figure out. And, it wasn’t just at my station that this happened, a meteorologist friend of mine reported the same thing happened to him at his station in the San Francisco bay area. I vividly remember one week he was on vacation and I saw a news report about a plane that crashed that had just minutes before been doing a low level run over the airfield as part of a show. The reporter had video taped the plane’s run, and then used that video to proudly demonstrate “as as you can see, just minutes before the crash, the propellers on the plane were turning very slowly”.
The reporter didn’t understand about how a video camera scanning at 30 frames per second can create a beat frequency that give the impression of slowly turning propellers that were actually running about 3000 RPM., and there was nobody there to tell her otherwise. She made an honest mistake, but her training didn’t even raise a question in her mind.
So when I see something obviously wrong, such as a dramatic drop in sea ice on a graph presented for public consumption, I think about a reporter (print, web, or video -take your pick) somewhere in the world who may be assigned to do a story about sea ice today and does an Internet search, landing on NSDIC’s web site and then concluding in the story “and as you can see in this graph, Arctic sea ice has gone through a dramatic drop just in the last few days, losing over a million square kilometers”.
Thinking about Walt’s statement, “ That’s not news” if the NSIDC graph had been picked up by a major media outlet today, would it be news then?
I understand about automation, about data dropouts, and about processing errors. I run 50 servers myself and produce all sorts of automated graphics output, some of which you can see in the right sidebar. These are used by TV stations, cable channels, and radio/newspaper outlets in the USA for web and on-air. While those graphics are there on WUWT for my readers, I also have an ulterior motive in quality control. Because I can keep an eye on the output when I’m blogging. When data is presented for public consumption, in a venue where 24 hour news is the norm, you can’t simply let computers post things for public consumption without regular quality control checking. The more eyes the better.
At the very least, a note next to NSIDC”s Learn about update delays, about how glitches in satellite data or processing might generate an erroneous result in might be in order. And also for consideration, adding a date/time stamp to the image so it can be properly referenced in the context of time. This is standard operating procedure in many places, why not at NSIDC?
NSIDC and other organizations need to realize that the interest in what they produce has been huge as of late. In NSIDC’s case, they have been promoted from relative obscurity to front page news by the recent unfortunate statements of an NSIDC employee, Dr. Mark Serreze, to the media, that have received wide coverage.
As commenter “just want truth” wrote in the previous thread on NSIDC:
Last year Mark Serreze, of the NSIDC (you may know him), said North Pole ice could be gone in the summer of 2008. He said then “The set-up for this summer is disturbing”. This, of course, was broadcast in all news outlets around the world. Everyone on both sides of the global warming debate was watching Arctic ice totals last summer to see what would really happen. You may have noticed hits on the NSIDC web site were high last summer.
Now Mark Serreze is saying North Pole ice is in a “death spiral”.
You can be certain that Arctic ice data will be scrutinized because of Al Gore and Mark Serreze. A line has been drawn by both. Both have placed it clearly on the radar screen. This is why NSIDC data is worth blogging about–especially since Mark Serreze is employed at the NSIDC.
Mark Serreze 2008 North Pole ice free :
http://abcnews.go.com/Technology/Story?id=4728737&page=1
and
http://www.youtube.com/watch?v=S6e3e4VzwJI
Mark Serreze North Pole ice in “death spiral” :
http://www.youtube.com/watch?v=HW9lX8evwIw
and
http://www.nypost.com/seven/08282008/news/worldnews/arctic_ice_in_death_spiral_126443.htm
Given the sort of attention that has been heaped on NSIDC, I think blogging about errors that have gone unnoticed and uncorrected by 10AM on a Monday morning isn’t an unreasonable thing to do.
I also think that reining in loose cannons that can do some terrible damage in the media is a good way to maintain scientific credibility for an organization, especially when predictions like “ice free north pole” don’t come true.
I have no quarrel with Dr. Meier, as I’ve said he’s been the utmost professional in my dealings with him. But I do have quarrel with an organization that allows such claims to be broadcast, all the while producing a data source that is now regularly scrutinized by the public and the media for the slighest changes. It’s a slippery slope.
pft (15:27:46)
In science as well as in the economy you need to take into account the cost of correcting an error in relation to its impact. It usually is not worth your time to hunt down an error, which if corrected will change your result in a non-significant digit only. Results don’t have to be perfect, they just need to be good enough to let you reliably test your hypothesis.
Now this site/link appears down completely
http://nsidc.com/arcticseaicenews/
Well it seems to me that the people who have “the data” think it is ok; how much checking is enough? So they go with it, and it turns out there is an error in the data. It is not unusual that those who are most familiar with the data, are the last to recognize the error.
Somebody eventually spots the error; in this case Anthony did, and he alerted his visitors (here); some of whom may already have puzzled over the cause of the ice “anomaly”. Eventually Dr Meier found out about the error, and it got fixed.
What’s wrong with this process? Not a damn thing that I can perceive; the system worked, and the error got corrected maybe sooner than it otherwise might have.
And Dr Meier doesn’t have egg all over his face; the error got fixed; Wonderful !
George
I would have thought Dr. Meier’s response would be heartfelt thanks that you picked up an error before a news organisation did. The subsequent correction process would have been very messy and embarassing.
Looks like:
Haste + Haste = Waste
Jumping on the published data was the correct answer and I didn’t read anything in your original post that was combative. Although, in haste you didn’t leave an easy out as already mentioned in prior posts.
I am sure it is tough for both, you and the NSIDC to provide the real time information we crave. I do find it interesting that a quick look at the timeline indicates the NSIDC may be watching your blog with a keaner eye than thier own data output.
Daniel
Silly Quote:
“Your keeping us alive now, you forget about that one, and you keep them coming in, your doing well.” – We Where Soldiers
John H says:
Well, it is back up.
What would be really interesting is side-by-side views of the NSIDC shot at that site and the Canadian version. After all, the stupid Canadians must be wrong. American technology is never wrong.
George E. Smith (17:11:52) :
Well it seems to me that the people who have “the data” think it is ok; how much checking is enough? So they go with it, and it turns out there is an error in the data. It is not unusual that those who are most familiar with the data, are the last to recognize the error.
Somebody eventually spots the error; in this case Anthony did, and he alerted his visitors (here); some of whom may already have puzzled over the cause of the ice “anomaly”. Eventually Dr Meier found out about the error, and it got fixed.
What’s wrong with this process? Not a damn thing that I can perceive; the system worked, and the error got corrected maybe sooner than it otherwise might have.
And Dr Meier doesn’t have egg all over his face; the error got fixed; Wonderful !
*******************************************
One thing wrong with “this” process is that Anthony isn’t part of the process, unless he draws a NSIDC paycheck. Since I doubt he is, that is one thing wrong with the process.
We have no idea really whether this was an error or what the error was, or whether it would have been corrected *at any time in the future* or whether it will get corrected if it were an error, and we don’t know that any error has been “fixed”. Unless I have missed some news, Walt Meier has not provided any real information to date as to what did happen or what might happen in the future as a result of this incident. He left more questions than answers, and said they would “look into it”, but as of this post their latest graph has not been updated and is several days short of being current, and their ice image is from 16 Feb.
If it takes someone from outside the agency to bring attention to “errors”, I wonder if they would really ever get “fixed”. Seems to me the best time to verify and QC data is as soon as practicable after receiving data, not waiting a month or six to do some “rigorous” QC as the NSIDC website claims. But all Walt said was that they do “some manual QC”, but didn’t mention when this happens in the process of getting, processing and archiving data, or what QC even means.
I think Walt and the NSIDC have egg all over their faces.
Funny thing about American technology. Near the time we had all the problems with hanging chads in Florida, the Canadians also had an election. They simply made an “X” in a box beside their candidate… no problems. KISS…
as i think about this further, perhaps dr meier you can explain how you calculate 15% sea ice extent…what remote sensing techniques are being used and what are their margins of error? specifically, why is it 15%?
Let’s hope Mark Serreze is being reamed by Dr. Meier for being so sloppy if indeed the data used to produce the error was Serreze’s doing.
REPLY: Given that there seems to be be no change from the famous “ice free north pole” Serreze zinger, doubtful. -Anthony
Anthony:
Have you seen the comparison of Feb 16 and 17 at
http://igloo.atmos.uiuc.edu/cgi-bin/test/print.sh?fm=02&fd=17&fy=1980&sm=02&sd=17&sy=2009
In 24 hours Hudson Bay has become 1/2 to 3/4 ice free. Siberia also developed large ice free areas in the Arctic Ocean. The Siberian Pacific coast was Ice free on 2-16 and has major ice on building on 2-17.
Something is terribly wrong…WAWT
Cryosphere Today has lots of missing data and duplicate sets, sometimes days. But that is expected and I can live with missing data. The changing data not so much.
I noticed right after that weird missing slice triangle anomaly that appeared a month or so back that the red outer edge new ice references were disappearing from the new data as older purple ice took over. I thought that it was just adjustments of sensitivity. I’ve noticed how the older data new red ice sets have slowly became shaded into purple, I wish now I had taken snapshot of before and after, I would of never thought it necessary.
I suspected some messaging is going on at both ends of the data and stopped going there, and as in the last few days, it’s like three card Monty with ice.
Keep your eye on the lady and lay your money down … ; )
Anthony:
You might be interested in a reply received today from NSIDC Science Communications with respect to a request that a follow up note detail the causes of error, and plan to mitigate future occurrences:
“Dear Sir or Madam,
First, thank you for your interest in the daily sea ice extent images. We are working to solve the problem, which appears to have been caused by a problem with the SSM/I sensor on the F15 satellite. Our scientists are also preparing a more detailed explanation of the error, which will be posted later today or early tomorrow.
Regarding our funding, Arctic Sea Ice News and Analysis is a public service that is only partially funded by NASA–much of the work that goes into the Web site is essentially a volunteer effort by NSIDC scientists and communications staff.
Please let me know if you have any further questions.
Regards,
Katherine Leitzell
NSIDC Science Communications”
1:45 PM PST 2/17/2009
Undoubtedly the reason for this prompt response was my e-mail copied to the supervisors of NSIDC sponsors. I will look into the question of volunteerism since it may be a factor in the politicization of data – intended or not.
bluegrue (16:29:06) :
“In science as well as in the economy you need to take into account the cost of correcting an error in relation to its impact. It usually is not worth your time to hunt down an error, which if corrected will change your result in a non-significant digit only. Results don’t have to be perfect, they just need to be good enough to let you reliably test your hypothesis.”
Not a good approach. Problem identification is part of understanding whether a found error may be pointing to something far worse just under the covers. Ignoring an error just because you think it *might* not be important is often evidence of overconfidence (or maybe arrogance) as anyone who has worked in engineering fields can tell you. This may be the situation in the Steig et al paper. We’ll see if the original error ends up being unique or an indication of things to come.
OTOH, the NSIDC error fits into another category of error. Let’s say you walked into a flower shop to have an arrangement made. As you walked in you noticed a couple of arrangements with wilted flowers. I don’t know about you but I would turn right around an leave. Clearly, the shop owner does not have an emphasis on quality products. That is the image being projected by NSIDC.
My previous post from 2-17@19:00 should have this corrected link for the two date images:
http://igloo.atmos.uiuc.edu/cgi-bin/test/print.sh?fm=02&fd=16&fy=2009&sm=02&sd=17&sy=2009
The volcano fumes must be getting to me.
My apologies
OK folks , the evening update for the AMSR-E sea ice extent is in and it shows an increase for 2/17/09. The NSIDC plot should move up tomorrow — some retroactively.
As Anthony has pointed out, any problems with the NSIDC plot are completely innocent. The overreaction by some folks somewhat justifies their concern over the publicity. The fascist style of Gore and his minions has kinda put everyone on edge.
People and organizations make mistakes all the time. While this particular mistake was harmless enough (and that is only because Anthony and others had warned them before MSM run with it), the big question is if NSIDC has learned anything from it. Have they added slope checking to their QC procedure? Have they created a QC procedure if they didn’t have one? Have they done anything to make sure this error doesn’t happen again? If they did, they should mention it in some way. If they didn’t do anything then they should be vilified and badgered until they do. In commercial world, if I made mistake like that and just dismissed it like nothing happened I would have lost my job. Not for making a mistake, but for refusing to learn anything from it. In my many years of making various mistakes I got it beaten into me that every time I have to:
1. Document what happened
2. Document why it happened
3. Document actions needed to prevent or minimize chances of this happening in the future
4. Implement them.
First 3 do not always required, at least not formally, but #4 is a must.
Making stupid mistakes is mildly embarrassing. Making same ones many times in a row is (and should be) a career killer.
Frank: “If I were a reporter and visited the web site two days ago, I could have written- “According to data provided by the NSIDC, there has been a shocking drop in the arctic sea ice extent.…”
No reporter worth their salt would have written such a sensationalist story without first checking with the source. Elementary stuff.
Robert A Cook: “That it was the NSIDC that was “pushing” the “story” out on the public without checking THEIR facts.”
I wasn’t aware that NSIDC provided an editorial comment to the graph. The thread headed “NSIDC makes a big sea ice extent jump – but why” doesn’t say anything about that.
“That the NSIDC did not recall or change ANY of their graphics UNTIL the story was caught from the web…”
As I understand it, the graph is automatically updated in real time. That’s a matter of quality control rather than propaganda.
Anthony: “Scientists that produce publicly available data on a production basis should have a care about quality control.”
Fair point. I guess I have reservations about the way these mistakes are handled. Some people use them to flog their favourite agendas, imply sinister intent and and generally beat up on the climate agencies and AGW in general. These distractions muddy the water and impede understanding of the issue.
Having worked in small organizations as well as large ones as a professional civil engineer, in the large organizations I have run across gross errors . . . so gross as to make me blink my eyes, and say to myself, “Surely, this can’t possible be right.” Sure enough, they weren’t right.
When I took it upon myself to run these “can’t possibly be right” things down, as I always did, the response was always the same.
“I noticed that myself, but I thought since it seemed so obviously wrong, that surely someone had noticed along the line, and verified it.”
Everyone “along the line” had the same response. “I assumed that someone else had checked this data.”
Good on you, Anthony, for running down what almost everyone else thought had been verified by someone other than themselves.
@ur momisugly Richard M
My response that you quote was directed at pft (15:27:46). He was advocating most stringent error control for preliminary data irrespective of cost and effort, although a final, error corrected version would be produced at a later point of time. pft further posited, that any error whatsoever needs to be corrected.
I was not advocating turning a blind eye to errors, of course you need investigate error soureces. However, once you have identified the error, you do need to evaluate, whether the error is worth the effort in future. In production, it can cost you less to leave a faulty production line unchanged and just identify and throw away faulty products.
Pramatic (19:12:13)
Never underestimate your own importance, never fail to ascribe mallice where an innocent explanation is most likely.
The more I read the more I see one thing that has gone wrong.
The error was spotted by multiple people, who informed Anthony. All of them, Anthony included, seem to have been too busy blogging about the glitch rather than do the obvious thing first: inform NSIDC. I’d be happy to stand corrected on this, however I have yet to see a mention of an such e-mail being sent to NSDIC by any of the bloggers in question. Informing NSDIC was finally done by the readers of the blog, in a multitude of e-mails instead of a single one.
Would it really have hurt, to send an e-mail to NSDIC, informing them of the error and of the fact that you are about to put a blog post up. It would then have been a good idea to include a sentence like “I have informed the NSDIC of the error, let’s see how they react. In the meantime, what is your take on the error and possible causes. I will collect interesting ideas and forward them.” Why?
1) This would have been the swiftest way to deal with the error
2) The blog post would still be there, with all the discussion
3) NSDIC would not have had to deal with a multitude of e-mails of more or less the same content, answering each instead of working on fixing the error.
4) If the near-realtime data posting is indeed an volunteer effort – I have no reason to doubt it, given the e-mail response cited by Pramatic (19:12:13) – the way Anthony seems to have done it, blog but no e-mail, is the best way to get this service culled.
Sylvia (00:28:05) :
Here are references to books on chaos recommended by the person who organizes the academic seminars on chaos and comlexity in my old institute. I asked for Goldstein Mechanics level.
– Nonlinear Dynamics and Chaos
By S. H. Strogatz
– Chaotic Dynamics
By T. Tel and M. Gruiz
– Chaotic Dynamics: An Introduction
By G. L. Baker and J. P. Gollub
Also any book on Nonlinear Dynamics
will have a large part devoted to Chaos.
P.S also I am told by somebody else that the third edition of Goldstein has a good introduction to Chaos
Anthony wrote:
How many page views does your blog have per day? The entire NSIDC domain has about half as many page views as your blog does, according to ALEXA. Keep in mind, that page views is not the same as individual visitors. Your blog, Anthony, has about twice the reach of NSIDC!