In the prior thread I raised a question of why there was a large downward jump in sea ice extent on the graph presented by NSIDC’s Artic Sea Ice News page. The image below was the reason, dozens of people called my attention to it in emails and comments overnight because in the space of a weekend, a million-plus square kilometers of Arctic sea ice went missing. Note the blue line.
Click for larger image
When I checked NSIDC’s web site this morning, about 8:30 AM PST (9:30AM MST Mountain time in Boulder where NSIDC is located) the image was still up. A half hour later it remained. I checked all around the NSIDC web site for any notice, including the links they provide for the data issues.
Learn about update delays, which occasionally occur in near-real-time data. Read about the data.
Finding nothing, and knowing that it was now 10AM in Boulder, which should have been plenty of time to post some sort of notice, I decided to write a quick post about it, which was published at 9:10AM PST (10:10MST) and drove to work.
The corrected image (with the million square kilometers of sea ice restored) appeared on the NSIDC web site just shy of 3 hours later, about noon PST or 1 PM MST.
Click for larger image
About the same time this comment was posted on WUWT by NSIDC’s chief research scientist, Dr. Walt Meier:
Anthony,
We’re looking into it. For the moment, we’ve removed the data from the timeseries plot.
You need to remember that this is near real-time data and there can be data dropouts and bad data due to satellite issues. While the processing is automatic, the QC is partly manual. Thus errors do happen from time to time and one shouldn’t draw any dramatic conclusions from recent data.
I’m not sure why you think things like this are worth blogging about. Data is not perfect, especially near real-time data. That’s not news.
Walt Meier
Research Scientist
NSIDC
ps – FYI, the JAXA data is from a different sensor, so it is not consistent with our data, but it provides a good independent check. If the JAXA data does not show a dramatic change while the NSIDC data does (or vice versa), then it’s likely an issue of missing data or bad data.
First let me say that I have quite a bit of respect for Dr. Meier. He has previously been quite accessible and gracious in providing answers, and even a guest post here. But I was a bit puzzled by his statement “I’m not sure why you think things like this are worth blogging about…. That’s not news“
First let us consider a recent event. The BBC ran really badly researched video report just a couple of days ago where the reporter obviously didn’t know the difference between positive and negative feedbacks in the climate. I wrote about it. The video is now gone. Now I ask this question; if nobody speaks up about these things, would the video still be there misinforming everyone? Probably.
The point I’m making here is that in my experience, most reporters know so little about science that they usually can’t tell the difference between real and erroneous science. Most reporters don’t have that background. I say this from experience, because having worked in TV news for 25 years, I was always the “go to guy” for questions about science and engineering that the reporters couldn’t figure out. And, it wasn’t just at my station that this happened, a meteorologist friend of mine reported the same thing happened to him at his station in the San Francisco bay area. I vividly remember one week he was on vacation and I saw a news report about a plane that crashed that had just minutes before been doing a low level run over the airfield as part of a show. The reporter had video taped the plane’s run, and then used that video to proudly demonstrate “as as you can see, just minutes before the crash, the propellers on the plane were turning very slowly”.
The reporter didn’t understand about how a video camera scanning at 30 frames per second can create a beat frequency that give the impression of slowly turning propellers that were actually running about 3000 RPM., and there was nobody there to tell her otherwise. She made an honest mistake, but her training didn’t even raise a question in her mind.
So when I see something obviously wrong, such as a dramatic drop in sea ice on a graph presented for public consumption, I think about a reporter (print, web, or video -take your pick) somewhere in the world who may be assigned to do a story about sea ice today and does an Internet search, landing on NSDIC’s web site and then concluding in the story “and as you can see in this graph, Arctic sea ice has gone through a dramatic drop just in the last few days, losing over a million square kilometers”.
Thinking about Walt’s statement, “ That’s not news” if the NSIDC graph had been picked up by a major media outlet today, would it be news then?
I understand about automation, about data dropouts, and about processing errors. I run 50 servers myself and produce all sorts of automated graphics output, some of which you can see in the right sidebar. These are used by TV stations, cable channels, and radio/newspaper outlets in the USA for web and on-air. While those graphics are there on WUWT for my readers, I also have an ulterior motive in quality control. Because I can keep an eye on the output when I’m blogging. When data is presented for public consumption, in a venue where 24 hour news is the norm, you can’t simply let computers post things for public consumption without regular quality control checking. The more eyes the better.
At the very least, a note next to NSIDC”s Learn about update delays, about how glitches in satellite data or processing might generate an erroneous result in might be in order. And also for consideration, adding a date/time stamp to the image so it can be properly referenced in the context of time. This is standard operating procedure in many places, why not at NSIDC?
NSIDC and other organizations need to realize that the interest in what they produce has been huge as of late. In NSIDC’s case, they have been promoted from relative obscurity to front page news by the recent unfortunate statements of an NSIDC employee, Dr. Mark Serreze, to the media, that have received wide coverage.
As commenter “just want truth” wrote in the previous thread on NSIDC:
Last year Mark Serreze, of the NSIDC (you may know him), said North Pole ice could be gone in the summer of 2008. He said then “The set-up for this summer is disturbing”. This, of course, was broadcast in all news outlets around the world. Everyone on both sides of the global warming debate was watching Arctic ice totals last summer to see what would really happen. You may have noticed hits on the NSIDC web site were high last summer.
Now Mark Serreze is saying North Pole ice is in a “death spiral”.
You can be certain that Arctic ice data will be scrutinized because of Al Gore and Mark Serreze. A line has been drawn by both. Both have placed it clearly on the radar screen. This is why NSIDC data is worth blogging about–especially since Mark Serreze is employed at the NSIDC.
Mark Serreze 2008 North Pole ice free :
http://abcnews.go.com/Technology/Story?id=4728737&page=1
and
http://www.youtube.com/watch?v=S6e3e4VzwJI
Mark Serreze North Pole ice in “death spiral” :
http://www.youtube.com/watch?v=HW9lX8evwIw
and
http://www.nypost.com/seven/08282008/news/worldnews/arctic_ice_in_death_spiral_126443.htm
Given the sort of attention that has been heaped on NSIDC, I think blogging about errors that have gone unnoticed and uncorrected by 10AM on a Monday morning isn’t an unreasonable thing to do.
I also think that reining in loose cannons that can do some terrible damage in the media is a good way to maintain scientific credibility for an organization, especially when predictions like “ice free north pole” don’t come true.
I have no quarrel with Dr. Meier, as I’ve said he’s been the utmost professional in my dealings with him. But I do have quarrel with an organization that allows such claims to be broadcast, all the while producing a data source that is now regularly scrutinized by the public and the media for the slighest changes. It’s a slippery slope.
Anthony, all I can say is: You go boy! Someone must speak up to point out the errors before the reporters blow them out of all proportion!
Good for you, Anthony! Mega-kudos!
In stark contrast to the 1960’s when most of us had no choice but to believe Walter Cronkite when he told us, “And that’s the way it is” in that deep, believable voice, these days there are not only eyes in the sky, but many millions of people with internet access and all that that implies for data questioning, checking, cross-referencing, and publishing.
Ironic, just a bit, since Al Gore tells us he invented the internet. Thank you, Al!
Having much experience with various government agencies I can say they do not like this kind of power in the hands of the citizenry.
To be able to call them on the carpet at a moments notice is too much for their busy little bureaucrat selves to fathom.
They are so seldom held accountable for anything they find it insulting when they are.
They do not view themselves as public employees. More like public bosses.
Unfortunately the MSM get access to Joe Public – right or wrong or whatever the agenda of the day suits them . Retractions are never made public. Joe Public doesn’t check sites like WUWT. Joe Public is the 30 second gran specialist.
However one overriding thing. Joe Public is very aware of the nightly news weather report. If more T.V. meteorologists were allowed to put in their two bob’s worth maybe a more balanced approach to “climate Change” could be perceived.
Walt Meyer: “ps – FYI, the JAXA data is from a different sensor, so it is not consistent with our data, but it provides a good independent check. If the JAXA data does not show a dramatic change while the NSIDC data does (or vice versa), then it’s likely an issue of missing data or bad data.”
But JAXA data is constantly and significantly different. NSIDC is showing arctic sea ice to be the same as (or even lower than) 2007 for most of this year. JAXA has it much higher and similar rather to 2008 than 2007.
http://www.ijis.iarc.uaf.edu/en/home/seaice_extent.htm
Anthony,
It is worth reporting! “Blogging is the wrong term. I consider your work “Reporting”. Reporting is a better term for what you do because in my view there are very few in the media who report. The quality of their reporting would be better described as blogging.
If the media would “report” then we would’nt have all this misinformation the public is exposed to by ignorant so called reporters. Rather, you are providing a service that fits true reporting.
If the Dr. had his head in the game he would understand what you are doing and he would not respond with a chip on his shoulder. If he knows better, he should recognize the work you do for what it is.
Maybe we we should refer to your efforts as “Ground and Pound”. Don’t let them off the ground Anthony and keep pounding.
Further to my on post 22:20:18
And so is Nansen sea ice extent:
http://arctic-roos.org/observations/satellite-data/sea-ice/ice-area-and-extent-in-arctic
Why can’t the NSIDC see the questionable data/graphs as quickly as everyone else can? If there are going to be “errors” made in favor of haste, forget haste. If the NSIDC is going to catch the errors quickly anyway, why put the questionable data up to begin with. If it is not going to catch the posted errors quickly, then why shouldn’t someone else do it so as to minimize any damage which might occur? And why should we accept the premise that the NSIDC is indeed even going to catch the errors itself, when it appears to operate under such a lax and strange procedural logic concerning its own data and its importance.
Thanks, Anthony, for holding this blog and the way you are guiding it.
It becomes the voice of the hoi polloi, an amplifier gathering the concerns of individuals that cannot be/are not heard when clicking on the “communicate” link of all these data presenting web pages.
Anthony, I agree that errors like this are worth blogging about, but you’re previous posting on the ice data may have implied some questionable intent on the part of NSIDC. It would have been helpful if you would have pointed out that this was likely a data error, and not a human/bias error in your first posting. Real-time data source obviously are more susceptible to data errors.
REPLY: I hadn’t realized that anyone read suggestions of “bias” coming from me into that post. OTOH, I was under the gun, being late for work for giving opportunity for the problem to be fixed on it’s own, in which case I would have published nothing. So perhaps in my own haste I didn’t do the best job on conveyance possible. It just seems to me that if you put your organization squarely in the public eye as NSIDC has done, your publicly presented data is your reputation. Why such things aren’t checked and corrected first thing in the morning, and remain 2 hours later, is the true puzzle. – Anthony
In today’s environment where any crisis is to be used to further political power grabs, Dr. Meier seems to be extremely insensitive or naive in handling this data. A million plus square kilometers of missing ice is certainly a news worthy item for those who advocate global warming and your suggestion of a warning on real time data errors should be taken to heart and implemented quickly by NSIDC. This makes me wonder what data Mark Surreze uses to back his statements.
If Dr. Meier wanted to add a postscript, it should have been to thank you for pulling his fat out of the fire before any one of a hundred mainstream media global warming alarmists made front page news out of his data glitch. Does this guy really not realize that you saved his bacon?
REPLY: I’ve never thought of it that way (saving bacon), and I expect no thanks. If the data is wrong it needs to be fixed before somebody uses it, simple as that. – Anthony
The point I’m making here is that in my experience, most reporters know so little about science that they usually can’t tell the difference between real and erroneous science.
That’s for sure.
You’re used to being famous by now, so you know every word you say is scrutinized. Sometimes they are taken wrongly by the individuals being looked at. I have as much respect for the NSIDC right now as I I have for any government agency I’ve run into. That’s not to say they don’t need to be watched. Your post will do nothing but improve the QC for next time.
One thing’s for sure the NSIDC reads WUWT, can’t blame ’em.
Why is the noise for the trace from end of 2008 and now 2009 is worse than all the previous years? If “bad” data passed through their system this time, what about the other times in recent past? Like, what happen in December 2008?
Everything is worth blogging about, heck even the ranting of James Hansen is worth blogging about … whether its newsworthy or not is another question.
It’s always good when someone can spot goofs and alert the generator before things get out of hand.
I’m sure govt types in charge of the technical end of things (including Dr H) don’t like having their mistakes picked up by MSM reporters totally
ignorantunschooled in the subject, and reported as “Truth”. I’ve seen far too many military and aviation stories mangled by journalism majors, and I’ve often wished I’d had a phone number to call.May be this is not out of topic. Back in december, while looking at links given by posters here I saw this intriguing plot in cryosphere:
http://igloo.atmos.uiuc.edu/cgi-bin/test/print.sh?fm=12&fd=06&fy=2008&sm=02&sd=16&sy=2009
Look at the left december 6 2008 plot.
We had a discussion in the blog at that time.
I sent an inquiry to the link provided then ( I think it was a person, this has changed in the current home page), politely framing my puzzlement and asking if the conclusion of the blog discussion that it is an artifact of the way the satellite data are combined, was true.
I never got an answer. And the plot is still there. It was there in other views in their archives last time I checked.
Is it reasonable that a necklace of beads appears on a scientific plot and there is no discussion of the effect?
Vast iceberg breaks off Wilkins Ice Shelf in Antarctic
Satellite images have revealed that about 160 square miles of the Wilkins Shelf have been lost since the end of February, suggesting that climate change could be causing it to disintegrate much more quickly than scientists had predicted. “The ice shelf is hanging by a thread,” said David Vaughan, of the British Antarctic Survey(BAS). “We’ll know in the next few days or weeks what its fate will be.” …
… The Wilkins Shelf is now protected by only a thin thread of ice between two islands. It covers an area of 5,600 square miles (14,500 sq km).
Somethings not right here. I read through that whole article by the Times, and not once did it quote Walt Meier on how the break up of 14k sq. km isn’t news worthy.
That article is almost a year old. In the grand scheme of things, 160 sq. km. isn’t very big. Antarctica itself is 14,000,000 sq. km. (14 million sq. km) in area.
Antarctica’s sea ice has increased and it’s gotten colder over the last 50 years. The ice extent is running ONE MILLION sq. km. above average. So losing a measly 160 sq. km. (60 sq. mi) isn’t a big deal at all.
A fine, deliberate, rational response to a problem noted on the NSIDC site.
A quick look at the imagery of ice extent should have alerted the most novice operator or intern that something was wrong when large expanses of open water suddenly appeared in the Bering Sea or Hudson Bay (among others). They could have held off putting the data up; NSIDC has skipped days many times in the past.
Even with their caveat about the quality of near real time data, a scientifically challenged reporter would miss that note and just see the first thing that would catch his eye — the sudden disappearance of large amounts of ice — and run with it.
Good onya Anthony. It behoves all organisations and reporters to check their facts before reporting anywhere. It’s the same in medicine. I’m an RN working in Intensive Care. Woe betide the pathology lab that gives us erroneous results we use to then treat patients. Perhaps all organisations and reporters could remember the builder’s maxim: “Measure twice. Cut once.” If all their facts were checked first before making a statement all this kerfuffle could have been avoided and no one would be wiping egg off their face. 🙂
why is it when we find an error we bad?
but when they find error in our favor it is keep quite, and fixed fast?
just saying.
Dr. Meier:
I’m not sure why you think things like this are worth blogging about. Data is not perfect, especially near real-time data. That’s not news.
Note the meme, “that’s not news”. Repeating it then makes it true that it’s ok to publish data with errors in it?
What is real news is that Dr. Meier thinks it’s so important to get data out in public quickly, that he is ok with errors in that data. That view is self-contradictory, imo, valuing haste over the value of the product which the haste delivers, which was supposed to be the only reason for the haste.
It’s worth blogging about for the excellent reasons Anthony gives, but also because Dr. Meier is not making any sense as concerns his responsibiliy in making NISDC data available, and he needs to know that in a public way.
A question to Dr. Meier: who wants data with errors in it? Please name these people.
You have to assume a basic level of competence for viewers of this sort of site, so it doesn’t worry me that a semi-automated chart sometimes has glitches. It’s not in the same ball park as repeating one months data points for the next and not spotting it for instance as we recently had with global temps 🙂
I think even a reporter who doesn’t know what dilithium feedback is could spot that.
Regards
Andy
REPLY: Ah but there’s the rub, the reporters often don’t have the advantage of knowing about these automation issues, nor do they usually care. Remember, most reporters mine for data and quote nuggets. In depth is not the norm.
“At least 30 people died”…”more than a 50 thousand dollars in damages were reported”…”sources, speaking on condition of anonymity say” and “I did not have sex with that woman” are just as relevant as “over a million square kilometers of sea ice melted in the Arctic”. – Anthony
Give to em Anthony!!
These people in public bureaucracies (BBC, NSIDC, GISS, etc etc) just don’t get the idea of quality control. It’s a concept totally foreign to them.
Well done Anthony; just keep on at them.