Guest Post by Willis Eschenbach
A while back, I noticed an oddity about the Hadley Centre’s HadISST sea ice dataset for the Arctic. There’s a big change in variation from the pre- to the post-satellite era. Satellite measurements of ice areas began in 1979. Here is the full HadISST record, with the monthly variations removed.
Figure 1. Anomaly in the monthly sea ice coverage as reported by the HadISST, GISST, and Reynolds datasets. All data are from KNMI. Monthly average variations from the overlap period (1981-1994) have been subtracted from each dataset. All data are from KNMI (see Monthly Observations).
There’s a few points of note. First, the pre-1953 data is pretty useless, much of it is obviously not changing from year to year. Second, although the variation in the GISST dataset is doesn’t change in 1979, the variation in the HadISST dataset changes pretty radically at that point. Third, there is a large difference between the variability of the Reynolds and the GISST datasets during the period of their overlap.
I had filed this under unexplained curiosities and forgotten about it … until the recent publication of a paper called Observations reveal external driver for Arctic sea-ice retreat, by Notz and Marotzke, hereinafter N&M2012
Why did their paper bring this issue to the fore?
Well, the problem is that the observations they use to establish their case are the difference in variability of the HadISST during period 1953-1979, compared to the HadISST variations since that time. They look at the early variations, and they use them as “a good estimate of internal variability”. I have problems with this assumption in general due to the short length of time (25 years), which is way too little data to establish “internal variability” even if the data were good … but it’s not good, it has problems.
To their credit, the authors recognized the problems in N&M2012, saying:
Second, from 1979 onwards the HadISST data set is primarily based on satellite observations. We find across the 1978/1979 boundary an unusually large increase in sea-ice extent in March and an unusually large decrease in sea-ice extent in September (Figures 1b and 1d). This indicates a possible inconsistency in the data set across this boundary.
Ya think? I love these guys, “possible inconsistency”. The use of this kind of weasel words. like “may” and “might” and “could” and “possible”, is Cain’s mark on the post-normal scientist. Let me remove the GISST and Reynolds datasets and plot just the modern period that they use, to see if you can spot their “possible inconsistency” between the 1953-1979 and the post 1979 periods…
Figure 2. As in Figure 1, for HadISST only.
The inconsistency is clearly visible, with the variability of the pre- and post-1979 periods being very different.
As a result, what they are doing is comparing apples and oranges. They are assuming the 1953-1979 record is the “natural variability”, and then they are comparing that to the variability of the post-1979 period … I’m sorry, but you just can’t do that. You can’t compare one dataset with another when they are based on two totally different types of measurements, satellite and ground, especially when there is an obvious inconsistency between the two.
In addition, since the GISST dataset doesn’t contain the large change in variability seen in the HadISST dataset, it is at least a working assumption that there is some structural error in the HadISST dataset … but the authors just ignore that and move forwards.
Finally, we have a problematic underlying assumption that involves something called “stationarity”. The stationarity assumption says that the various statistical measures (average, standard deviation, variation) are “stationary”, meaning that they don’t change over time.
They nod their heads to the stationarity problem, saying (emphasis mine):
For the long-term memory process, we estimate the Hurst coefficient H of the pre-satellite time series using detrended fluctuation analysis (DFA) [Peng et al., 1994]. Only a rough estimate of 0.8 < H < 0.9 is possible both because of the short length of the time series and because DFA shows non-stationarity even after removal of the seasonal cycle.
Unfortunately, they don’t follow the problem of non-stationarity to its logical conclusion. Look, for example, at the variability in the satellite record in the period 1990-2000 versus the period 2000-2005. They are quite different. In their analysis, they claim that a difference in variability pre- to post-1979 establishes that human actions are the “external driver” … but they don’t deal with the differences pre- and post-2000, or with the fact that their own analysis shows that even the variability of the pre-1979 data is not stationary.
Finally, look at the large change in variability in the most recent part of the record. The authors don’t mention that … but the HadISST folks do.
03/DECEMBER/2010. The SSM/I satellite that was used to provide the data for the sea ice analysis in HadISST suffered a significant degradation in performance through January and February 2009. The problem affected HadISST fields from January 2009 and probably causes an underestimate of ice extent and concentration. It also affected sea surface temperatures in sea ice areas because the SSTs are estimated from the sea ice concentration (see Rayner et al. 2003). As of 3rd December 2010 we have reprocessed the data from January 2009 to the present using a different sea ice data source. This is an improvement on the previous situation, but users should still note that the switch of data source at the start of 2009 might introduce a discontinuity into the record. The reprocessed files are available from the main data page. The older version of the data set is archived here.
08/MARCH/2011. The switch of satellite source data at the start of 2009 introduced a discontinuity in the fields of sea ice in both the Arctic and Antarctic.
Curious … the degradation in the recent satellite data “probably causes an underestimate of ice extent and concentration,” and yet it is precisely that low recent ice concentration that they claim “reveals an external driver” …
In any case, when I put all of those problems together, the changes in variability in 1979, in 2000, and in 2009, plus the demonstrated non-stationarity pre-1979, plus the indirect evidence from the GISST and Reynolds datasets, plus the problems with the satellites affecting the critical recent period, the period they claim is statistically significant in their analysis … well, given all that I’d say that the N&M2012 method (comparing variability pre- and post-1979) is totally inappropriate given the available data. There are far too many changes and differences in variability, both internal to and between the datasets, to claim that the 1979 change in variability means anything at all … much less that it reveals an “external driver” for the changes in Arctic sea ice.
w.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Willis said Well, the problem is that the observations they use to establish their case are the difference in variability of the HadISST during period 1953-1979, compared to the HadISST variations since that time.
What IS the “difference in variability of the HadISST during period 1953-1979, compared to the HadISST variations since that time”, as reported by the paper, Willis ? Hint : answer is on page 2 of the paper.
And while we are at it, where in the paper did you see that this difference is relevant “to establish their case” ?
Peter, This is brilliant – it dismisses the statistical analysis used in a peer-reviewed paper in a prestigious journal without actually doing any statistics!
It’s not just that Willis does not use any statistics at all. The bigger problem is that Willis seems to be completely clueless about the data analysis used as a basis for the conclusions in this paper. It’s almost as if he deliberately avoids the statistics and data analysis, and instead pretends to reveal issues in the various climate records, even though these have already been addressed thoroughly in the paper.
Dennis Ray Wingo, would you care to share some of that pre-1979 data with us ?
1979-2008 is used as anomaly mean there – which means anomalies are artificially minimized for this interval. Anything outside that interval is going to have increasing anomalies unless the system is stationary.
Pre-satellite era is characterized by way lower certainity of results. Someone here recently stated that it’s not science if there are no error bars. I’m missing them in these graphs.
And regarding the recent change, I wouldn’t blame the satellite too much. The change did not start in 2009, it started in half 2007 and in 2009 it was going on for two years already. It would be sure interesting to find a satellite problem causing it as it was rather abrupt but I don’t see any mentioned here. Quite likely the 2009 satellite measurement error is order of magnitude smaller than the real change which occurred.
Very neat. The paper should have been called Observations reveal discontinuities for Arctic sea-ice.
What a surprize ! More garbage in yields more garbage out. Who would have thunk it ?
The problem with any kind of measurement of reality is ALWAYS estimating how close to reality one is approaching. All experimental and field scientists worth their salt understand that statistically validated data is required in order to yield estimates that at least approach reality.
Only climate scientists and charlatans seem to be able to blow smoke up everyone else’s butts about the probitive statistical validity of a sample size one ONE.
A sample size of ONE. That’s the takaway from all this….. One thermometer (forget that the types change over time), one satellite (forget about the fact that serial satellite comparisons show their internal / external inconsistencies), no replicates, no random sampling, no field validation of remote sensing data nor proper calculation of stats, nor estimation of the size of instrument error, etc.
Just pitiful.
It is important not to confuse satellite ‘imagery’ from visible and passive microwave. Passive microwave retrievals started in 1979 but visual images from satellite started earlier.
Rob Dekker says:
May 4, 2012 at 1:37 am (Edit)
Hint: the difference in variability is meaningless.
As to where in the paper it says the difference in variability is relevant, it is the point of the entire paper. See the introduction:
What do you think they are talking about if not the difference in variability?
Or see e.g. Section 3, called “Internal Variability”. What they are doing is showing that the variability of the later period exceeds the “internal variability” exhibited by the earlier pre-satellite period. You sure you read the paper? It’s all about variability, that’s what they are using to make their case.
Rob, let me try explaining the underlying problem again. It’s as though they estimated a sample of people’s weights by looking at the size of their pants, and then they estimated the weights of another sample of people by looking at the size of their shirts. Then they say that because one sample exceeds the “internal variance” of the other, it shows that there is an “external driver” making them different.
You still don’t seem to grasp the nettle—you can’t simply grab two datasets from different ways of measuring something, splice the two datasets together, and then claim that the difference between the two is meaningful.
No, the issues haven’t been “addressed thoroughly” in the paper. They have been mentioned in the paper. Where, for example, have they “addressed thoroughly” the issue of non-stationarity? Yes, they mention it, but that’s all they do.
And some issues are not even mentioned, like the satellite problems in the recent data.
You have the same problem that Peter had. You are happy to accuse me of a variety of sins, but you say nothing about what I might have done wrong.
Rob, I ask of you what I asked of peter. If you disagree with something I’ve said, stop prancing around asking Socratic questions and going hint hint. If you disagree, QUOTE MY WORDS and explain exactly why they are wrong.
Finally, you seem to think it’s a crime that I haven’t gone into the statistical nuts and bolts. There was no need to, since the underlying premise was flawed. But if you like statistical questions, how about this one.
They assume that despite the fact that they find the pre-1979 records to be non-stationary, that they can estimate “internal variability” of the system from that short 25-year record.
Given what we know about the general long-term variability of the climate, and the known existence of ~ 60-year cycles in the Arctic temperatures, and the fact that they themselves say the pre-1979 dataset is non-stationary … what are the error limits on their calculation of “internal variability”?
And given those problems and the shortness of the record, can the errors on their “internal variability” even be calculated?
I’d be interested in your answers to those questions … be sure to use lots of statistics, they seem to impress some people …
w.
Dennis Ray Wingo says:
May 4, 2012 at 12:34 am
Thanks, Dennis. If you have citations to that earlier data it would be great.
However, when you say “the referenced paper is completely wrong” about the satellite era, you mistake their meaning. They are talking about “satellite era” in reference to the datasets that they used. The HadISST data is put together like this:
SOURCE: Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century (large PDF from Hadley Centre)
Note that it is non-satellite data up to November of 1978, and satellite data thereafter. That is why they can refer to the “satellite era” starting in 1979.
In a more general sense, I think that “satellite era” in reference to 1979 means the start of continuous observations by satellite. As far as I know, the earlier Nimbus satellites, although quite useful, didn’t provide a continuous record.
Curiously, the authors don’t even make use of the HadISST satellite data. Instead, they are using the NSIDC data after 1978, viz:
For those insisting that they need statistics, I note that the authors have not provided any statistics or any sensitivity analysis justifying this choice of one of the two satellite datasets, or demonstrating its effect on the final result.
w.
William McClenney says:
May 3, 2012 at 6:53 pm
///////////////////////////////
I agree with William on this particular criticism (which does not distract from the thrust of your article as a whole).
One of the main problems in this so called ‘science’ is the certainty with which data or trends or results or projections or predictions or proxies are claimed. Every claim and every interpretation in this so called ‘science’ should be couched in terms of uncertainty. I therefore consider it wrong to critise when some authors of a paper are indicating uncertainty
That said, I think your article raises some interesting issues and suggests flaws in the methodology used in the N & M paper. Unfortunately, no great surprise to see that data is being improperly handled.
Willis, when you write them I can follow them, pure genius, you should be President.
As with everything else AGW the paper described here hits another nail on the head. Obviously sea ice is diminishing due to mankind’s activities: in this case they have uncovered the nefarious effect of a man-made satellite on the entire Arctic’s stability.
Schrödinger and his cat would be proud, if we could only see them.
On the other hand, maybe this is just one more for Maurizio Morabito’s list of fortuitously timed observations. We switched to satellite observations in exactly the same month that natural variability was overrun by man-made forcing! What luck!
Partington et al 2003: http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1058&context=usdeptcommercepub
“Both chart data and passive microwave data show a negative trend in integrated arctic-wide concentration over the period 1979-1994. The difference between the passive microwave and chart trends is statistically significant only in the summer, where it is about 2 percent per decade steeper in passive microwave data.”
“Differences between the NIC ice chart sea ice record and the passive microwave sea ice record are highly significant despite the fact that the NIC charts are semi-dependent on the passive microwave data, and it is worth noting these differences.”
There is a significant divergence between non-satellite (including visible derived satellite) and satellite passive microwave Arctic sea ice measures. There are also different trends between satellites with decaying orbits (DMSP) and controlled orbit (AQUA).
Could Instrumentation Drift Account for Arctic Sea Ice Decline? http://www.scribd.com/doc/89395853/Could-Instrumentation-Drift-Account-for-Arctic-Sea-Ice-Decline
Before anyone jumps to conclusions, GISST stands for Global sea Ice and Sea Surface Temperature. It is a product of the Hadley Centre, not GISS.
Willis: hadisst and gisst are not sea ice records in themselves. They collect other sources of ice information. If you want to do you analysis of sea ice information with primary ice data, you want NSIDC, not hadisst or gisst.
Re Nimbus I:
National Geographic, Feb. 1965, pp. 189-193. “Incredible photograph shows Earth from pole to pole.” Nimbus I was only up for 26 days but made 27,000 pictures. Both North and South poles are shown in the pic on pp. 189-190. They were created by heat scanning, so show the photo through heat contrast. The photo of Europe on page 191-193 is quite good as there aren’t as many clouds which show up as “cool.”
Although I wouldn’t trust any of it to not be more subtly *adjusted*, what they did for the pre-1950 part of the graph is even amazingly blatant. I’ve saving this graph as an example. Peaks/troughs in exactly flat lines unchanging over decades never exist in the real world if one knows anything about weather and climate, how such is never exactly constant. (If they want to pretend zero data, although that isn’t really true, then they have no justification for depicting that time period at all within their graph).
The HadISST and GISST claims so utterly contrast to
http://earthobservatory.nasa.gov/Features/ArcticIce/Images/arctic_temp_trends_rt.gif (temperatures
as warm or warmer in a peak in the 1930s than in the late 20th century) as well as sea ice maps discussed at the recent http://wattsupwiththat.com/2012/05/02/cache-of-historical-arctic-sea-ice-maps-discovered/ article.
In one way, this makes sense. Someone like me or almost all other skeptics would never even consider trying to enter the field and get employment at institutions like that in the post-Mann-era, because we know we would never fit in and not be penalized, whereas the kind of ideologues who do not mind dishonesty for the cause of some environmentalists have increasingly gravitated towards them. Almost anything on climate published in the late 1990s and beyond, especially that published in the past several years, has to be double-checked for likely intentional skewing (not always but far too often). In contrast, though, one can relatively trust any older scientific publications up to around the 1970s at least. At this point, I think the world essentially needs the uncertain but possible scenario of solar cycle 24 after its peak declining over subsequent years into a Grand Minimum with severe temperature drop over most of the next dozen or so years (aside from short fluctuations on the scale of one to several years at a time from ocean variation and other precipitation-temperature fluctuations like El Ninos versus La Ninas), to destroy the spread of corruption.
Let’s say I weigh myself every day using a primitive balance-and-counterweight scale, and dutifully record the values I measure with this contraption. Then one day in 1979, I upgrade to a new, accurate digital scale and continue my weight record with the new, more accurate values. One day I look back at my weight record and find I’ve been gradually but surely gaining weight. I would like to examine the data to find if the reason I’ve been gaining weight is that I’ve been eating, but it occurs to me that I should carefully examine the older weight records first because I know the data is less accurate. But Willis says, “you just can’t do that.” “You can’t compare one dataset with another when they are based on two totally different types of measurements,” Willis tells me. I try to explain to Willis that I am going to analyze the older data carefully to insure it can be compared against the newer data. But to Willis, that doesn’t matter. He looks at my chart of data and says, “The inconsistency is clearly visible, with the variability of the pre- and post-1979 periods being very different.”
Well, there we have it. I guess I haven’t been gaining weight! Thanks Willis!
Peter says:
May 4, 2012 at 7:29 am
There is a difference between your absolute weight gain and the variability of your weight gain record from one measurement system to another. It is the accuracy of the measurement of this variability that is the point of the discussion and the basis for the claims in the subject article.
No Peter, you don’t understand what “different types of measurements” means. The proper analogy would be that you “measured” your weight until 1978 by using a measuring tape and recording the circumference of your thigh, and then from 1979 on, you switched to a digital scale to weigh yourself. Since your thigh circumference did not increase with your weight gain (because it was all going into your belly), you conclude with near certainty from your measurements that you started gaining weight in 1979 and it must be due to AGW.
Don’t mind me. I’m trying to stop the emails about follow-up comments I keep getting from this thread. I’ve tried a number of other things and none have worked.
[Reply: Go to ‘settings’ on the page you received. Turn off emails. Worked for me. ~dbs, mod.]
“Dennis Ray Wingo says:
May 4, 2012 at 12:34 am
First of all, the referenced paper is completely wrong about the “beginning” of the satellite era. There is an almost continuous record of Arctic and Antarctic ice from satellites,stretching back to the early 1960s’.
My team has personally helped the NSIDC with Nimbus I, II, and III HRIR data. Beyond that, it has recently been discovered that there is a film copy of the Nimbus AVCS (visible light) images.
We need to put this 1979 boundary crap to bed.”
I checked the interannual std.dev. in the HADISST ( downloadable from here http://climexp.knmi.nl/get_index.cgi ) and calculated the year-to-year differences. Before 1953 the data are not useful for calculating the variability (differences 1945…1953 are zero!), this point is reflected correctly in the paper ( Introduction, point 6). Anyway, from 1953 to 1965 the differences are much more tiny than in the years after 1964 so one can say, the data from 1953 to 1965 are some kind of suspicious.
From 1965 on the data seem to be useful for calculating the variability. This is consistent with your statement. So the paper should have used only data from 1965 on.
dh7fb
…cause we had satellites and computers to measure that sea ice before 1953. That data is a waste of time.
Check NASA SP-489 for the years 1973-76.
Nimbus 1, 2, 3, and 4, have records as well as other satellites. We have done the work just for the HRIR data for these birds. The AVCS images are being scanned even as we speak. While no one satellite has a continuous record, if you paste all of them together the record goes back quite far.
There are papers on this from the AGU. I don’t want to steal anyone’s thunder from the NSIDC as they have done a major service in this area.
An addendum is that the HRIR images are worth far more than just the ice. We have been able to track mid 1960’s hurricanes and the first calculation of the global heat balance was done using Nimbus II HRIR images. In my opinion, a very good scientific study of the configuration of the atmosphere in the colder conditions of the mid 1960’s and compare that with the 70’s, 80’s, and through today would provide major insights into atmospheric circulation, rainfall patterns, and actually allow some really good predictions of weather based on global temperatures….
Larry says:
May 4, 2012 at 10:44 am
“…cause we had satellites and computers to measure that sea ice before 1953. That data is a waste of time.”
Didn’t know Sputnik I measured sea ice and it was the first to orbit Earth in 1957. Or was it the monkey on the Jupiter launch in 1959? What satellite was in 1953? Not all “space launches” reached orbital velocity which is required to be considered a “satellite”.