Guest Post by Willis Eschenbach
A while back, I noticed an oddity about the Hadley Centre’s HadISST sea ice dataset for the Arctic. There’s a big change in variation from the pre- to the post-satellite era. Satellite measurements of ice areas began in 1979. Here is the full HadISST record, with the monthly variations removed.
Figure 1. Anomaly in the monthly sea ice coverage as reported by the HadISST, GISST, and Reynolds datasets. All data are from KNMI. Monthly average variations from the overlap period (1981-1994) have been subtracted from each dataset. All data are from KNMI (see Monthly Observations).
There’s a few points of note. First, the pre-1953 data is pretty useless, much of it is obviously not changing from year to year. Second, although the variation in the GISST dataset is doesn’t change in 1979, the variation in the HadISST dataset changes pretty radically at that point. Third, there is a large difference between the variability of the Reynolds and the GISST datasets during the period of their overlap.
I had filed this under unexplained curiosities and forgotten about it … until the recent publication of a paper called Observations reveal external driver for Arctic sea-ice retreat, by Notz and Marotzke, hereinafter N&M2012
Why did their paper bring this issue to the fore?
Well, the problem is that the observations they use to establish their case are the difference in variability of the HadISST during period 1953-1979, compared to the HadISST variations since that time. They look at the early variations, and they use them as “a good estimate of internal variability”. I have problems with this assumption in general due to the short length of time (25 years), which is way too little data to establish “internal variability” even if the data were good … but it’s not good, it has problems.
To their credit, the authors recognized the problems in N&M2012, saying:
Second, from 1979 onwards the HadISST data set is primarily based on satellite observations. We find across the 1978/1979 boundary an unusually large increase in sea-ice extent in March and an unusually large decrease in sea-ice extent in September (Figures 1b and 1d). This indicates a possible inconsistency in the data set across this boundary.
Ya think? I love these guys, “possible inconsistency”. The use of this kind of weasel words. like “may” and “might” and “could” and “possible”, is Cain’s mark on the post-normal scientist. Let me remove the GISST and Reynolds datasets and plot just the modern period that they use, to see if you can spot their “possible inconsistency” between the 1953-1979 and the post 1979 periods…
Figure 2. As in Figure 1, for HadISST only.
The inconsistency is clearly visible, with the variability of the pre- and post-1979 periods being very different.
As a result, what they are doing is comparing apples and oranges. They are assuming the 1953-1979 record is the “natural variability”, and then they are comparing that to the variability of the post-1979 period … I’m sorry, but you just can’t do that. You can’t compare one dataset with another when they are based on two totally different types of measurements, satellite and ground, especially when there is an obvious inconsistency between the two.
In addition, since the GISST dataset doesn’t contain the large change in variability seen in the HadISST dataset, it is at least a working assumption that there is some structural error in the HadISST dataset … but the authors just ignore that and move forwards.
Finally, we have a problematic underlying assumption that involves something called “stationarity”. The stationarity assumption says that the various statistical measures (average, standard deviation, variation) are “stationary”, meaning that they don’t change over time.
They nod their heads to the stationarity problem, saying (emphasis mine):
For the long-term memory process, we estimate the Hurst coefficient H of the pre-satellite time series using detrended fluctuation analysis (DFA) [Peng et al., 1994]. Only a rough estimate of 0.8 < H < 0.9 is possible both because of the short length of the time series and because DFA shows non-stationarity even after removal of the seasonal cycle.
Unfortunately, they don’t follow the problem of non-stationarity to its logical conclusion. Look, for example, at the variability in the satellite record in the period 1990-2000 versus the period 2000-2005. They are quite different. In their analysis, they claim that a difference in variability pre- to post-1979 establishes that human actions are the “external driver” … but they don’t deal with the differences pre- and post-2000, or with the fact that their own analysis shows that even the variability of the pre-1979 data is not stationary.
Finally, look at the large change in variability in the most recent part of the record. The authors don’t mention that … but the HadISST folks do.
03/DECEMBER/2010. The SSM/I satellite that was used to provide the data for the sea ice analysis in HadISST suffered a significant degradation in performance through January and February 2009. The problem affected HadISST fields from January 2009 and probably causes an underestimate of ice extent and concentration. It also affected sea surface temperatures in sea ice areas because the SSTs are estimated from the sea ice concentration (see Rayner et al. 2003). As of 3rd December 2010 we have reprocessed the data from January 2009 to the present using a different sea ice data source. This is an improvement on the previous situation, but users should still note that the switch of data source at the start of 2009 might introduce a discontinuity into the record. The reprocessed files are available from the main data page. The older version of the data set is archived here.
08/MARCH/2011. The switch of satellite source data at the start of 2009 introduced a discontinuity in the fields of sea ice in both the Arctic and Antarctic.
Curious … the degradation in the recent satellite data “probably causes an underestimate of ice extent and concentration,” and yet it is precisely that low recent ice concentration that they claim “reveals an external driver” …
In any case, when I put all of those problems together, the changes in variability in 1979, in 2000, and in 2009, plus the demonstrated non-stationarity pre-1979, plus the indirect evidence from the GISST and Reynolds datasets, plus the problems with the satellites affecting the critical recent period, the period they claim is statistically significant in their analysis … well, given all that I’d say that the N&M2012 method (comparing variability pre- and post-1979) is totally inappropriate given the available data. There are far too many changes and differences in variability, both internal to and between the datasets, to claim that the 1979 change in variability means anything at all … much less that it reveals an “external driver” for the changes in Arctic sea ice.
w.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Willis, Einstein never showed that Newton was wrong, Newton is correct, has been correct for a long time. We use Newton’s laws for almost all non-relativistic problems – celestial mechanics, fluid mechanics, etc. Einstein found exceptions or modifications for Newton’s law, under certain conditions – such as for objects with very high velocities. I can also talk about Wallace, but let us get to the point. If you are correct and if you are explaining to a climate scientist, who believes in AGW, why he is wrong in great detail using data available to all, and if he still cannot catch what you are saying and why his theory is wrong even after extensive discussions, I would say he is incompetent in this area – he may not know it. On the other hand if AGW is true and after so many explanations if the AGW skeptics cannot understand what the climate scientist who believes in AGW is saying, I would say the AGW skeptics are either incompetent in this area or dishonest because of some bias. We are not talking about generational conflicts in scientific views, sure those who believed that the earth is flat used all their logic with the available evidence to get to that conclusion, that was the best conclusion at that time – I do not consider them to be incompetent at all. But at a specific time in history when new evidence is presented to them that they also can verify to show that earth is not flat, and if they are unable to understand it or refuse to believe it, then they are incompetent in this particular area. There is no fallacy. I was using the same analogy here for climate scientists.
I understand where you take an issue (even though I never stated anything in that area) that science is not settled by consensus. My question, as I asked before which you have not addressed, how do you settle science? When there are disagreements on conclusions in most scientific areas (read this for example on a settled theory: http://www.nytimes.com/2012/05/17/health/research/hdl-good-cholesterol-found-not-to-cut-heart-risk.html ), you can evaluate some of the available literature in that area (I am sure you cannot evaluate everything) and you can come to your own conclusion. I am sure the 98 % scientists came to their own conclusions. My question is, why your conclusion is more reliable to yourself and others than other conclusions? Are you suggesting that truth is essentially a personal truth? I am reading Willis’s article and that of some famous climate scientist with a long publication record in this area – if there is significant difference between yours and his conclusions, please give me a reason why I should believe you as opposed to the other fellow? I am not a willing subscriber of science by consensus, but I have not found a good alternative either. If you can give me a good answer here, I would be very happy that I posted that passing short comment that dragged me into this debate.
There are not 98% of scientists who believe in Catastrophic Anthropogenic Global Warming.
That number is a complete fabrication. There is no empirical data to support that number. It is falsehood. It is propaganda.
That’s the problem with authoritarianism. The “authorities” lie. All the time. Case in point: the 98% lie.
Why do you bow and scrape to authoritarians, robgee? What’s in it for you? Why do you perpetuate their lies? Big Brother is not going to reward you. You haven’t even got the integrity to identify yourself.
I think that since you are hiding your identity, you have much more to hide as well. Dark secrets, eh, robgee?
That’s another trait of authoritarians. They have perversities. Often very evil ones. Which is why they sneak around. You’re a sneak, robgee. Must be something you’re hiding. What is it, robgee?
Rob G. says:
May 17, 2012 at 11:25 am
Rob, at the risk of repeating myself, let me repeat myself:
w.
PS—You say:
Fallacy of the excluded middle again. You should not believe either of us. Science is not about belief.
Wiilis, I have repeated this question few times, I see you do not want to answer – or you are unable to answer (is there an excluded middle here?). You keep telling me what science is not, but you have not told what science is, although I asked that few times. Let us say science is not consensus, then what is it really?
Willis said: PS—You say: “Fallacy of the excluded middle again. You should not believe either of us. Science is not about belief.”
I am appalled at your logical interpretations. In this hypothetical case I took ONE of your papers (let us say, this one states global warming is untrue) and ONE from a leading climate scientist who is opposing your position (saying global warming is true), and I asked you to help me to pick the correct position. One of those views has to be true, there is no excluded middle here. But I think you do not have a guideline or rule as an answer to my question so you go back to this misinterpretation of your list of fallacies again. I expected a lot more, from all the credentials you have claimed – at least please read more on real logic, starting with let us say, counterfacturals (http://web.mit.edu/holton/www/courses/freewill/counter.pdf )?
Everything at some level is a belief, but let us not worry about that now. I DID NOT say science is belief in my previous post (you are assigning too many opinions to me that I have not stated as mine). But assume that I am not a scientist and I do not know anything about thermodynamics or kinetics, and I am faced with two opposing views and two papers – I can understand parts of both papers, but not all. Then I have to pick and believe one of those sides, since I do not know how to analyze all those data to pick one side. This is where the belief comes, not as science, but as my option between two sides. Majority of our population is like this hypothetical me, without deep knowledge in scientific disciplines.
Mike Dubrasich is a lot more straight forward than you – he did not have any trouble with the excluded middle. Between incompetence and dishonesty, he easily picked dishonesty for scientists – he says “the 98% lie”. No consensus here between you two.
Mike Dubrasich says: “There are not 98% of scientists who believe in Catastrophic Anthropogenic Global Warming. …….I think that since you are hiding your identity, you have much more to hide as well. Dark secrets, eh, robgee?”
Thank you very much, at least you have not designated me as one of the cockroaches, as you did last time.
Now, am I hiding something since I did not expand my name? Let us see…. let us look at the names of people who posted above, Manfred, peter, Len, P. Solar, Juraj V, Andrew, The Infidel, Old England, Kasuha, mfo, BioBob, a reader, Jim G, JR, dh7fb, Larry, ……. shall I go on? Many of them are AGW skeptics. You do not worry about them hiding their identify, but this is different. There is a possibility that I could be Kevin Trenberth trying to give trouble to Willis. I may have to hide all my dark secrets.
98 % are not believers of Catastrophic Anthropogenic Global Warming, they believe Anthropogenic Global Warming. Let us not exaggerate, please.
BS robgee. Falsehood again. You pulled the 98% number out of a dark hole. You can’t stop lying. Something about you. A glaring lack of integrity.
Expand your name? No robgee. Identify yourself. Be real, be honest, stand behind your words like a man.
But you are not a man, are you robgee? You are a coward. A pissant. A crackhead.
Right? On the buzz. A junky. Typical nazi perv. We get trolls like you all the time. The dregs of society. A looter. A leach on your fellow man. On the dole. Got nothing going on, so you invade the spaces of others to give yourself some partial sense of worth.
It won’t work, robgee. We’ve got your number. We see through you. We know what your are. A pathetic creep. Aqualung. Prowling the alleys for whatever evil you can find. A waste of a human life. A danger to others.
Bye bye, robgee. We have work to do. No time to parry with a junky in the shadows.
Rob G. says:
May 17, 2012 at 7:10 pm
Rob, perhaps I was not clear when I asked the following … three times:
You said you’d come back to discuss it but you didn’t have time
Since then, despite your claimed lack of time due to your demanding work, you’ve spent pages and pages and lots of time doing nothing but going on and on endlessly about anything but the scientific aspects of my work. Let me make this clear.
I don’t care about your theories of science. I am totally uninterested in your ideas about logical fallacies. I couldn’t care less about your brilliant insights into the manifold advantages of scientific consensus.
Thus far, you have said nothing about my work. You have not raised a single objection to my claims in the head post. You have not criticized my math, my logic, or my data. Instead, you want to talk about consensus, logical fallacies, the theory of science, hypothetical questions, theoretical dilemmas, anything but the actual subject under consideration which you treat like a vampire treats garlic … and frankly, Scarlett, I don’t give a damn.
w.
Willis Eschenbach says: “I don’t care about your theories of science. I am totally uninterested in your ideas about logical fallacies. I couldn’t care less about your brilliant insights into the manifold advantages of scientific consensus……..”
Willis, I was not the one who brought up these topics in the first place, you did – logical fallacies, consensus, , etc.. you brought them up. My first comment was a very short comment – as I repeatedly said, I was not looking for a debate. But I got one, since you initiated it. Then in the process of defending your criticisms on fallacies, science, etc, I expanded your assertions to show you are completely wrong. You accused me of committing logical fallacies, I did not, I only showed that you are wrong in your claims. But you brought them up to gain an advantage in the debate, I defended to prevent that.
There is really no science in the head post, it is statistics and curve fitting, as I said before. When I see a future scientific post from you, I might pick that up. But from all I see, you are more interested in looking at the trees than the forest as a whole.
Again, I see what you think science is not, I have not seen anything on what science actually is.
Mike Dubrasich says: “BS robgee. Falsehood again. You pulled the 98% number out of a dark hole. You can’t stop lying. Something about you. A glaring lack of integrity. Expand your name? No robgee. Identify yourself. Be real, be honest, stand behind your words like a man. But you are not a man, are you robgee? You are a coward. A pissant. A crackhead. Right? On the buzz. A junky. Typical nazi perv. We get trolls like you all the time. The dregs of society. A looter. A leach on your fellow man. On the dole. Got nothing going on, so you invade the spaces of others to give yourself some partial sense of worth.”
Wow…. Oh my… A pissant? I guess that is worse than a cockroach. I have never seen a collection of name calling such as this before. Very original!!!!
“You pulled the 98% number out of a dark hole.” No Mike, everything I wrote here has a reference. Check the PANS paper for example on this.
” Expand your name? No robgee. Identify yourself.”, OK – I guess you want my phone number, SS# too, it seems.
I am starting to really enjoy your posts.
I finally scanned through the Notz and Marotzke paper, your post and all other comments, and I have to disagree with you on your basic premise. We can and we often do compare different data sets based on different measurement procedures to study a substance or a phenomenon. No one will keep the same instrument or measurement methodology for ever (for example, modern methods like XPS took over the old chemical methods to do elemental analysis), so such data set merging is necessary. Researchers do such things for time scales (which is the case here) or length scales (matching different length scales to incorporate smaller length scale phenomenon in larger length scales, like coupling molecular jumps which occurs in Angstrom length scale to a diffusion coefficient in the continuum scale). Of course the calibration and data matching at the overlap domain has to be done carefully, there are problems with resolution/sensitivity of measurements, etc., so it is not so easy to match them, but that does not mean it cannot be done. They are not like apples and oranges,as you say. Peter showed a good example, there are plenty of other examples, I also agree with Rob Dekker’s criticism – which makes perfect sense. Moreover, even if you are correct, I am not sure how your criticisms affect the conclusions in Notz’s paper, which are based on trends, not on variability (they said, “for such an analysis it is instructive to split the satellite record up into two components: a component that is based on the significant negative trend that we have described in section 3, and a component caused by internal variability [cf. Serreze et al., 2007]. Splitting up the satellite record accordingly gives a standard deviation ssat, detrend = 0.35 .10^6 km^2, virtually identical to the value of the pre-satellite record spresat = 0.36 10^6 km^2 ……..”. They also addressed PDO stating “The indices of the Pacific Decadal Oscillation (PDO) (Figure 4d) and of the Arctic Oscillation (AO) (Figure 4e) show only a very weak direct impact on the observed sea-ice retreat”. But your disagreement was on the variability.
Although I would like to plot the graphs at the overlap time to see how they matched the two data sets, I do not have enough time to do that. Besides, as you wrote, your criticisms did not arise from detailed statistical analysis, it was mainly based on your disagreements with data set merging. I disagree with that premise (I have done such merging in chemistry, and I belive it can be safely done in other areas as well), so going through the procedure in detail will only illuminate some arithmetical procedural errors, rather than some underlying scientific rules.
Also, Ernst Beck has done almost the same matching (“I compared the chemical measurements of atmospheric CO2, represented by the Steinhauser series for 1957/58 in Vienna (Austria), with the IR measurements at Mauna Loa. The average for chemical data for Vienna was 320 ppmv by chemical methods, and this is similar to the 318 ppmv obtained from Mauna Loa. This proves the validity of both types of measurement within the documented error range.” in http://www.biomind.de/treibhaus/180CO2/author_reply9-2.pdf ), and in spite of all the problems in the paper (http://www.biomind.de/nogreenhouse/daten/EE%2018-2_Beck.pdf ), it was a sensational work for the skeptics. So I do not understand the selective objections.