Guest post by Willis Eschenbach
Inspired by this thread on the lack of data in the Arctic Ocean, I looked into how GISS creates data when there is no data.
GISS is the Goddard Institute for Space Studies, a part of NASA. The Director of GISS is Dr. James Hansen. Dr. Hansen is an impartial scientist who thinks people who don’t believe in his apocalyptic visions of the future should be put on trial for “high crimes against humanity”. GISS produces a surface temperature record called GISTEMP. Here is their record of the temperature anomaly for Dec-Jan-Feb 2010 :
Figure 1. GISS temperature anomalies DJF 2010. Grey areas are where there is no temperature data.
Now, what’s wrong with this picture?
The oddity about the picture is that we are given temperature data where none exists. We have very little temperature data for the Arctic Ocean, for example. Yet the GISS map shows radical heating in the Arctic Ocean. How do they do that?
The procedure is one that is laid out in a 1987 paper by Hansen and Lebedeff In that paper, they note that annual temperature changes are well correlated over a large distance, out to 1200 kilometres (~750 miles).
(“Correlation” is a mathematical measure of the similarity of two datasets. It’s value ranges from zero, meaning not similar at all, to plus or minus one, indicating totally similar. A negative value means they are similar, but when one goes up the other goes down.)
Based on Hansen and Lebedeff’s finding of a good correlation (+0.5 or greater) out to 1200 km from a given temperature station, GISS show us the presumed temperature trends within 1200 km of the coastline stations and 1200 km of the island stations. Areas outside of this are shown in gray. This 1200 km. radius allows them to show the “temperature trend” of the entire Arctic Ocean, as shown in Figure 1. This gets around the problem of the very poor coverage in the Arctic Ocean. Here is a small part of the problem, the coverage of the section of the Arctic Ocean north of 80° North:
Figure 2. Temperature stations around 80° north. Circles around the stations are 250 km (~ 150 miles) in diameter. Note that the circle at 80°N is about 1200 km in radius, the size out to which Hansen says we can extrapolate temperature trends.
Can we really assume that a single station could be representative of such a large area? Look at Fig.1, despite the lack of data, trends are given for all of the Arctic Ocean. Here is a bigger view, showing the entire Arctic Ocean.
Figure 3. Temperature stations around the Arctic Ocean. Circles around the stations are 250 km (~ 150 miles) in diameter. Note that the area north of 80°N (yellow circle) is about three times the land area of the state of Alaska.
What Drs. Hansen and Lebedeff didn’t notice in 1987, and no one seems to have noticed since then, is that there is a big problem with their finding about the correlation of widely separated stations. This is shown by the following graph:
Figure 4. Five pseudo temperature records. Note the differences in the shapes of the records, and the differences in the trends of the records.
Curiously, these pseudo temperature records, despite their obvious differences, are all very similar in one way — correlation. The correlation between each pseudo temperature record and every other pseudo temperature records is above 90%.
Figure 5. Correlation between the pseudo temperature datasets shown in Fig. 3
The inescapable conclusion from this is that high correlations between datasets do not mean that their trends are similar.
OK, I can hear you thinking, “Yea, right, for some imaginary short 20 year pseudo temperature datasets you can find some wild data that will have different trends. But what about real 50-year long temperature datasets like Hansen and Lebedeff used?”
Glad you asked … here are nineteen fifty-year long temperature datasets from Alaska. All of them have a correlation with Anchorage greater than 0.5 (max 0.94, min 0.51, avg 0.75). All are within about 500 miles of Anchorage. Figure 6 shows their trends:
Figure 6. Temperature trends of Alaskan stations. Photo is of Pioneer Park, Fairbanks.
As you can see, the trends range from about one degree in fifty years to nearly three degrees in fifty years. Despite this huge ~ 300% range in trends, all of them have a good correlation (greater than +0.5) with Anchorage. This clearly shows that good correlation between temperature datasets means nothing about their corresponding trends.
Finally, as far as I know, this extrapolation procedure is unique to James Hansen and GISTEMP. It is not used by the other creators of global or regional datasets, such as CRU, NCDC, or USHCN. As Kevin Trenberth stated in the CRU emails regarding the discrepancy between GISTEMP and the other datasets (emphasis mine):
My understanding is that the biggest source of this discrepancy [between global temperature datasets] is the way the Arctic is analyzed. We know that the sea ice was at record low values, 22% lower than the previous low in 2005. Some sea temperatures and air temperatures were as much as 7C above normal. But most places there is no conventional data. In NASA [GISTEMP] they extrapolate and build in the high temperatures in the Arctic. In the other records they do not. They use only the data available and the rest is missing.
No data available? No problem, just build in some high temperatures …
Conclusion?
Hansen and Lebedeff were correct that the annual temperature datasets of widely separated temperature stations tend to be well correlated. However, they were incorrect in thinking that this applies to the trends of the well correlated temperature datasets. Their trends may not be similar at all. As a result, extrapolating trends out to 1200 km from a given temperature station is an invalid procedure which does not have any mathematical foundation.
[Update 1] Fred N. pointed out below that GISS shows a polar view of the same data. Note the claimed coverage of the entirety of the Arctic Ocean. Thanks.
[Update 2] JAE pointed out below that Figure 1 did not show trends, but anomalies. boballab pointed me to the map of the actual trends. My thanks to both. Here’s the relevant map:
OT – congrats!
“A small group of dedicated people coming from a diverse range of positions and perspectives but working together as a loose federation held together by shared values and beliefs succeeded in accomplishing the most impressive PR coup of the 21st century.”
http://www.profero.com/unsimplify/index.html
[Profero]
“So how did this group of bloggers succeed in bringing the climate establishment to its knees…”
http://curry.eas.gatech.edu/climate/towards_rebuilding_trust.html
[Judith Curry]
Quick sip of Champaign and back to work!
These fools thought they would push through a get rich quick scheme / pay the mortgage in quick time etc., but it’s unravelling fast now.
I use Rsq all the time for seeking correlation (but of course I am a “hard” scientist by profession, and not a climate “scientist”). We usually dismiss as unreliable any Rsq less than 0.8. This means an R LESS THAN 0.9 is Bogus, man!
For Hansen et.al. – Solution: don’t waste our time with inter- or extra-polations. Just make more measurements, stupid (McubedS) or go back to sleep!
“So how did this group of bloggers succeed in bringing the climate establishment to its knees…”
Answer: by persistantly presenting contrary evidence despite MASSIVE public funding of their detractors. The ‘truth’ always wins out in the end.
If the facts change I’ll change my mind. Until then I remain a former believer and now a sceptic.
Dr. Hansen is an impartial scientist
On what planet would this be believed?
@Dave Springer (16:46:32) :
I think Pielke is right on the money in this article:
http://wattsupwiththat.com/2009/08/21/soot-and-the-arctic-ice-%E2%80%93-a-win-win-policy-based-on-chinese-coal-fired-power-plants%E2%80%9D/
No source of black soot can make it to the south pole which handily explains why the antarctic interior isn’t warming.
This is an interesting consideration when you realize that sea ice “strength” depends greatly on chemical composition. In fact, the salinity tends to turn sea ice into a less-than-solid-and-easily-broken-up mush as compared to ice on frozen freshwater sources. I’m beginning to wonder if any chemical analysis of typical sea ice has been done to rule out possible changes in chemistry from pollution sources as a contributing cause of the arctic sea ice decline.
Refer to: http://www.mpimet.mpg.de/fileadmin/staff/notzdirk/Notz_2009_JGR.pdf
It’s amazingly warm in all those areas where they no longer actually measure temperatures.
Frankly, the conclusion seems obvious: given the exceedingly sparse temperature data sets for the Arctic, the construction of forecasts based on this data can only be regarded (at best) as being ‘unreliable’.
The more reasonable position to take would be: “on the basis of data that we currently have, we do not know what the recent temperature trends in the Arctic have been. Nor can we say much about what they will be.”
In the absence of real data to examine, argumentation must rely upon anecdote, hubris, and blind faith.
…..Figure 2. Temperature stations around 80° north. Circles around the stations are 250 km (~ 150 miles) in diameter. Note that the circle at 80°N is about 1200 km in radius, the size out to which Hansen says we can extrapolate temperature trends……
……Can we really assume that a single station could be representative of such a large area?……
……Figure 3. Temperature stations around the Arctic Ocean. Circles around the stations are 250 km (~ 150 miles) in diameter. Note that the area of the Arctic Ocean is about three times the area of the state of Alaska….
…..they extrapolate and build in the high temperatures in the Arctic……
………………………………………………………………………………………………………..
This is unfair and everyone should know this is being done.
If James Hansen feels he is so correct to do this with the data then he himself should be proud to tell the world about his method.
Everyone should be made aware that ‘global warming’ is being manufactured! And manufactured by someone working at ‘NASA’ no less!
I’m just beginning to read the comments, so the following matter may already have been covered, but….. In the following sentence, I believe that the average should be 0.75, not .075.
“Glad you asked … here are nineteen fifty-year long temperature datasets from Alaska. All of them have a correlation with Anchorage greater than 0.5 (max 0.94, min 0.51, avg .075)”
IanM
Why can’t the data from the IABP bouys? http://psc.apl.washington.edu/northpole/ there’s always one not far from the pole every time I look…
Willis:
Others may have already got this but the way I see what you have shown is as follows:
Let’s assume that X11 is the temperature at station 1 at time 1 and X21 is the temperature at station 2 at time 1. Let Y11 be the unknown temperature at a location 1200 KM from station 1 and Y21 the unknown temperature at a location 1200 KM from station 2.
Lets grant that all possible pairs are highly correlated, i.e., Corr( X1,X2), Corr (X1,Y1), Corr(X2,Y2) , Corr(X1, Y2), Corr(X2,Y1) and Corr(Y1, Y2) are all greater than 0.9 .
Since we know the actual temperatures at X1 and X2, we can then say that
X1 = bX2 , where b is the empirically determined coefficient that allows us to determine the temperature at X1 for any temperature at X2 and is equivalent to the slope of the line between pairs of station temperatures.
In other words if a temperature at X1 was missing we could accurately estimate the missing temperature if we knew X2. All this is great. However when we come to write the equations for Y1 and Y2 we can say Y1 = cX1 and Y2 = dX2. BUT in order to fix the anomaly temperatures for Y1 and Y2 you must know the values of c and d. The trend lines you show suggest that there is no reason to believe that b = c = d and that somehow Hansen must come up with values for b and c in order to actually fix the temperatures at Y1 and Y2.
You have neatly reminded everybody that Hansen et al have no basis for determining the actual temperatures at Y1 and Y2 by demonstrating that “b” is not a constant across pairs of Arctic stations. A high correlation actually only tells you the level of accuracy you can have in predicting Y from X not the actual level of Y for a change in X.
For Hansen to infill he must have some basis for determining “c” and “d” which as you show could vary widely. I see no way that he can do this.
Anu (16:32:13) :
Anu,
I see you are fulfilling the role of the mindless advocate by playing the cigarette smoke card.
You have pigeon holed yourself.
Will you soon be playing the holocaust denial card? Or did you do it already and I overlooked it? If you have then I offer my apologies for short-changing you.
“extrapolating trends out to 1200 km from a given temperature station”
and
“The big problem is with the GISS baseline period, which was largely a period of unusual cold, so the anomaly maps always look warm.”
This is one of many instances of questionable data handling that makes many people skeptical of the theory of AGW. As a prior commenter said, to the casual observer, all of that red infill on the map creates the strong impression of warmth and warming. “Look at all of that red at the Arctic! They must be right about the planet warming.”
The other item that contributes to scepticism is an even bigger problem: the theory has already been disproven. The theory of AGW had predicted that the global temperature should have risen by 3.8 degrees F by now; it has only risen by 1.4 degrees F.
http://www.sciencedaily.com/releases/2010/01/100119112050.htm
Steve Goddard (17:12:47) :
The big problem is with the GISS baseline period, which was largely a period of unusual cold, so the anomaly maps always look warm.
………………………………………………………………………………………………….
And of course we’ve all been told the ‘real climatologists’ around the world only use anomaly.
Or did they mean ‘RealClimate’ologists?
I am no expert on statistics -it is too many years since I was using T and F tests for significance. However, I thought that a correlation was only significant if greater than 0.9 (or less than -0.9). A correlation factor below 0.66 is not significant to determine cause and effect and should not be used for infilling or extrapolating data. In this case GISS is extrapolating temperatures measured in warmer areas to a colder area without correcting for latitude direction which makes it even more wrong. It is surprising that some of the more honest amongst the climate pseudo-scientists have not questioned the practice. Maybe they are frightened of tackling Hansen and ilk. The data in Hansen’s paper shows large temperature deviations in the higher latitudes which overwhelm any trend.
Good work Willis
Funny thing is that I was doing some cells imaging today, got a correlation coefficient of 0.89 for mitochondria function and reactive oxygen species; looked at the scatter and so am going to look at the bimodal population statistics. A correlation coefficient of <0.75 generally means that you aren't looking at a direct relationship. I would rather like to see a plot of correlation coefficient vs distance for all of those stations shown in the figure, that are within 1,200 km of each other. Draw a circle around each station and do the cc for each of the stations (NSEW) that are within 1,200 km, then remove it and move to the next one.
What does ‘The Reference Frame’ think about p = 0.05 ?
A leader is needed to bring all the ‘balls’ together; politically neutral but scientifically literate.Someone one can trust.
Step forward Prof. Richard Lindzen
At the very least, demand (somehow) a debate on MS tv.
A rally of 10,000 voices at the Albert Hall, London.
Serious damage is about to be inflicted on US and European economies.
Time is short.
Come on all you pro. bloggers on both sides of the pond.
Communicate and lead. (I feel like John the Baptist).
It reads a bit extreme but I’m quite serious.
Willis Eschenbach (17:18:35) :
Interestingly, Anu’s arguments are therefore not irrelevant. They are, in fact extremely relevant. Just not in the way Anu intended, of course: “It’s a travesty” … hide the decline … rather delete the data than … etc, etc, etc.”
Why use the words “Temperature co-relation” when there are perfectly good four letter Anglo-Saxon words that better describe it.
I’m still not sure I understand what GISS actually did. Where did they establish their r’s? Surely they didn’t get their correlation coefficients using data anywhere but within the Arctic circle? A distance of 1200 km is huge, in terms of climate variability, especially if you’re extrapolating. Worse, if GISS is extrapolating pole-wards and using linear fits, they’re really on thin ice. The pole is a geographic and climactic discontinuity. There’s little reason to expect linearity unless Hansen derived his fit using actual data at the pole. If not, then I’d say Dr. Hansen is just guessing and doesn’t know his r’s.
Eisenbach: “Dr. Hansen is an impartial scientist”
It’s always Marcia Marcia: “On what planet would this be believed?”
You missed the sarcasm. Willis meant it as such….
Chris
Norfolk, VA, USA
Cement a friend (18:51:18) : “…A correlation factor below 0.66 is not significant to determine cause and effect…”
NO correlation factor, even 1.0000, is sufficient to determine cause and effect. Correlation is not causation.
My basic stats book said to be wary of any linear correlations less that 0.8. Isn’t 0.5 basically a coin-toss as to the degree of association?
Willis:
You need to show pairs of Arctic stations where their temperatures create a regression line with very different slopes. In reality so long as the correlation coefficients are greater than 0.7, Hansen can pretty much do his extrapolation if he can assume the slope of the regression line. However, the slopes of the regression lines must essentially be the same in order to actually generate the temperature anomalies for the uncharted space. If they are different then I have no idea how Hansen can do his extrapolation. Your charts suggest (though they do not prove) that there is likely to be considerable differences in the slopes of these curves. For example, pairing an open water coastal location with ice bound coastal location should produce a very different slope than two open water or two ice bound locations.
“It is hard to make data where none exist.” –Kevin Trenberth