Shakun, Not Stirred, and Definitely Not Area-Weighted

I’d like to highlight one oddity in the Shakun et al. paper, “Global warming preceded by increasing carbon dioxide concentrations during the last deglaciation” (Shakun2012), which I’ve discussed here and here. They say:

The data were projected onto a 5°x5° grid, linearly interpolated to 100-yr resolution and combined as area-weighted averages.

The oddity I want you to consider is the area-weighting of the temperature data from a mere 80 proxies.

Figure 1. Gridcells of latitude (North/South) and longitude (East/West)

What is area-weighting, and why is it not appropriate for this data?

“Area-weighting” means that you give more weight to some data than others, based on the area of the gridcell where the data was measured. Averaging by gridcell and then area-weighting attempts to solve two problems. The first problem is that we don’t want to overweight an area where there are lots of observations. If some places have 3 observations and others have 30 observations in the same area, that’s a problem if you simply average the data. You will overweight the places with lots of data.

I don’t like the usual solution, which is to use gridcells as shown in Figure 1, and then take a distance-weighted average from the center of the gridcell for each gridcell. This at least attenuates some of the problem of overweighting of neighboring proxies by averaging them together in gridcells … but like many a solution, it introduces a new problem.

The next step, area-averaging, attempts to solve the new problem introduced by gridcell averaging. The problem is that, as you can see from Figure 1, gridcells come in all different sizes. So if you have a value for each gridcell, you can’t just average the gridcell values together. That would over-weight the polar regions, and under-weight the equator.

So instead, after averaging the data into gridcells, the usual method is to do an “area-weighted average”. Each gridcell is weighted by its area, so a big gridcell gets more weight, and a small gridcell gets less weight. This makes perfect sense, and it works fine, if you have data in all of the gridcells. And therein lies the problem.

For the Shakun 2012 gridcell and area-averaging, they’ve divided the world into 36 gridcells from Pole to Pole and 72 gridcells around the Earth. That’s 36 times 72 equals 2592 gridcells … and there are only 80 proxies. This means that most of the proxies will be the only observation in their particular gridcell. In the event, the 80 proxies occupy 69 gridcells, or about 3% of the gridcells. No less than 58 of the gridcells contain only one proxy.

Let me give an example to show why this lack of data is important. To illustrate the issue, suppose for the moment that we had only three proxies, colored red, green, and blue in Figure 2.

Figure 2. Proxies in Greenland, off of Japan, and in the tropical waters near Papua New Guinea (PNG).

Now, suppose we want to average these three proxies. The Greenland proxy (green) is in a tiny gridcell. The PNG proxy (red) is in a very large gridcell. The Japan proxy (blue) has a gridcell size that is somewhere in between.

But should we give the Greenland proxy just a very tiny weight, and weight the PNG proxy heavily, because of the gridcell size? No way. There is no ex ante reason to weight any one of them.

Remember that area weighting is supposed to adjust for the area of the planet represented by that gridcell. But as this example shows, that’s meaningless when data is sparse, because each data point represents a huge area of the surface, much larger than a single gridcell. So area averaging is distorting the results, because with sparse data the gridcell size has nothing to do with the area represented by a given proxy.

And as a result, in Figure 2, we have no reason to think that any one of the three should be weighted more heavily than another.

All of that, to me, is just more evidence that gridcells are a goofy way to do spherical averaging.

In Section 5.2 of the Shakun2012 supplementary information, they authors say that areal weighting changes the shape of the claimed warming, but does not strongly affect the timing. However, they do not show the effect of areal weighting on their claim that the warming proceeds from south to north.

My experiments have shown me that the use of a procedure I call “cluster analysis averaging” gives better results than any gridcell based averaging system I’ve tried. For a sphere, you use the great-circle distance between the various datapoints to define the similarity of any two points. Then you just use simple averaging at each step in the cluster analysis. This avoids both the inside-the-gridcell averaging and the between-gridcell averaging … I suppose I should write that analysis up at some point, but so many projects, so little time …

One final point about the Shakun analysis. The two Greenland proxies show a warming over the transition of ~ 27°C and 33°C. The other 78 proxies show a median warming of about 4°C, with half of them in the range from 3° to 6° of warming. Figure 3 shows the distribution of the proxy results:

Figure 3. Histogram of the 80 Shakun2012 proxy warming since the most recent ice age. Note the two Greenland ice core temperature proxies on the right.

It is not clear why the range of the Greenland ice core proxies should be so far out of line with the others. It seems doubtful that if most of the world is warming by about 3°-6°C, that Greenland would warm by 30°C. If it were my study, I’d likely remove the two Greenland proxies as wild outliers.

Regardless of the reason that they are so different from the others, the authors areal-weighting scheme means that the Greenland proxies will be only lightly weighted, removing the problem … but to me that feels like fortuitously offsetting errors, not a real solution.

A good way to conceptualize the issue with gridcells is to imagine that the entire gridding system shown in Figs. 1 & 2 were rotated by 90°, putting the tiny gridcells at the equator. If the area-averaging is appropriate for a given dataset, this should not change the area-averaged result in any significant way.

But in Figure 2, you can see that if the gridcells all came together down by the red dot rather than up by the green dot, we’d get a wildly different answer. If that were the case, we’d weight the PNG proxy (red) very lightly, and the Greenland proxy (green) very heavily. And that would completely change the result.

And for the Shakun2012 study, with only 3% of the gridcells containing proxies, this is a huge problem. In their case, I say area-averaging is an improper procedure.

w.

Advertisements

  Subscribe  
newest oldest most voted
Notify of

The data were projected onto a 5°x5° grid, linearly interpolated to 100-yr resolution and combined as area-weighted averages.
Where do they say that they weight with the area of each grid cell? The way I would weight would be to divide the globe into a number of equal-area pieces [not the grid cells, obviously, as they don’t have equal area] and then calculate the average value of a proxy by computing the average of the grid cells that fall into each equal area piece, then average all the pieces. This is the standard [and correct] way of doing it. Why do you think they didn’t do it this way?

The two Greenland proxies show a warming over the transition of ~ 27°C and 33°C. The other 78 proxies show a median warming of about 4°C
Could it be the the Greenland proxies are closer to polar, and others more tropical? I’ve not followed where the proxies originate, but I suspect a warming vs latitude chart might show something.

Leif Svalgaard says:
April 9, 2012 at 11:28 am
then calculate the average value of a proxy by computing the average of the grid cells
One could argue if that average should be weighted by the area of each grid cell. I’m inclined not to, but in any event the variation of the grid cell areas with each ‘piece’ would be small, so it may not matter.

the grid cell areas within each ‘piece’ would be small, so it may not matter

W. ,
I just thought of a way that CO2 could lead temp during a transition.
Latent Heat. When ice is melting, temperature doesn’t rise. CO2 would be released though.
You would have an initial warming without much CO2 release (pretty much just where water is warmed). As older ice melts, CO2 would be released in greater amounts. Because of the large volume of melt, temperature would not rise much since a very large amount of heat would be used to break molecular bonds. As the rate of melt decrease, temp would begin to rise faster.

Willis Eschenbach

Leif Svalgaard says:
April 9, 2012 at 11:28 am

The data were projected onto a 5°x5° grid, linearly interpolated to 100-yr resolution and combined as area-weighted averages.

Where do they say that they weight with the area of each grid cell? The way I would weight would be to divide the globe into a number of equal-area pieces [not the grid cells, obviously, as they don’t have equal area] and then calculate the average value of a proxy by computing the average of the grid cells that fall into each equal area piece, then average all the pieces. This is the standard [and correct] way of doing it. Why do you think they didn’t do it this way?

Hey, Leif, good to hear from you. You can do it with equal-area cells as you suggest. However, many folks (including apparently Shakun et al., since they don’t mention equal-area anywhere) just area weight by the area of the 5°X5° gridcells. For example, GISS uses equal-area gridcells, whereas HadCRUT uses area-weighted 5°X5° gridcells.
w.

Pull My Finger

Climate reconstructions are like picking three games over the 20+ seasons Ty Cobb played and determing his career batting average. You could state that Ty Cobb was worhtless, batting .000 for his career by picking three games where he was 0 for 4, or that he was a career .800 but didn’t play very often picking games where he went 4 for 4, 0 for 1, and one game where he didn’t play. We all know this is an absurd way to rate baseball players, but somehow perfectly acceptable to determine the economic fate of the world.

Joe Public

Willis, some of the comments should be in degrees K.

xham

Wouldn’t a geodesic grid system be the way to go?

Pull My Finger

As a former GIS analyst and cartographer, I can state with grim authority that when it comes to spatial statistical analysis, the vast majority of people across all disciplines haven’t a clue as to what they are doing when it comes to studies like this one.

Willis Eschenbach says:
April 9, 2012 at 11:41 am
You can do it with equal-area cells as you suggest. However, many folks (including apparently Shakun et al., since they don’t mention equal-area anywhere) just area weight by the area of the 5°X5° gridcells.
How do you know that they just weight by grid cells? If I had written the paper, I would also not have elaborated on how to weight by area, as one would simply just do it correctly, so no need to dwell on the obvious, just say ‘area weighted’ implying the correct meaning.

JJ

Leif Svalgaard says:
This is the standard [and correct] way of doing it. Why do you think they didn’t do it this way?

Willis’ point seems to be that they did do it that way, but that it is not correct for them to have done so. For the reasons given, I concur.

BioBob

I have a hypothetical for you all.
Is there something wrong with admitting that there is not enough data nor is the data available of good enough quality to draw the feverishly desired conclusions ? Is there something wrong with admitting that we just do not know all the answers or even most of them or even a few of them ?
“I can live with doubt and uncertainty and not knowing. I think it’s much more interesting to live not knowing than to have answers which might be wrong.” Richard Feynman

Phil Cartier

The big problem with this paper is that they don’t have a significant amount of data. If there is only data in 3% of the gridcells, any kind of averaging and projecting to the whole globe is meaningless, unless all the proxies have samples equally spaced around the globe. Even then, the data is just too sparse to be meaningful.

As I was reading the discussion I was thinking you need an equal area projection type of approach, and then I saw Dr. Svalgaard’s comment. I agree. I also question why anyone would attempt to extrapolate ± 80 points, that aren’t necessarily representing the same thing, to the entire globe in the first place. Shakun2012 proves nothing, demonstrates little except their questionable motives, and fails to apply the scientific method to a degree of rigor needed to qualify as being anything other than fiction, or wishful thinking.

jeanparisot

Gridding is major problem. They should be areas related by content, such as: %land, %ice, annual sunload, etc., with the related proxies used therein. At least use a cell based on the true earth shape, not a spherical map projection.
Has any of this been analyzed by spatial statisticians?

You know my feelings about this already. Using 5×5 lat/long cells is enormously stupid. I personally favor a rescalable icosahedral map — one that can be subdivided into enough grid cells that important geographic features — coastlines, mountain ranges, deserts, larger islands, forests, cities — are typically resolvable, so that a continental map done in cells is recognizable (and can be done at double the resolution easily).
That still doesn’t resolve the many other problems with Shakun — the utter dominance of coastal sites in a world the odds are at least 10 or 20 to 1 against a pin dropped at random onto the planet landing within 50 or 100 miles of a coast, for example — but it does handle the problem with cell weights a lot better.
I don’t think the climate scientists realize how important it is to address and resolve this geometric issue before they spend all of the time that they spend making blind assertions about “global average temperature”. Numerical integration on the surface of the sphere — which is what this is, whether or not they understand that — is difficult business, and one that it is almost impossible to carry out correctly in spherical polar coordinates. There is an entire literature on the subject, complete with alternative tilings and coordinatizations. But even using the Jacobean doesn’t fix spherical polar coordinates when one relies on a sparse grid of samples, let alone a sparse grid of samples biased by their geography.
For example, is there anyone who thinks that a coastal sample from Greenland (assuming one could generate such a thing from the same span of times) would in any way resemble an interior glacial ice core? Or that a glacial ice core from Greenland in any way would resemble a sample drawn from (say) Minnesota or Idaho or Alaska or Mongolia? Yet there are two samples from there and none from Minnesota, which is probably a far more typical inland environment.
rgb

thelastdemocrat

The geocoding / geospatial analysis people have excellent ways of addressing limited information across geography. They would not use a lat/long approach, but would use a mosaic approach, whih would look more like a soccer ball pattern.

‘Nul points’ au soleil !
http://www.vukcevic.talktalk.net/img3.htm

thelastdemocrat

There is another way to look at the issue of 80 data points. If you have a hypothesis, that increased CO2 leads to warming, you could grab one location and see if the data fit. That would not be very conclusive. If you find a reliable pattern across the majority of 80 sites, that would be fairly persuasive. This is what is being done with evidence for the MWP: people keep finding local evidence of a temp pattern fitting MWP.
I am not saying I support this study – just sayin’ that 80 data points, each taken individually as one bit of info, would be persuasive regarding the MWP pattern, and would be if local CO2-and-temp data fit a certain pattern.
No, 80 data points cannot hardly serve to represent global temp. You need decent sampling from each of the various regions – ocean/land, the various latitudinal regions, etc.

JJ says:
April 9, 2012 at 11:51 am
Willis’ point seems to be that they did do it that way
My point was that how do Willis know that?

Well, it looks like there are multiple proxies per grid cell.
So when they average per grid cell you will have some grid cells with say 2-3 measures while
other gridcells have only one measure. equal area weighting would not be correct either.
In any case looks like you get C02 going from about 180 to 260 and the temp response is 4C
lukewarmer.

What all of this Skakun and a Rattlin’ is really about is that they are (rather shakily) trying to prop up the basic meme hypothesis: CO2 causes warming. That clearly indicates that there is some rather nagging doubt in the AGW community about their holy null hypothesis…and that they are basically in a rush to trowel some gumbo over the rather large leak in their sinking vessel. Grab the trowel, Willis!

Mosher, isn’t a 4C response to a 45% increase in CO2 physically impossible?

Steven Mosher says:
April 9, 2012 at 12:24 pm
So when they average per grid cell you will have some grid cells with say 2-3 measures while
other gridcells have only one measure.

I would not see that as a problem. Suppose there were 10,000 measurements in a grid cell and 1 in a neighboring one. Since temperatures are strongly autocorrelated spatially, the 10,000 measurements may not be any better than the 1, so it would be OK to include both cells without taking into account that there are many more in one than in the other.

The authors also looked at unweighted data and a “meridional weighting” procedure described in the supplementary, and they also tested the degree to which 80 randomly selected sites in the instrumental record reflect the global average (pretty good). It’s easy to nitpick details or tell what you’d do differently but none of this changes the key conclusions, and they are in line with what some previous papers have argued.

Yes nitpick details like inverted tiljander and deleted proxies after 1960 never change the conclusions in the type of science Colose supports.

Jon

You have to weight for latitude and is suddendly the effect of CO2 linear and not longer logarithmic?

Chris Colose says:
April 9, 2012 at 12:35 pm
the key conclusions, and they are in line with what some previous papers have argued.
that is like arguing that smoking is healthy because so many people do it…

Sure, whatever…

Beam me up Scotty

It is odd that you are trying to use science to debunk science.

srvdisciple

Anthony, just a typo note : “Japan proxy(red)” should read “Japan proxy(blue)”

Eric Webb

Shakun’s paper just seems like more warmist crap to me, they should know that CO2 responds to tempertature, not the other way around. i wish we could conduct an experiment to prove that.This post made me wonder if NOAA uses similar tactics to average their data and make it look warm? I know they’ve removed many data stations from cold areas and moved them to cold areas. What also strikes me as odd is when they have random large red circles near the arctic
which are sometimes off by themselves or surrounded by much smaller circles, makes the data look suspect and it doesn’t look right. In other news, supposedly the US saw it’s warmest march on record, according to NOAA, I however, think otherwise, considering that NOAA doesn’t include the UHI effect in their results, and for their pro AGW views.

daved46

Two things.
First, your paragraph under figure 1 says the Japan proxy is red instead of the blue it actually is.
Second, since the point of the paper is showing whether CO2 or temperature leads, I’m not sure why it’s of much interest whether the averaging is done correctly or not. If changing the weights of the individual proxies by a factor of 2 or so can change the lead, trail order, then the data set is useless for deciding the question at hand. You need to go back and examine the individual proxies to see what their physical attributes are.

Two remarks, obviously you can average out a temperature trend, but can you average out timing of events? A few decades ago Wallie Broecker and friends already noted that the northern hemisphere warming was much later than the southern. So there is nothing new here, only that the first events was the start of the isotope peak in the antarctica proxies, before the CO2 records. So you can average out whatever but but none of this changes the key conclusion, that whatever happened, it started in the southern hemisphere and it was not CO2.
Secondly, good to note that the greenland ice cores are outliers. They are indeed and it greatly falsifies the idea of the isotopes in precipitation being a proxy for temperature. Instead, they are a proxy for dewpoint though (most basic meteorology) in combination with rain out rayleigh effect. Non calor sed umor, it”s not the heat but the humidity.

JDN

Gridding is probably inappropriate, period!
If you start by picking 80 surface stations around the world, many of them won’t correlate with a square or geodesic grid. As an example, this year in the USA, average temperatures will correlate well on a north-south line on the east coast thanks to so much air moving up the coast inland during the winter. In most years, however, temperature will correlate well along roughly east-west great circles, at least with the midwest temperatures. None of this correlation lies on a regular grid. How can any grid be justified for sparse data if you were to use actual temperature meaasurements, much less proxies?

George E. Smith;

Well wake me up when these gridders discover the theory of sampled data systems, and the Nyquist theorem. In the mean time; 80 samples of “anything” is just that; 80 samples of anything; and it isn’t necessarily DATA about ANYTHING.

Steve from Rockwood

I was thinking about how Shakun’s “averaging” worked the other day. My original concern was the high correlation in some areas (such as sites in Greenland) and the low correlation in other areas, both one site to the next closest and also mid-latitude sites with either Greenland or Antarctica. So the proxy series contain apples and oranges.
After reading the grid was 5 x 5 degrees I did a cross-plot of LAT versus LON from the Metadata. Just in the SH from -90 to -45 LAT there are 648 cells and only 6 data points with no more than one data point per cell. How can this be a global average if less than 1% of the cells in the SH have a single point?
In the NH it is almost as bad. Of the 648 cells from 45 to 90 degree LAT there are 9 data points.
So from 45 degrees to each pole there are 15 data points in almost 1300 cells.
To make matters worse, most of the data points are clustered around the same latitude (varying along longitude), especially near +30 and also the equator.
From 100 degrees west to 180 west from pole to pole (1440 cells at 5×5) there are 6 proxies.

Steve from Rockwood says:
April 9, 2012 at 1:11 pm
How can this be a global average if less than 1% of the cells in the SH have a single point?
since you have the data already tallied it would be nice if you could post a lat-long grid of the number of data points in each cell.

Steve from Rockwood:
If you have the data as a text file or an Excel file, send it to me leif@leif.org and I’ll graph it.

Leif Svalgaard says:
April 9, 2012 at 12:32 pm (Edit)
I would not see that as a problem. Suppose there were 10,000 measurements in a grid cell and 1 in a neighboring one. Since temperatures are strongly autocorrelated spatially, the 10,000 measurements may not be any better than the 1, so it would be OK to include both cells without taking into account that there are many more in one than in the other.
###############
That’s not the problem. The problem is when you have 10000 measures in one cell reading
20C and ONE measure in the next cell reporting 10C. A simple average gives you 15C.
Now, if the 10000 agree with the oneneighbor then averaging is not a problem. So, The method I use ( Nick stokes actually ) is inverse density weighted.

JDN
“If you start by picking 80 surface stations around the world, many of them won’t correlate with a square or geodesic grid. As an example, this year in the USA, average temperatures will correlate well on a north-south line on the east coast thanks to so much air moving up the coast inland during the winter. In most years, however, temperature will correlate well along roughly east-west great circles, at least with the midwest temperatures. None of this correlation lies on a regular grid. How can any grid be justified for sparse data if you were to use actual temperature meaasurements, much less proxies?”
The correlation length is a function of latitude and season. In the end whether you use regular gridding, verroni tesselation, EOFs, or kridging the answer comes out the same.
1. CRU: equal angle grid
2. GISS: equal area grid
3. Nick stokes: equal angle ( with inverse density) AND veronni tesselation
4. NCDC: EOFs
5. Berkeley Earth : kridging.
The answers given by each and every one of these approaches to averaging spatial data is……………… THE SAME.. ok.. differences the size of mousenuts.

Steven Mosher says:
April 9, 2012 at 1:27 pm
That’s not the problem. The problem is when you have 10000 measures in one cell reading
20C and ONE measure in the next cell reporting 10C. A simple average gives you 15C.

And that would be correct, as there is not much extra information in the 10,000 measure average. Imagine, you increase that to 1000,000,000 measurements with a thermometer every square meter, the average [20C] would not change significantly, yet the 10C data point would be swamped out by all those new measurements with no new information..

george, let us know when you discover spatial auto correlation.

Eric Webb says:
April 9, 2012 at 12:54 pm (Edit)
Shakun’s paper just seems like more warmist crap to me, they should know that CO2 responds to tempertature, not the other way around.
###################
Its actually BOTH. Added C02 will warm the earth and the ocean will respond by outgassing more C02.

Steven Mosher says:
April 9, 2012 at 1:36 pm
Its actually BOTH. Added CO2 will warm the earth and the ocean will respond by outgassing more CO2.
nice positive feedback loop there…

Chris Colose says:
April 9, 2012 at 12:35 pm (Edit)
The authors also looked at unweighted data and a “meridional weighting” procedure described in the supplementary, and they also tested the degree to which 80 randomly selected sites in the instrumental record reflect the global average (pretty good).
##############
ya it looks like nobody read the SI. Looking at the 80 sites they picked and the latitudinal distribution I would say the 80 locations they have would do pretty well. Then again, you are talking to some folks who think that Viking settlements represent the entire world and that frost fairs in England can reconstruct the temperature in australia.
Here is what you will find Chris. When a skeptic has one data point they like, they forget about the global average. When they have 80 they dont like, they crow about the small number.
60 optimally chosen site is enough. I’m not surprised they did well with 80.

Rogelio escobar

so does water, methane, nitrogen etc… excess heat is mostly lost to space, so its all c%%%, strongly recommend you read Lindzen, Spencer’s recent papers etc, who are actually atmospheric physicists BTW)

Andreas

Why 5°x5° grid boxes?
The answer is in the supplementary information (http://www.nature.com/nature/journal/v484/n7392/extref/nature10915-s1.pdf)
Shakun et al tested other methods and found that the methods lead to quite similar results, only the amplitude of warming differs about 0.7°C, which doesn’t influence their results (there’s a instructive graph in the suppl.).
More important, it’s a convenient choice, because they tested with present HadCrut temperature data, whether their proxy locations are representative for average global temperature (quite well). HadCru uses 5°x5° grids, so this choice was easy to compare.

pochas

Steven Mosher says:
April 9, 2012 at 1:34 pm
“Added C02 will warm the earth and the ocean will respond by outgassing more C02.’
Very good, Steven!

Gary Pearse

If you have a limited number of data points, you are going to have a poor temp estimation no matter what. Perhaps rather than grid cell averaging, it would be better to give each data point (possibly adjusted to sea level) a latitudinal band with the idea that temp roughly decreases with distance from the equator. It would be a lousy estimate but probably better than grid cell, weighted average one. The bands would be area weighted since they get shorter as you move toward the poles.