Guest Post by Willis Eschenbach
There is a new paper in Nature magazine that claims that the tropics are expanding. This would be worrisome because it could push the dry zones further north and south, moving the Saharan aridity into Southern Europe. The paper is called “Recent Northern Hemisphere tropical expansion primarily driven by black carbon and tropospheric ozone”, by Robert Allen et al. (paywalled here , supplementary information here , hereinafter A2012). Their abstract says:
Observational analyses have shown the width of the tropical belt increasing in recent decades as the world has warmed. This expansion is important because it is associated with shifts in large-scale atmospheric circulation and major climate zones. Although recent studies have attributed tropical expansion in the Southern Hemisphere to ozone depletion the drivers of Northern Hemisphere expansion are not well known and the expansion has not so far been reproduced by climate models. Here we use a climate model with detailed aerosol physics to show that increases in heterogeneous warming agents—including black carbon aerosols and tropospheric ozone—are noticeably better than greenhouse gases at driving expansion, and can account for the observed summertime maximum in tropical expansion.
Setting aside the question of their use of a “climate model with detailed aerosol physics“, they use several metrics to measure the width of the tropics—the location of the jet stream (JET), the mean meridional circulation (MMC), the minimum precipitation (PMIN), the cloud cover minimum (CMIN), and the precipitation-evaporation (P-E) balance. Figure 1 shows their observations and model results for how much the tropics have expanded, in degrees of latitude per decade.
FIGURE 1. ORIGINAL CAPTION FROM A2012: Figure 2 | Observed and modelled 1979–1999 Northern Hemisphere tropical expansion based on five metrics. a, Annual mean poleward displacement of each metric, as well as the combined ALL metric. … CMIP3 models are grouped into nine that included time-varying black carbon and ozone (red); three that included time-varying ozone only (green); and six that included neither time-varying black carbon nor ozone (blue). Boxes show the mean response within each group (centre line) and its 2σ uncertainty. Observations are in black. In the case of one observational data set, trend uncertainty (whiskers) is estimated as the 95% confidence level according to a standard t-test.
I note in passing that the error bars of the observations are very wide. In fact, they barely establish the change as being different from zero, and in a couple cases are not statistically significant.
Now, several people have asked me recently how I can analyze a paper so quickly. There are some indications that set off alarms, or that tell me where to look. In this case, the wide error bars set off the alarms. I also didn’t like that instead of giving the claimed expansion per decade, they reported the total expansion over the 28 years of the study … that’s a second red flag, as it visually exaggerates their results. Finally, the following paragraph in A2012 told me where to look:
We quantify tropical width using a variety of metrics5,11: (1) the latitude of the tropospheric zonal wind maxima (JET); (2) the latitude where the Mean Meridional Circulation (MMC) at 500 hPa becomes zero on the poleward side of the subtropical maximum; (3) the latitude where precipitation minus evaporation (P-E) becomes zero on the poleward side of the subtropical minimum; (4) the latitude of the subtropical precipitation minimum (PMIN); and (5) the latitude of the subtropical cloud cover minimum over oceans (CMIN). To obtain an overall measure of tropical expansion, we also average the trends of all five metrics into a combined metric called ‘ALL’. Expansion figures quoted in the text will be based on ALL unless otherwise specified.
What told me where to look? Well, the sloppy citation. Note that they have not given citations for each of the 5 claims. Instead, they have put no less than seven citations at the head of the list of the five groups of observations and model results. That, to me, is a huge red flag. It means that there is no way to find out the source of each of the five individual observational results in A2012. So I went to look at the citations. They are as follows:
5. Zhou, Y. P., Xu, K.-M., Sud, Y. C. & Betts, A. K. Recent trends of the tropical hydrological cycle inferred from Global Precipitation Climatology Project and International Satellite Cloud Climatology Project data. J. Geophys. Res. 116, D09101 (2011).
6. Bender, F., Ramanathan, V. & Tselioudis, G. Changes in extratropical storm track cloudiness 1983–2008: observational support for a poleward shift. Clim. Dyn. http://dx.doi.org/10.1007/s00382-011-1065-6 (2011).
7. Son, S.-W., Tandon, L. M., Polvani, L. M. & Waugh, D. W. Ozone hole and Southern Hemisphere climate change. Geophys. Res. Lett. 36, L15705 (2009).
8. Polvani, L. M., Waugh, D. W., Correa, G. J. P. & Son, S.-W. Stratospheric ozone depletion: the main driver of twentieth-century atmospheric circulation changes in the Southern Hemisphere. J. Clim. 24, 795–812 (2011).
9. Son,S.-W. et al. Impact of stratospheric ozone on Southern Hemisphere circulation change: a multimodel assessment. J. Geophys. Res. 115, D00M07 (2010).
10. Kang, S. M., Polvani, L. M., Fyfe, J. C.& Sigmond, M. Impact of polar ozone depletion on subtropical precipitation. Science 332, 951–954 (2011).
11. Johanson, C. M. & Fu, Q. Hadley cell widening: model simulations versus observations. J. Clim. 22, 2713–2725 (2009).
For no particular reason other than that it was available and first in the list, I decided to look at the Zhou paper, “Recent trends of the tropical hydrological cycle inferred from Global Precipitation Climatology Project and International Satellite Cloud Climatology Project data”. Also, that was a citation that refers to the minimum precipitation (PMIN) for both hemispheres, as used in A2012. Figure 2 shows results from the Zhou paper:
Figure 2. ORIGINAL CAPTION FROM ZHOU: Figure 4. Time‐latitude cross sections of zonal mean seasonal precipitation and the corresponding linear trend with latitude. Solid orange lines mark the 2.4 mm d−1 precipitation threshold which is used as the boundaries of subtropical dry band. The boundary at the high and low latitude of the dry band is used as a proxy of the boundary of Hadley cell and ITCZ, respectively. Solid black lines indicate latitude with minimum precipitation. Dashed red lines mark the Hadley cell boundary determined by the 250 Wm−2 threshold using HIRS OLR data.
Now, the black line in these four frames show the minimum precipitation, so that must be where they got the PMIN data. So I went to look at what the Zhou paper says about the trend in the minimum precipitation PMIN. That’s shown in their Figure 5:
Figure 3. ORIGINAL CAPTION FROM ZHOU: Figure 5. Linear trends of the latitude of minimum precipitation, ITCZ, and Hadley cell boundaries inferred from GPCP for each season and the year marked on the horizontal axis for (a) the Northern Hemisphere and (b) the Southern Hemisphere. … Leftmost, middle, and rightmost bars in each group are for minimum precipitation, Hadley cell, and ITCZ boundary, respectively. For quantities significant at the 90% level, bars are shaded green, blue, and orange, respectively.
Now, let me stop here and discuss these results. I’m interested in the “Year” category for minimum precipitation (green), since that’s what they used in the A2012 paper. Note first that the minimum precipitation results that they are using are not even significant at the 90% level, which is very weak. But it’s worse than that. This paper shows one and only one result that is significant at the 90% level out of a total of six “YEAR” results.
This brings up a very important and routinely overlooked problem with this kind of analysis. While we know that one of these six “YEAR” results appears to be (weakly) significant at the 90% level, they’ve looked at six different categories to find this one result. What is often ignored is that the real question is not whether that one result is significant at the 90% level. The real question is, what are the odds of finding one 90% significant result purely by chance when you are looking at six different datasets?
The answer to this is calculated by taking the significance level to the sixth power, namely 0.96, which is 0.53 … and that means that the odds of finding a single result significant at the 90% level in six datasets are about fifty/fifty.
And that, in turn, means that their results are as meaningless as flipping a coin to determine whether the tropics are expanding on an annual basis. None of their results are significant.
It also means that the data from the Zhou paper which are being used in the A2012 paper are useless.
Finally, I couldn’t reproduce either the average value, or the error bars on that average, in the A2012 “ALL” data. Here are the “ALL” values from my Figure 1 (the A2012 Figure 2):
Item, Value, Error JET, 0.45, 1.09 P-E, 0.75, 0.29 MMC, 0.24, 0.08 PMIN, 0.17, 0.51 CMIN, 0.33, 0.06 ALL, 0.33, 0.12
When I average the five values, I get 0.39, compared to their 0.33 … and the problem is even greater with the error bars. The error of an average is the square root of the sum of the squares of the errors, divided by the number of data points N. This calculates out to an error of 0.25 … but they get 0.12.
Does this mean that the tropics are not expanding? Well, no. It tells us nothing at all about whether the tropics are expanding. But what it does mean is that their results are not at all solid. They are based at least in part on meaningless data, and they haven’t even done the arithmetic correctly. And for me, that’s enough to discard the paper entirely.
w.
PS: I suppose it is possible that they simply ignored the results from the Zhou paper and used the results from another of their citations for the minimum precipitation PMIN … but that just exemplifies the problems with their sloppy citations. In addition, it brings up the specter of data shopping, where you look at several papers and just use the one that finds significant results. And that in turn brings up the problem I discussed above, where you find one significant result in looking at several datasets.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
DocMartyn says:
May 22, 2012 at 5:06 am
The data in the Zhou paper is obviously not normally distributed, and yet is treated as if it is. It is then combined with other data-sets and again treated as Gaussian..
========
This is a very good point. Time and time again we see climate science performing statistical analysis under the assumption the data is normally distributed, without any evidence this is a correct assumption.
Time series data is rarely normally distributed. Yet climate science proceeds under the assumption that it is – without having shown this to be true.
————————————————————————————-
I too will strongly agree with this point. I see this far too often, especially in climate research, where the data are assumed to be normally distributed. In our work with thunderstorm and tornado data, the data sets are rarely normally distributed, so we use non-parametric tests for significance. If t-tests, or other tests applicable to normal distributions, are used then it is incumbent on the authors to show the data are, in fact, normally distributed. When reviewing a paper such as this I would certainly request revisions to explain the statistical approach used.
curryja: Willis, check out this paper
http://webster.eas.gatech.edu/Papers/Hoyos_Webster2011.pdf
Though that was not aimed at me, I thank you as well.
ImranCan says (May 22, 2012 at 4:10 am): “Superb ….. just shows how little one has to dig to expose the smell of bullshit. Honestly – what has science come to ?”
Like a lot of other things, “science” has come to the single-minded pursuit of taxpayers’ money.
Willis:
Many moons ago when I was just a weedhopper I acquired a copy of Darrel Huff’s “How to Lie with Statistics”
http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728/ref=sr_1_1?s=books&ie=UTF8&qid=1337709611&sr=1-1
For years it was a prized highlight of my personal library until I made the mistake of lending it out and never saw it again. The concepts in it are still entirely valid but it’s almost as old as I am and the writing is charmingly dated at this point. It strikes me that the world is sorely in need of an updated “How to Lie with Statistics for the 21st Century”. Since you seem to share some of Mr. Huff’s gift for conveying mathematical concepts in understandable language you could be the right guy to produce such a tome. Perhaps you could rope McIntyre and Briggs in as coauthors and get Josh to do the artwork.
Although I am not a fan of top down mandates, we could assure the financial success of the project by making it a mandatory text for every school child in the country, with mastery of the material a requisite for high school graduation or preferably for getting out of the 8th grade. That might seem to be a bit draconian but if it could be implemented I strongly suspect that it would be one of simplest and most effective things we could do to reverse what seems to be an inexorable trend toward totlitarianism that the the world is presently experiencing.
Whoops! Make that Totalitarianism
[Fixed -w.]
Gary Hladik:
At May 22, 2012 at 12:09 pm you say;
No. ‘Climate science’ and much other science has become that “single-minded pursuit”, but most of science has not.
Many of us predicted that ‘climate science’ was likely to damage the reputation of all science. Sadly, as your post shows, our prediction has come true.
Richard
R Barker says: “…Your analysis brought to mind an editorial in the 10 May 2012 Nature by Daniel Sarewitz about the bias making its way into scientific research if for no other reason than expecting to get the desired result. http://www.nature.com/news/beware-the-creeping-cracks-of-bias-1.10600“
That editorial deserves a thread of its own. It states: “…It would therefore be naive to believe that systematic error is a problem for biomedicine alone. It is likely to be prevalent in any field that seeks to predict the behaviour of complex systems — economics, ecology, environmental science, epidemiology and so on….The first step is to face up to the problem — before the cracks undermine the very foundations of science.”
The fact that the author [Daniel Sarewitz] lacks the temerity to include climate science by name doesn’t take away from his message or its obvious applicability to the output of Jones, Mann, Travesty Trenberth, self-anointed-messiah Hansen, et al. Will Nature itself “face up to the problem?” I doubt it very much. Dr. Sarewitz.is no lightweight, but neither are others [e.g., Judith Curry] who’ve carried the same message in recent times.
Like I said.
More energy in the system results in a faster hydrological cycle which accelerates energy to space so as to offset any deceleration of energy to space caused by GHGs.
Energy content for the system as a whole remains exactly the same but it is distributed differently and the climate ‘price’ is a shift in the permanent climate zones.
Then one must consider how far human CO2 emissions would shift the climate zones as compared to shifts caused by solar and oceanic variability.
Going by the MWP and LIA the sun and oceans shift the zones by 1000 miles or so.
I’d guess our emissions would shift them by less than a mile.
And there I was foolishly thinking the tropics were defined by the tilt of the Earth’s axis to the plane of the ecliptic as it orbits the Sun.
Another triumph for the settled science!! Who’da thunk it???
I must say, the issuance of a press release to notify the world of a group of people doing public (bovine) defecation is getting a little tiresome. I don’t care how “carefully” the reviewers have scrutinized it.
On the other hand, I continue to learn from Willis’ forays into the wilds of “Climate Science.” Mostly how to improve my bovine defecation detection apparatus.
Thanks Willis!
Whenever I read one of your perspicacious forensic dissections of a statistical analysis (and the comments that follow), I almost always have a small “Oh Yeah!” flash back moment to some half forgotten tidbit from my college statistics classes!
As for your satirical paraphrase “Does This Analysis Make My Tropics Look Big?”, I’d opine “No! But I do admire your Temperate Latitude with the critics!”
MtK
richardscourtney has clearly never reviewed a paper or done science work, other than that…
You should have continued to forget. It’s not true here in the US Pacific Northwest, where fall and winter are the wettest seasons, spring and summer the driest. I imagine your general statement is falsified elsewhere as well.
Why that’s easy! All it takes is the word “model” as part of the main conclusion.
ChrisH says:
May 22, 2012 at 4:46 am
About that.
http://www.nature.com/news/beware-the-creeping-cracks-of-bias-1.10600
Unfortunately much of what passes as medical research today is not worth the paper it’s written on due to botched experimental design and misuse of statistics:
If your experiment needs statistics, you ought to have done a better experiment.
—– Rutherford
Thanks to several of you for discussing non-normal distributions. I am now a slightly better scientist.
And thanks to others for pointing out that the whole concern may be misplaced anyway, as warmer is better even if true.
The basis of Life is reduction of carbon dioxide. This comes from fossil fuels so only this source of energy can feed the world over the next century. Well, that and real warming, which causes release of carbon dioxide from the oceans.
Rosco says:
May 22, 2012 at 2:51 pm
Thanks, Rosco. There are actually two tropics, the physical tropics and the meteorological tropics.
The physical tropics, as you point out, runs from 23.45°S to 23.45°N. They are defined by the angle of the axis of the earth with respect to the plane of its orbit.
The meteorological tropics, on the other hand, are delineated by the Hadley cells. As the Zhou analysis points out, you can define the edges and the center in various ways. However you define it, in general it’s the width of the cells. Here’s a drawing of the Hadley circulation I did a while ago:
Note that the actual width of the Hadley cells run from about 30°S to 30°N. They are bounded on the outside by the great desert belts that circle the world at those two latitudes.
All the best,
w.
“Jeff Alberts says:
May 22, 2012 at 5:25 pm
John Marshall says:
May 22, 2012 at 4:29 am
Warm times are wetter. Cool times are drier. (i forgot to add that)
You should have continued to forget. . . . .”
I’ll make a WAG that John M. was thinking of glacial advances and interglacials. The notion is that a massive increase in ice also decreases the surface area of the ocean, produces cold winds, decreases precipitation, and provides wind-blown silt (loess; see Palouse).
Willis Eschenbach says: “……When I average the five values, I get 0.39, compared to their 0.33 … and the problem is even greater with the error bars. The error of an average is the square root of the sum of the squares of the errors, divided by the number of data points N. This calculates out to an error of 0.25 … but they get 0.12.”
When the authors say “…and the combined metric ALL shows ….” you are assuming that they took the average of the mean values of each metric and the error bars are calculated from the mean values assuming N = 5. That is most probably not what they did – since there are error bars already associated with each mean value and the observed data population may be different for MMC, PMN, CMIN, etc. – in which case the direct averaging of the means does not make much sense. I think what they did is different (although I have not read the sup. materials or Zhou paper) – knowing the number of data points, the mean and standard error for each group, they calculated sum x and (sum x^2) for each group, and then added all the (sum x) divided by the total sum of all data points to get the mean and using the sum of (sum x^2), the combined error bar was also calculated. In other words if the five means are m1, m2, m3, m4, and m5 with number of data points n1, n2, n3, n4 and n5, in each group, then your calculation of the net mean is (m1+m2+m3+m4+m5)/5, while the real combined (weighted) mean is (m1*n1+m2*n2+….+m5*n5)/(n1+n2+n3+n4+n5) – they are different unless n1, n2, n3 etc are all the same. Same problem goes with the net error bars also. As before, Willis, I really enjoy reading your criticisms.
bsk:
At May 22, 2012 at 4:41 am you said (in full);
And I explained the logical error of that at May 22, 2012 at 5:28 am saying;
[snip]
[snip]
The only rational dispute of my point would have been a demonstration that the perceived flaw in the paper is trivial and insignificant so my analogy is not correct. But the analogy is correct because the paper’s flaw is so serious that it does damage the validity of the paper’s arguments.
So, at May 22, 2012 at 5:25 pm you have replied to me by saying (in full);
This is another display of your lack of logical ability and it adds a display of your ignorance.
Therefore, I suggest you leave trolling of technical threads to people with some competence as trolls.
Richard
Does the movement south of the Tropic of Cancer and north of the Tropic of Capricorn have any influence on the above? Due to the obliquity of the ecliptic the tropics and polar circles are moving by about 14 metres a year. The tropics are decreasing by some 1100 square metres a year
I remember 20 odd years ago when expanding tropics and poleward migration of climate zones were the predicted signatures of global warming. When these things didn’t happen, the focus switched to other metrics that could be argued did show global warming; surface temps, Arctic ice, glacier melt, etc.
That papers are again being published arguing that the tropics are expanding, even though the data is as weak as ever, is probably due to the other metrics not behaving in the predicted way, and a generally desperate search for something that shows global warming still continues.
Willis, I should have added that once you figure out how the authors have used weighted averages for calculating the average and the error bar for the “ALL” from the mean and standard errors for each metric, I hope you can explain to your ardent followers why (1) Zhou’s input is largely irrelevant in affecting the conclusions in this paper, and (2) why the glaringly large error bars for JET, P-E, are PMIN are also immaterial – in other words why your entire criticisms that started this discussion are invalid.
Rob G. says:
May 22, 2012 at 10:04 pm
Thanks, Rob, but you are so intent on finding some error in what I’ve done that you neglect to read what I’ve quoted. The authors say, as I quoted above (emphasis mine):
Now, that is the sum total of their explanation. When they say that they “average the trends of all five metrics”, I take it to mean that they average the trends of all five metrics. You may claim all you want that that doesn’t make much sense, or that they did something else; and perhaps it doesn’t and they did … but it’s what they say that they did.
You also say:
Rob G. says:
May 23, 2012 at 3:37 am
“Ardent followers”? This is science, please, leave the side commentary aside, it damages your case.
If you think that Zhou’s input is largely irrelevant, then please explain why the authors concluded that they should include it. They must think it is not irrelevant.
If you think that the “glaringly large error bars … are also immaterial”, you’ll have to explain that as well. Please note that in fact the Zhou error bars should be even larger, to account for the fact that they have mined the data to obtain the results they use.
w.