Guest Post by Willis Eschenbach
In three previous posts here, here, and here, I discussed problems with the paper by Shakun et al., “Global warming preceded by increasing carbon dioxide concentrations during the last deglaciation” (PDF,hereinafter S2012)
Commenters said, and reasonably so, that I had not fully addressed their claim that warming progressed from south to north. Their Figure 5a shows the trends by latitude band. It purports to show that the further north, the later the warming.
Figure 1. Figure 5a from S2012. ORIGINAL CAPTION: Figure 5 | Temperature change before increase in CO2 concentration. a, Linear temperature trends in the proxy records from 21.5–19 kyr ago (red) and 19–17.5 kyr ago (blue) averaged in 10° latitude bins with 1 sigma uncertainties.
Now, that seems pretty clear. Less blue and more red as you go up towards the north pole. What could be wrong with that? Well, as usual, nature is not that neat. When you look at it closely, it’s nowhere near as clear as that chart seems to indicate.
The first thing that’s wrong is that out of the fourteen bands with data, only five of them show a significant difference between the early trends (21.5 to 19 thousand years ago [kyr BP]) and the late trends (19 to 17.5 thousand years ago [kyr BP]). In the other nine bands, the uncertainties overlap, so we can’t even say if they are different. (The uncertainties for each band are shown as red and blue long thin lines with short vertical ends.) As a result, they are meaningless, and should not be shown.
But that’s just a symptom of the real problem, which is that there is very little data in many latitude bands, and the proxies are very different from each other.
To investigate each of the bands, I started by expressing all of the temperatures as anomalies around the average temperature from 21.5 to 17.5 kyr BP. Then I divided them by bands and graphed them. Figure 2 shows the results for the Northern Hemisphere.
Figure 2. Trends by latitude band. The background colors correspond with Figure 1, with red for 21.5 to 19 kyr BP, and blue for 19 to 17.5 kyr BP. Dark red lines are centered Gaussian averages of the individual proxies, with the data shown by the green squares.
Let me discuss these panel by panel. First, let me note an oddity—why is the early period longer than the later period? But I digress …
Panel a: Only two proxies, and one of them has a hump right at 19 kyr BP.
Panel b: Five proxies. Three have a hump right at 19 kyr BP.
Panel c: Two proxies. One is dead level, one rises during the later (blue) period.
Panel d: Three proxies, but one of them starts just before 19 kyr BP. Seriously, folks, do you think an average of these is meaningful?
Panel e: Hard to tell what’s happening here. Several of the proxies go either up or down just after 19 kyr BP.
Panel f: All trends in the blue section are about the same, except the poor proxy taking a dive right after 19 kyr BP
Panel g: Another goofy one. Right after 19 kyr BP, two of the proxies head for the sky.
Panel h: No trend before, no trend after.
Figure 3. As in Figure 2, but for the Southern Hemisphere
Again, by panel.
Panel a: Not much difference, red or blue period.
Panel b: Three proxies. Two go up at 19 kyr BP. One goes down at 19 kyr BP. Is this supposed to be meaningful?
Panel c: Three proxies. Neither the red period nor the blue period shows much.
Panel d: Two proxies. Two. One goes up after 19 kyr BP. So what?
Panel e: Here, a lot of the proxies have a low point at about 19 kyr BP … and they have a high point about 500 years before that.
Panel f: These four, all ice cores from Antarctica, agree pretty well. However, only one of them has a significant trend, and that only in the blue area.
Now, to me those results don’t mean much. Of the eighty proxies, only eight of them have a significant trend in both the red and blue periods … and that’s without adjusting for autocorrelation. The proxies show no clear pattern. They are too varied, and too few, to tell us much of anything.
Let me close with what may be a more revealing graph, dividing the globe up into 45° latitudinal bands.
Figure 4. S2012 proxies divided into four bands. Colors go from blue at the north pole, to yellow at the equator, and end up with red at the south pole.
Let me say that Panel a shows something very curious. The rise in temperature started quite early the two Greenland proxies … and timing of the others are all over the map. I can’t see how that supports any claim of late warming in the north.
BOTTOM LINE: I see no evidence in any these latitudinal bands of proxies to support the claim that the warming progressed northwards. It certainly may have done so … but these proxies are not useful for supporting that claim.
My goodness, I certainly hope that I’m done with these proxies.
w.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Willis,
Your last line reads funny.
“My goodness, I’m certainly hope that I’m done with these proxies.”
[REPLY] Thanks, fixed. -w.
Willis,
I’m almost certain a rebuttal sent to Nature would fail but send it anyway for the record and later post it on WUWT.
Someday Winston, you will understand CO2 can and does do whatever Big Brother says it does.
We just have to re-educate you, that’s all. You’ll see………..someday.
Well done. A good illustration of what can be done by simple data analysis methods, using good displays and lucid supporting descriptions. But it can take a great of hard work and steady concentration to get there and then make it look simple! I am impressed, and I think at this stage in the approach to AR5 it is particularly important to get penetrating reviews of papers which the agenda-pushers of the IPCC are likely to seize upon for their next wave of machinations.
OK, so something happened at 19 kyr BP, was that the point of the paper?
Why bother with Nature, crowd source a paper that deconstructs the whole thing. Until the journals fix themselves, ignore them.
I await the ritual debunking of this by the folks at SkS…
Two things we can be sure of are:
i) The BBC and other pro-CAGW organisations will not be taking notice of, or publishing, Willis’ findings.
ii) The next IPCC fantasy report will include the Shakun report, but can be guaranteed to ignore the shredding of its findings by Willis and other commentators.
Jimbo says:
April 11, 2012 at 8:44 am
I agree, Jimbo. They do have to reply, don’t they? And if they don’t reply, that’s an indicative response, too.
“The prospect of domination of the nation’s scholars by Federal employment, project allocations, and the power of money is ever present – and is gravely to be regarded.
Yet, in holding scientific research and discovery in respect, as we should, we must also be alert to the equal and opposite danger that public policy could itself become the captive of a scientific-technological elite.”
-Dwight D. Eisenhower, Farewell Address, Dartmouth College, 1961
“If you want to get laid, go to college; if you want an education, go to the library.”
-Frank Zappa
I am sure Shakun is aware of Willis’ rebuttals. If his research is so sound then why doesn’t he post a counter here on WUWT and show us why and how his conclusions are valid.
Come on Mr. Eschenbach: Don’t brabble here too long about the Shakun paper.
If the Shakun paper is so full of mistakes. Just submit the rebutal to Nature magazine and ask for the chief editor’s resignation, because the peer-review process clearly failed here.
“peer review”..what an absolute FRAUD. More like “tribal approval process”.
NO ONE in the “peer review” ever goes into these papers in this depth. PEER REVIEW IS COMPLETE NONSENSE!
And the other problem with these sort of NONSENSE DATA REVIEWS PARADING AS “SCIENCE” is that they are INHERENTLY WORTHLESS.
There are far too many variables and the data is far to NOISEY to really see anything.
It’s like saying, “It’s summer, it’s warm..” It’s winter, it’s cold.
Mental “self gradification” for those who can’t build, can’t design, and can’t do real research. (I recommend looking at the work of Dr. Stephan Hell, Max Planck, with regard “Breaking the Abbe Limit”, or Kawanee Bohanden of Stanford (Brains in Silicon), or the work of Dr. Barry Marshall (H. Pylori bacterial, Nobel Prize, Medicine, 2005) to find out what REAL science is about.)
Max
Peer review is not nonsense: it is a necessary filter to keep the nutters out of the scientific press. At the same time, one does not expect the Reviewer to check the paper’s arithmetic, or soil samples, or observations: that would be to do as much, or more, than the original author. Peer review is a sanity check, not a guarantee of truthfulness or accuracy.
That said, it does seem that the system can be corrupted, or – less seriously – just get sloppy. But Quis Custodiet ipsos Custodes? Can we peer review the Peer Reviewers?
“So nobody found these issues in peer review?”
Well, I’ve done my share of “peer-reviewing” for engineering journals (it was simply called “reviewing”).
What I have always understood was required was a statement to the editor:
– This paper is worth publishing as it contains original material of value to the readership (either as it is or with minor changes)
OR
– It would be worth publishing if re-written, bearing in mind the following things (X,Y, Z)
OR
It should be rejected for the following reason (not original, erroneous, outside the scope of the journal – maybe submit to journal J)
A reviewer will check that the paper is well structured so that it flows logically, that it makes proper reference to related work by other authors with appropriate references. They will point out any errors they have noticed and minor improvements that could be made.
I think it would be very unusual, in any field, for a reviewer to attempt to duplicate the work as a check. If others could not reproduce the work – a complete rarity in normal physical science – this would eventually lead to “communications to the editor” discussing the problem. An editor would normally forward such communications to the original authors for their comments, which very often will be published together with the “communication to the editor”.
Tom_R says:
April 11, 2012 at 8:19 am
My observation of peer review is that the reviewers mainly look to make sure any and all of their own papers even remotely related to the topic are cited in the text.
Yes! Although reviewers are supposed to be anonymous, you can often figure out who they are by their insistence that paper X, Y and Z (by a particular author) should be cited.
The way this is playing out makes me think that “peer review” is practised at a local sports bar featuring half priced drinks between 5-7pm. We would know for sure if there were stains on the paper from the chicken wings!
@ur momisugly Brent Hargreaves …
well done !
(same to you, Willis ! … 🙂 … )
Willis Eschenbach: In the other nine bands, the uncertainties overlap, so we can’t even say if they are different. (The uncertainties for each band are shown as red and blue long thin lines with short vertical ends.) As a result, they are meaningless, and should not be shown.
The conclusion is to strong. For an analogy, consider two groups at risk for a disease, one of which has been vaccinated (or otherwise treated, say aspirin for heart attack), the other not. After a week, there is no difference between the groups, after a month mabey a little difference, etc. Eventually the difference is clear and statistically significant with p < 0.0001. It would be foolish to suppress the intermediate stages where there is no statistically significant difference. Rather, you should model the result as a function of duration, and estimate the statistical significance of the duration. In this case, you should model the warming as a function of distance from Antarctica.
Further, you say that there are not enough proxies with enough global coverage. Well sure: this is a pioneering study. Its results should encourage other groups of people to conduct the follow-up studies. It took more than 1 study to confirm the effects of aspirin in reducing heart attack and stroke rates.
Everything that is known is known through proxies, such as mercury expansion for temperature increase. The only scientific questions are the reliability and accuracy of the estimated proxy relationships.
oops, I do not mean that “you should model the warming as a function of distance” as an imperative, only that it is the functional relationship over the whole range that matters, not exactly which intermediate distances produce statistically distinguishable effects.
Hopefully this analysis of these proxies will end up in an easily accessed and cross referenced database to prevent similar errors in the future. Quality control and verification is the only way to improve accuracy of future studies and predictions.
Willis, you shook this paper to death. All in all, some very nice deconstructing. I don’t think peer review was ever meant to be as thorough as this. I wonder how many climate science papers would survive this level of analysis?
Still, I think the Shakun et el paper was a worthy publication. It really stirred things up, the thought that CO2 may have led us out of the last ice age. So maybe shaken, not turd.
Martin A’s comments on the peer review process are very useful, and confirm the expectations I would have, having reviewed documents in a commercial environment. It’s a sanity check but not a confirmation that the paper is correct..
However, once a paper is published, the “it’s been peer reviewed” mantra is used to imply that the reviewers endorse its conclusions.
So either peer review is a thorough review of the paper’s methods and conclusions so that the authors can legitimately claim that the reviewers endorse it, or it’s a review of methods that weeds out poor science without endorsing the content. The scientific world can’t have it both ways: a simple sanity check not intended to find errors in the results and a full endorsement of content.
My observation of peer review is that the reviewers mainly look to make sure any and all of their own papers even remotely related to the topic are cited in the text.
Naaaa, not so much. Look, I review papers in physics, not exactly all of the time, but at least occasionally. I review papers in a certain narrow range of statistics as well. Editors choose reviewers in lots of ways, but they don’t necessarily send a paper to be reviewed by either a competitor or a star in the field or a “pal” of the author, as many have alleged above. Yes, I think there is evidence that gatekeeping is alive and well and living on Nature’s editorial board, but it isn’t universal or even particularly common across the board. Even at Nature, everything isn’t about the climate.
“Usually” an editor will pick somebody who is participatory and knowledgeable enough to be able to see if egregious claims are being made or blatent errors being asserted, but who is not directly affirmed or challenged by the work being reviewed. They may or may not also send it to an interested or party — rather more likely if the work is highly critical of somebody or their work or conclusions — but they then read the resulting comments with a grain of salt.
If I were reviewing Shakun’s paper, I wouldn’t be inclined to reject it. At most I would pass back some questions, things that they (he) didn’t look at that might be interesting or things about the results they present that might be questionable. I certainly wouldn’t — as part of the review process — go to download the data and work through it myself as Willis has done to see if I can or cannot tell a different story from the same data. There may be many stories that can be told from the data, and it is up to the reader to judge which stories make sense and hence support the conclusions the authors offer and which ones do not. A referee review should not itself be a rebuttal article — it should confine itself to a narrow critique not of the conclusion but of the methodology and support offered.
As I’ve pointed out and suggested (as have many others) there is nothing stopping Willis from taking the Shakun data, augmenting it with data from other sources (as he has done), and writing his own paper and arriving at his own conclusions. Those conclusions can also either be framed as “rebutting Shakun” or “conclusions in and of themselves”. To be honest, given the extension of the results with the additional CO_2 data (and perhaps the further extension up to the modern era instead of stopping mid-Holocene), and the distinct methodology used to both present and analyze the results, I think it should rather be a paper in its own right, not a “refutation” paper. In fact, it would be most effective by simply presenting all of the data just as Willis has presented it and asserting no conclusion, asserting the null conclusion that it is not possible to infer from the data whether CO_2 fully lagged or fully led temperature, that there is no particularly convincing scheme of southern warming preceding northern warming on a global basis, that CO_2 clearly followed temperature (and not the other way around) across the Younger Dryas, that CO_2 and temperature have if anything countervaried across most of the Holocene, that the temperatures now are not the warmest encountered in the Holocene, and that all of these statements are still rather dubious as global statements because the proxy sites are tremendously biased with respect to coastal, inland, and oceanic and do not uniformly sample latitudes.
That kind of paper would actually have a rather good chance of being accepted, actually. First of all, it is saying nothing that any given reviewer doesn’t already know — that the data is weak to support any sort of definite conclusion. That this overrides Shakun is almost irrelevant — simply pointing out the lack of samples from continental interiors or mid-ocean locations makes that a foregone conclusion anyway. The main point is that by presenting only “cooked” figures — the results of some partially specified aggregation processes — Shakun may have been misled by their own methodology, not from malice or confirmation bias but by simply not looking at the actual data as thoroughly as Willis has, or in the same way. By presenting the data itself in a digestible form, the reviewer can really decide for themselves what, if anything, it means.
rgb
Septic Matthew/Matthew R Marler says:
April 11, 2012 at 10:10 am
…you should model the result as a function of duration, and estimate the statistical significance of the duration. In this case, you should model the warming as a function of distance from Antarctica.
———————————————————
This is problematic. If you review each proxy individually you will see good correlation with the Antarctic as you move further north. But some proxies, even close by, are quite different. So already you have a problem (further proxies being more highly correlated to the Antarctic than local proxies). So what to do? You can throw all the proxies into an averaging filter and pull them out in slices of latitude (what Shakun et al did) or you can cry foul (kind of what Willis did) and claim the proxies can’t be combined.
It would be worthwhile to select bands of constant latitude and then move northward from Antarctica to the Arctic showing how the proxies change. I’m not sure they would withstand that analysis. And if they wouldn’t why do we think they would be any better if we lumped them all together?
The other issue I have with global averaging is whether the earth can even be treated as being relatively uniform. Taking proxies along an ocean shore is one thing, but what about the entire mountain range of North and South America? Does this have any effect on climate? Or most of China, Russia and Europe? Almost all of Canada and the continental US? There is virtualy no data in any of these places (i.e. where people live). In fact the sampling density of the Shakun paper is the exact opposite to that of the world land based temperature record (which is dominated by American sites). Does leaving these entire regions out have any effect on the climate proxies?
Imagine if you were estimating race or skin color (not that we are allowed to do that anymore). What would happen if you left out only two countries out of 200 – say China and India. Would that make a difference? Is the climate record like that? Or is it so uniform that sampling basis (huge in the Shakun study) is not an issue. And if that is true why do many of the proxies vary so greatly over a relatively short distance?
Mr. Eschenbach, I have never seen such a complete and total evisceration of any piece of so-called scientific study as your series on this paper.
Well done!
Sorry, should have been bands of constant longitude.