Guest Post by Willis Eschenbach
With altogether far too much fanfare for my taste, the Berkeley Earth Surface Temperature (BEST) project has not released its preliminary results.
Or at least I can’t find them. I just wanted the month-by-month data that their hotrod new computer program spits out at the end of its run. The results they’re all hot and bothered about.
But despite releasing a massive database, 39,000 stations, along with the code in Matlab (which does me no good at all), I can’t find anywhere their freakin’ results. You know, the actual results of their work? The monthly average global temperature, the stuff that they mangled to produce things like their PR graph shown in Figure 1:
Figure 1. Purports to show that the BEST temperature record, and all the others as well, are all going “in one direction”, nowhere but up. I’m sure you remember the Climategate mantra, “hide the decline”? Keep that in mind as we proceed.
So … that will show those shifty skeptics, even BEST says it’s warming nonstop, evidence is right there before your eyes.
What’s not to like? How can you argue with that? The science is in.
Since I couldn’t get their results, I did the next best thing, and digitized their results. Even then, I was frunstrated. As far as I could tell, they never showed their actual results. The closest I found is in Figure 1 of their paper here:
Figure 2. Figure 1 of BEST’s “Decadal Variations” paper. Everybody’s going up, up, up, although you can’t really see what anyone is doing.
I blew that Figure up, and digitized it. Sixty years, 720 data points, boooring. Plus I hate it that they’ve smoothed the data, that makes it useless for statistical work. But it could have given us an idea of what’s going on in each of the records … if they hadn’t printed them atop one another in confusing colors. Enough of the spaghetti graphs already, you mad scientist persons, they show nothing! Figure 3 shows the BEST dataset along with the other datasets, this time displaced from each other so that we can actually see what’s happening:
Figure 3. The BEST land-only temperature record, compared to other surface and satellite land-only temperature records. 12 month moving average data, sadly. Note the decline.
[UPDATE: An alert reader noticed what I did not, that this is a subset of the BEST dataset that does not contain the stations used by the other groups (NOAA, etc). He points out that the full dataset is again different, in that it in fact rises more than the partial dataset. I have updated the figure and struck out some text to include that.
However, this doesn’t fix the questions. The post 1998 record from all of the BEST data is much more poorly correlated with the current records (~0.65) than prior to 1998 (~0.90). So this does not verify or validate the current groups datasets.
Hmmm … that gives a very different picture than Figure 1. Even with the bizarre 12-month moving average, the BEST record is clearly the outlier since 1998. You would think that in the modern era, the BEST would agree more closely with the other records. And indeed, from about 1975 to 1998 they were moving in something like lockstep.
But both before and after that time period, the BEST results are a clear outlier. And since 1998, BEST has been in a slow decline … funny how that didn’t show up in Figure 1. Yes, I know, a ten-year moving average shouldn’t show anything within five years from the end of the dataset. And I’m sure folks will argue that it’s just coincidence that they chose that exact smoothing length, and that it was the chance selection of colors that jumbled up the spaghetti graph so it’s unreadable … but y’know, after a while “coincidence” wears thin. I’m going with a more nuanced explanation, that it was a “deliberately unconscious choice to hide the decline”, although certainly you are welcome to stick to the story that it’s all just an unfortunate chain of events …
CONCLUSIONS:
Conclusion 1. It is extremely sneaky to send a truncated, smoothed result like Figure 1 out to the media to announce your results. That’s advocacy disguised as science. They did it to make it look like the temperature was headed for the sky and that BEST agreed. Instead, BEST actually disagrees with the other datasets by claiming that over the last decade, land temperatures are dropping, not staying stable or rising as per the other datasets. Using a graph that didn’t show that is … curious. As Gollum would say … “Oooooh, tricksy”. Including you, Judith. Figure 1 was nothing but “hide the decline” PR spin. Bad scientists, no cookies.
Conclusion 1. The correlation between the old data points used by the current groups, and the new data used only by BEST, is quite poor after 1998. This is visible in the plots of both the partial and full BEST datasets. The reasons for this are not clear, but it provides no support for the current datasets.
Conclusion 2. First point. The raw terror point, the thought that has the AGW alarmists changing their shorts, is the dreaded 2°C rise that is forecast from CO2. That is supposed to be the mythical “tipping point”. Second point. If we look at the 10-year smoothed data in Figure 1, BEST says that in the last two centuries, the temperature has risen about two degrees.
Let me note that over that two-century time period there have been:
a) No known increase in extreme weather events.
b) No known increase in catastrophes (other than from increased populations and property in vulnerable areas).
c) No major costs, deaths or damage from sea level rise. And don’t bother me with Katrina. A Category 3 hurricane took down ancient poorly maintained levees on a city below sea level. Absent that, no problem.
d) No climate-related spread of various infectious diseases.
e) No known increase in droughts or floods.
f) No loss of Tuvalu or other coral atolls.
g) Actually, none of the horrendous outcomes or biblical plagues of frogs and the like which are supposed to accompany the Thermageddon™ of a two degree temperature rise occurred over the last two centuries. To the contrary, the increased warming seems to have been a net gain for most humans, animals, and plants. Nobody likes freezing their asterisk off, after all, and the warming has mostly been in extra-tropical winter nights. That’s the theory, at least, although the BEST data should be able to tell us more.
Conclusion 3. BEST has done the world a huge service by collating and collecting all the data in one place, and deserves credit for that.
But they have done the world a huge disservice by becoming media whores, by putting out a shabby imitation of science in Figure 1, and by making a host of claims before peer review is complete.
This last one astounds me, that they’ve done it before peer review is finished. Doug Keenan and William Briggs have both raised separate and cogent arguments that the BEST analysis contains flaws. That would make me nervous, they’re kinda heavyweights, although any man can be wrong … but no, the BEST folks are making a host of claims as though their paper has already passed peer review. It’s the same publicity circus that Muller put on for Congress. And what three-ring media circus would be complete without their own brand new personalized “hide the decline” poster?
Since they have held out for extreme transparency, or at least given lip-service to the idea, I would be very interested to find out the names of the reviewers.
Because certainly, one possible explanation of their brazen trumpeting of their results before the peer review process is finished is that the fix is in. Why else the confidence that the reviewers will not find fault with their work? It is extreme hubris at a minimum, which is reputed historically to have unpleasant sequelae involving wax and feathers …
w.
PS—The world is warming. It has been for centuries. Rather than saying anything about anthropogenic global warming, all the BEST dataset does is confirms that. How that’s gotten twisted into some supposed “victory” for the AGW crowd escapes me.
“One more piece of the AGW puzzle cemented happily in place.”
Yet the biggest – indeed, the only – piece has yet to be supplied: that which demonstrates that it is the 6% of greenhouse gases that man generates (Planet Earth supplying the other 94%) as the cause of the warming that has occurred.
Or to put it more succinctly: demonstration of effect does not identify the culpable party.
I think your sample of stations must be carefully chosen and limited to say, 50
1) they must be well established for at least 50 years (airports are good, there is no urban build up and the CO2 emissions do nothing to temps anyway)
2) they must represent the earth 70/30 water ,
70% near coast or in or on islands in the oceans
30% on land, in land
3) you have to balance NH and SH similar to my tables here,
http://www.letterdash.com/HenryP/henrys-pool-table-on-global-warming
Stephen Wilde says:
October 23, 2011 at 1:15 am
“Is it right that the essence of trhe Berkely findings is that the rural and urban sites have been warming at a sinilar rate and so it is assumed that UHI effects are not a significant factor for determining the direction and rate of the temperature trend ?
Well, if so, how about the proposition that the incremental rate of nearby development is on average the same for both rural and urban sites ?
Wouldn’t that produce just such an outcome ?
During a period of development isn’t it just as likely that it will occur near a rural site as near an urban site ?”
Very well said. The CAGW crowd, including BEST, dare not touch these questions. Getting involved in empirical research just takes the magic out of the statistics.
steven mosher says:
October 22, 2011 at 11:44 pm
You are traveling farther down the road toward moral idiocy. BEST played Bait and Switch on Anthony. They wrongly substituted their 60 year analysis for the 30 year analysis that he expected. They produced work that did not address the empirical evidence in his station siting information from the most recent 30 year period. As regards Spencer publishing here, Spencer did not instigate a media blitz and Spencer did not violate a trust held by some other scientist. It is not enough to address the science when the scientists are knifing one another in the back.
Steve McIntyre says:
October 22, 2011 at 8:37 pm
“Willis, I’m puzzled why you place such weight on “peer review” as presently undertaken in climate journals. The articles will get and are getting far more effective review on the blogs than journals can provide.
In fields other than climate, circulation of working papers for comment is common practice. I wish that we’d had the opportunity to do this with our article on Steig et al.”
You seem unaware that you are making a proposal rather than describing peer review as it exists in academia. Journals published by industry sponsors have never followed peer review and pay for articles. But in academia peer review is solely a tool of the journal editor to be used as he/she desires. When the editor says peer review is finished then it is. An author who goes to the media before the editor says peer review is finished will not be treated kindly by that editor. Authors in academia are fully aware of these facts and that includes Muller. If the editor of the journal is not upset with Muller then I will be shocked. At best, the editor faces the difficulty of explaining why he is demanding even small changes to articles that have been pre-crowned in the media, assuming that the editor does not embrace pal-review.
Garrett @ur momisugly 9:50 a.m.
“This is….a defeat to those who were trying to discredit the AGW theory by implying that the temperature data was tampered with…”
May seem so to you but notice how you tamper with the data by omitting the “Catastrophic” element. Why evidence such disinterest in accurate instrumental records? When speaking of fractions of degrees, accurate measurements would be a boon to all mankind and if, despite the lack thereof (which has been demonstrated thanks to Anthony Watts et al), it could be shown that somehow we muddled into a fairly reliable estimate….good for us. Let us do better going forward.
Be grateful for the cloud.
Henry@ur momisuglyStephen Fisher
Look, if I were Berkeley, I would limit my sample of stations to say, 50 , and they must be carefully chosen,
1) they must be well established for at least 50 years (airports are good, there is no urban build up and the CO2 emissions is supposedly to make things “worse” but we know that that does nothing to temps anyway)
2) they must represent the earth 70/30 water ,
70% near coast or in or on islands in the oceans
30% on land, in land
3) you have to balance NH and SH similar to my tables here,
http://www.letterdash.com/HenryP/henrys-pool-table-on-global-warming
that is what I can think of off-hand.
reason why I found a connection with vegetation are the results that came from an Argentine station, cooling rate quite dramatically@ur momisugly 0.066 degrees C per annum since1974. Can only be due to de-forestation in that area.
Obviously the opposite is happening in the NH since the green movement saw everyone planting trees and gardens.
Note that the main driver of the increase in warmth is natural, between 0.03 and 0.04 degrees C /per annum (past 35 years) on maxima.
My point is that we must be looking at how much of that heat is trapped and why.
This is strange.
BEST is using a January 1950-December 1979 baseline. In Full_Database_Average_complete.txt, which is land-only, it says in the comments at the top:
For NCDC it says here that for their anomalies they are using the 20th century average, 1901-2000, which for annual on land is 8.5°C.
I took NCDC’s Annual Global Land Temperature Anomalies, threw them in a spreadsheet with BEST’s annual anomalies from Full_Database_Average_summary.txt, then added the baselines back into their respective anomalies to get the absolute temperatures.
BEST is running cooler than NCDC, consistently. From 1880 to 2009 inclusive, the overlap between the two sets, BEST averaged 1.339°C cooler than NCDC.
I know this won’t matter much for the “anomalies are what you want, only the trends matter” crowd (you know who you are). But given how much of the debate about the possible damaging effects of AGW hinge on such as permafrost warming a few more degrees from an absolute point, let alone all that ice piled up on land that is supposedly going to melt and drown many billions of people, don’t you think it’d be nice to have agreement on what the absolute temperatures actually are? An average difference of one and a third degrees matters quite a lot, I think.
Oh they’re dancing because they managed to make Anthony look silly. The great leader of the website they can’t stand managed to be wrong about something, or so they think. Of course it hasn’t dawned on them yet what they had to abandon of their own assumptions to make that happen. All that matters is Anthony pre-release said, “I’ll accept their results, good or bad,” and their early results seem to contradict him. Their whole process has been about pushing a media blitz on how warming was confirmed, and UHI was not found. Of course, forgotten in the background is the fact that these are preliminary results, their methods are not well-established, and the papers are not even peer-reviewed yet much less published.
All that matters to them is poking Anthony in the eye with half of the story, which frankly is just business as usual for these people. It should also speak very clearly to how thirsty these people are for any good news on their front.
I’m surprised that BEST didn’t ask research the UHI question more deeply–or at least ask skeptics for input. I seem to recall there have been suggestions posted here from time to time as to clues that indicate a strong UHI effect in particular cases, and for methods of teasing it apart more generally.
kadaka (KD Knoebel) says:
October 23, 2011 at 11:10 am
An average difference of one and a third degrees matters quite a lot, I think.
Too true but climate scientists take that in their stride. That is why they invented anomalies. Their GCM models do not give good absolute temperatures, have a look at an analysis by Lucia.
They may be so used to the distortion glasses of anomalies that consider a degree or so changes in absolute temperatures as part of the game they play with data ( tongue in cheek)
Dear Rich,
I need a new hockey stick for the front page of my next IPCC report. Can you help?
Yours in anticipation, Rachendra.
Dear Rachendra,
Good job you asked before I put our data out. I’ll hold off for a bit and see what I can come up with. What will be in it for me? love Rich.
Dear Rich,
If you can come up with the hockey stick just before the closing date so those nasty “D” poeple can’t get at it I promise I’ll use all your papers in the next report. Even the garbage ones. Think of all the citations you’ll get. love, Rachendra.
Dear Rachendra,
Thanks mate, I think I can do it by just using the land temps not the global average. None of those dumb journos will notice. I’ll bang in a few interviews with “Global warming confirmed” in them and hide the statements about “we have’nt looked at the relevance to CO2” right at the bottom somewhere where the “D’s” can’t find it. Love and kisses Rich
PS Can I have a signed copy of your latest “romance” book.
[img]http://oi55.tinypic.com/25rgm5d.jpg[/img]
Comparing BEST to CET, BEST looks too cold in the past, too warm in cold 80ties. Oh and the recent decline is not there.
@ur momisugly Disko Troop, 11:42 am:
lol
From anna v on October 23, 2011 at 11:39 am:
404 Not Found error. Site problem? Old link, article moved? Lucia flipped out after losing the battle with Lord Monckton, gave herself completely over to the Dark Side, and did a SkepSci rework of her site?
The more I think about it, with the annual global temperatures running around only 7 to 8°C, that one and a third degrees is quite a large difference. Someone should consider it significant enough to deserve consideration.
BTW, how’s it going over in Greece? You have that rainwater-fed cistern good for watering the garden, so possibly you won’t be starving or dying of thirst anytime soon. But the news from there is troubling, much talk of cutting pensions and other benefits. How’s life treating a particle physicist these days?
I’ve got as far as section 3 New Averaging Model of the Berkeley Averaging paper and I fail to understand how they can introduce such a sensible global definition of temperature and then decide to work only on land temperature measurements for which there is no logic that says the conditions 2 can reasonably be assumed for that subset. Hence what they are measuring for land is something other than their definition of temperature.
So how one can then derive a useful calculation for land warming other than the spatially corrected average of land weatherstations defeats me. What am I not understanding? As an example there might be a 60 year cycle that gradually increases the flow of warmer air and cloud from the oceans into the land space and so create a growing temperature of both Tmax and Tmin whilst causing a lowering of sea heat content. After all I gather some sea heat is missing. Sure the measured land temperature would rise but how does that help determine if there is real anthropogenic global warming?
The world may be warming, however it isn’t locally. I crossed over Hwy 88 in order to catch a jet from SFO to the tropics. In that crossing I noticed a lot of new ice fields probably left over from last years snow. These fields will become multi-year ice after this winter season which soon to arrive. The terrain I was gawking at was in the 8500 – 11,000 elevation. Probably down South where the moutains are higher there is even more leftover snow. The ice is back! What do I care if the whole earth warms a few tenths per some reanalysis of the data. Fiddlesticks! The ice doesn’t lie.
Theo
“You are traveling farther down the road toward moral idiocy. BEST played Bait and Switch on Anthony. They wrongly substituted their 60 year analysis for the 30 year analysis that he expected. They produced work that did not address the empirical evidence in his station siting information from the most recent 30 year period. As regards Spencer publishing here, Spencer did not instigate a media blitz and Spencer did not violate a trust held by some other scientist. It is not enough to address the science when the scientists are knifing one another in the back.”
1. Anthony, I believe, gave them a copy of his data, prior to his publication.
“There seems a bit of a rush here, as BEST hasn’t completed all of their promised data techniques that would be able to remove the different kinds of data biases we’ve noted. That was the promise, that is why I signed on (to share my data and collaborate with them). Yet somehow, much of that has been thrown out the window, and they are presenting some results today without the full set of techniques applied. ”
He was upset last time because they had not applied the scalle to the data
From what he wrote it looks like he shared data in advance of his publication
Then of course he published his data for anybody to use in any way they want.
Now, he seems to be upset because they did not do the analysis the way he
wants it done. Its not his paper. Its not his data. he published the data collected
by volunteers. One reason I push for open code and open data is to GET RID
of this notion that people OWN DATA, especially data that is paid for with my dollars
and data that is collected by volunteers, with the expressed INTENT of making it open
that data is no longer Anthony’s data. It is in the commons and can be used or mis used
as people see fit. On argument I had to deal with is climate scientists making the stupid
argument that they didnt want to release data to amatures. Now, I see people here
arguing along the same lines. That some how Anthony has control over data that he
has released per his promise. The right way to handle this is to let Muller make his mistakes if you think it is a mistake. Then take his code, change the period and guess what you will find?
Siting doesnt matter to Tave. Your uncertainty will increase but the mean will not change.
The world will STILL be warming and GHGs will still be a part of the cause.
Next, I raise spencer not to draw a point about PR. I raise Spencer to illustrate that you and others do not, as McIntyre points out, have a CONSISTENT philosophy when it comes to Peer review.
When its a paper you dont like, you say peer review is broken and you clamor for blog review.
When it is a paper you dont like that hasent been peer reviewed, you clamor for peer review.
Have a consistent philosophy and you will have no problems.
1. papers need to go through a peer review
2. that peer review need to be transparent
3. The data and code need to be published to enable blog review.
That is a consistent philosophy.
Also on the PR front, Dont make me point to all the PR stunts that skeptics have pulled in advance of publication. Focus on the science.
Let me run this by you all. Let us assume that temperature is water and Tmin and Tmax are high an low tide. We can measure the heights of the boats in the harbor with a radar stallite.
The signature of global heating, would be more water, so on average, the low tidal level.and high tidal level will boot rise.
It is also possible that the boats have been altered in time, some lightened and others ballasted. In this case, some bots would have an increase in height and others lower.
The BEST study tells us that 67% of stations show an increasing trend in heating, the rest cooling.
If there were a systemic increase in heating, or increase in water in the ocean, all thermometers will rise, just like all boats.
An heterogeneous change in the sign of heating, where one third go one way and two thirds another, tells us we have changing boats.
Nowhere have the CAGWer’s indicated that the ‘climate sensitivity’ would be inhomogeneous. on a local scale.
Can anyone give me an explanation as to how a well mixed gas, like CO2, with the ability to adsorb and re-radiate radiation to Earth, does not have a uniform, latitudinal, effect?.
Can anyone supply a reason for a 2:1 distribution of increasing/decreasing temperature, in the same continent, that is a uniform property of the atmosphere; rather than disparate changes in the micro-environment of the locations?
shows the latest version is 0.8.9
Additionally, the latest documentation is here
Arg: documentation at http://cran.r-project.org/doc/contrib/Hiebeler-matlabR.pdf
Garrett says:
October 23, 2011 at 9:50 am
Claims of victory by either faction where the paper has not even been peer reviewed and published are grossly premature. Let’s talk about victory after a few years of digestion by the scientific and blogging communities.
Steven Mosher writes:
“That some how Anthony has control over data that he has released per his promise.”
Let me try a different approach. Anthony should not have become involved with Muller and BEST. He should have known that they would use his participation for their benefit and care nothing for his interests. That is exactly what happened. Anthony is too trusting. His trust was violated. I believe that he has learned from this. Given what you say, I take it that you believe that no one has anything to learn from this.
“The right way to handle this is to let Muller make his mistakes if you think it is a mistake. Then take his code, change the period and guess what you will find?”
What Anthony expected and what every sceptic craves is for some Warmista to show some interest in empirical matters. Muller and BEST have demonstrated in spades that they have no interest in empirical matters. The warning “Lies, Damn Lies, and Statistics” must be amended to include “Blindness to the Empirical.”
“Siting doesnt matter to Tave.”
Nor to you. You have demonstrated on this website that you can take any aberrant weather station reading and incorporate it smoothly into the statistical weave of climate alarmism. In my opinion, all you did is show that your statistical methods give you powerful tools for hiding the pea. Notice that no Warmista is willing to discuss these matters. If you don’t believe that, just check with those who continue to defend the Hockey Stick. What is really important is that you demonstrate that you have no respect for the empirical. I am sure that you cannot give an example of an aberrant station reading that cannot be incorporated into the climate alarmism meme.
“Your uncertainty will increase but the mean will not change. The world will STILL be warming and GHGs will still be a part of the cause.”
Why is an increase in the uncertainty unimportant to BEST? Once the bars are wide enough, even the great unwashed will see that there is no evidence for alarmism that is not swamped by the error bars.
Muller and BEST did not claim that the world is STILL warming; rather, they claimed that scepticism is proved wrong. Can you not see that difference?
As regards Spencer, are you really saying that what Spencer did on this website was totally parallel to what Muller and BEST did? Please explain. I will describe the differences that you leave out.
Joel Shore says: October 22, 2011 at 7:05 pm
…Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.
As usual you miss the point.
Firstly we are not at all convinced that BEST has shown the station siting issues to be answered. Here are the relevant papers
(1) Berkeley Earth Temperature Averaging Process, describing their new method for station combination, homogenization, and spatial interpolation.
(2) Influence of Urban Heating on the Global Temperature Land Average, analyzing the impact of urbanization on global temperature trends. They find that rural station are actually warming faster ( by 0.02°C ± 0.02 per decade) than urban areas.
Paper (2) makes one suspect that many/most BEST “rural stations” are NOT truly rural and UHI-free but are, as Spencer and Ilarionov have shown, the stations subject to the highest UHI rate of warming. This, in turn, will totally distort the “homogenization” data of Paper (1).
The only way out is to find (a) totally uncontaminated stations, HOWEVER FEW; (b) to apply properly-verified UHI corrections to select stations with metadata of immediate instrument siting, siting changes, and neighbourhood changes. Today this looks damn near impossible. BEST have certainly not dealt with this, have not even mentioned it when they should. Perhaps the comparison of land and sea temperatures (Lansner, and above) is the way through this one.
Secondly, the UHI distortions DO matter, because they are likely to be the factor obscuring any post-1980 sun/temperature correlation (sun/temp. corr. would disprove CO2 AGW). Certainly land and sea temperature anomalies diverge from about the same time the solar/temp correlation fails.
DocMartyn says:
October 23, 2011 at 1:59 pm
Brilliant post! However, it points toward the empirical. Therefore, no Warmista will finish reading it.