A preliminary assessment of BEST's decline

Guest Post by Willis Eschenbach

With altogether far too much fanfare for my taste, the Berkeley Earth Surface Temperature (BEST) project has not released its preliminary results.

Or at least I can’t find them. I just wanted the month-by-month data that their hotrod new computer program spits out at the end of its run. The results they’re all hot and bothered about.

But despite releasing a massive database, 39,000 stations, along with the code in Matlab (which does me no good at all), I can’t find anywhere their freakin’ results. You know, the actual results of their work? The monthly average global temperature, the stuff that they mangled to produce things like their PR graph shown in Figure 1:

Figure 1. Purports to show that the BEST temperature record, and all the others as well, are all going “in one direction”, nowhere but up. I’m sure you remember the Climategate mantra, “hide the decline”? Keep that in mind as we proceed.

So … that will show those shifty skeptics, even BEST says it’s warming nonstop, evidence is right there before your eyes.

What’s not to like? How can you argue with that? The science is in.

Since I couldn’t get their results, I did the next best thing, and digitized their results. Even then, I was frunstrated. As far as I could tell, they never showed their actual results. The closest I found is in Figure 1 of their paper here:

Figure 2. Figure 1 of BEST’s “Decadal Variations” paper. Everybody’s going up, up, up, although you can’t really see what anyone is doing.

I blew that Figure up, and digitized it. Sixty years, 720 data points, boooring. Plus I hate it that they’ve smoothed the data, that makes it useless for statistical work. But it could have given us an idea of what’s going on in each of the records … if they hadn’t printed them atop one another in confusing colors. Enough of the spaghetti graphs already, you mad scientist persons, they show nothing! Figure 3 shows the BEST dataset along with the other datasets, this time displaced from each other so that we can actually see what’s happening:

Figure 3. The BEST land-only temperature record, compared to other surface and satellite land-only temperature records. 12 month moving average data, sadly. Note the decline.

[UPDATE: An alert reader noticed what I did not, that this is a subset of the BEST dataset that does not contain the stations used by the other groups (NOAA, etc). He points out that the full dataset is again different, in that it in fact rises more than the partial dataset. I have updated the figure and struck out some text to include that.

However, this doesn’t fix the questions. The post 1998 record from all of the BEST data is much more poorly correlated with the current records (~0.65) than prior to 1998 (~0.90). So this does not verify or validate the current groups datasets.

Hmmm … that gives a very different picture than Figure 1. Even with the bizarre 12-month moving average, the BEST record is clearly the outlier since 1998. You would think that in the modern era, the BEST would agree more closely with the other records. And indeed, from about 1975 to 1998 they were moving in something like lockstep.

But both before and after that time period, the BEST results are a clear outlier. And since 1998, BEST has been in a slow decline … funny how that didn’t show up in Figure 1. Yes, I know, a ten-year moving average shouldn’t show anything within five years from the end of the dataset. And I’m sure folks will argue that it’s just coincidence that they chose that exact smoothing length, and that it was the chance selection of colors that jumbled up the spaghetti graph so it’s unreadable … but y’know, after a while “coincidence” wears thin. I’m going with a more nuanced explanation, that it was a “deliberately unconscious choice to hide the decline”, although certainly you are welcome to stick to the story that it’s all just an unfortunate chain of events  …

CONCLUSIONS:

Conclusion 1. It is extremely sneaky to send a truncated, smoothed result like Figure 1 out to the media to announce your results. That’s advocacy disguised as science. They did it to make it look like the temperature was headed for the sky and that BEST agreed. Instead, BEST actually disagrees with the other datasets by claiming that over the last decade, land temperatures are dropping, not staying stable or rising as per the other datasets. Using a graph that didn’t show that is … curious. As Gollum would say … “Oooooh, tricksy”. Including you, Judith. Figure 1 was nothing but “hide the decline” PR spin. Bad scientists, no cookies.

Conclusion 1. The correlation between the old data points used by the current groups, and the new data used only by BEST, is quite poor after 1998. This is visible in the plots of both the partial and full BEST datasets. The reasons for this are not clear, but it provides no support for the current datasets.

Conclusion 2. First point. The raw terror point, the thought that has the AGW alarmists changing their shorts, is the dreaded 2°C rise that is forecast from CO2. That is supposed to be the mythical “tipping point”. Second point. If we look at the 10-year smoothed data in Figure 1, BEST says that in the last two centuries, the temperature has risen about two degrees.

Let me note that over that two-century time period there have been:

a) No known increase in extreme weather events.

b) No known increase in catastrophes (other than from increased populations and property in vulnerable areas).

c) No major costs, deaths or damage from sea level rise. And don’t bother me with Katrina. A Category 3 hurricane took down ancient poorly maintained levees on a city below sea level. Absent that, no problem.

d) No climate-related spread of various infectious diseases.

e) No known increase in droughts or floods.

f) No loss of Tuvalu or other coral atolls.

g) Actually, none of the horrendous outcomes or biblical plagues of frogs and the like which are supposed to accompany the Thermageddon™ of a two degree temperature rise occurred over the last two centuries. To the contrary, the increased warming seems to have been a net gain for most humans, animals, and plants. Nobody likes freezing their asterisk off, after all, and the warming has mostly been in extra-tropical winter nights. That’s the theory, at least, although the BEST data should be able to tell us more.

Conclusion 3. BEST has done the world a huge service by collating and collecting all the data in one place, and deserves credit for that.

But they have done the world a huge disservice by becoming media whores, by putting out a shabby imitation of science in Figure 1, and by making a host of claims before peer review is complete.

This last one astounds me, that they’ve done it before peer review is finished. Doug Keenan and William Briggs have both raised separate and cogent arguments that the BEST analysis contains flaws. That would make me nervous, they’re kinda heavyweights, although any man can be wrong … but no, the BEST folks are making a host of claims as though their paper has already passed peer review. It’s the same publicity circus that Muller put on for Congress. And what three-ring media circus would be complete without their own brand new personalized “hide the decline” poster?

Since they have held out for extreme transparency, or at least given lip-service to the idea, I would be very interested to find out the names of the reviewers.

Because certainly, one possible explanation of their brazen trumpeting of their results before the peer review process is finished is that the fix is in. Why else the confidence that the reviewers will not find fault with their work? It is extreme hubris at a minimum, which is reputed historically to have unpleasant sequelae involving wax and feathers …

w.

PS—The world is warming. It has been for centuries. Rather than saying anything about anthropogenic global warming, all the BEST dataset does is confirms that. How that’s gotten twisted into some supposed “victory” for the AGW crowd escapes me.

0 0 votes
Article Rating

Discover more from Watts Up With That?

Subscribe to get the latest posts sent to your email.

202 Comments
Inline Feedbacks
View all comments
Septic Matthew
October 22, 2011 6:26 pm

FWIW there is a Matlab interpreter for R. I’ll find it for you in a day or two.
Matthew

crosspatch
October 22, 2011 6:30 pm

A Category 3 hurricane took down ancient poorly maintained levees on a city below sea level. Absent that, no problem.

That is the part of the problem here. While you are correct that we really haven’t seen an increase in weather related calamities, the ones we are seeing are huge. Andrew, for example. That is simply because there are more people living where these things have the potential to strike. But as for Katrina, what many people are not aware of (because the news really didn’t cover it) is that the communities East of New Orleans in Mississippi were extremely hard hit. Some towns were basically scraped off the face of the earth. Those were mostly poor black and Viet Namese communities not associated with any major metro area. I had friends that went to the area around Biloxi, Gulf Port, and Long Beach Mississippi and were there for months.
http://en.wikipedia.org/wiki/File:Hurricane_katrina_damage_gulfport_mississippi.jpg
There were no levees at that location, that was all wind and storm surge.

observa
October 22, 2011 6:34 pm

Is that the BEST they can do?

October 22, 2011 6:44 pm

Dear Mr. Watts, if Matlab “does you no good at all” you probably are not qualified to talk about data analysis. Stand by while I download and run the code and included datasets, thereafter posting my results. I wish you the best of luck in downloading Octave (an open source version of Matlab), also known as GNU Octave and concurrently running the Berkeley code and data. If you have any questions, do not hesitate to email me for help. Your’s truly Chase Stoudt
REPLY: Here’s what I sent via email.

Dear Mr. Stoudt,
Thanks for your kind offer of assistance. Can you please help me identify who the author of the essay is? I may not be qualified to read either.
Best regards
Anthony Watts

John Tofflemire
October 22, 2011 6:56 pm

Steve McIntyre provides the following script in order to plot the BEST (presumably) monthly data:
download.file(“http://www.berkeleyearth.org/downloads/analysis-data.zip”,”temp.zip”,mode=”wb”)
handle=unz(“temp.zip”,”Full_Database_Average_summary.txt”)
x=scan(handle,sep=”\n”,what=””)
close(handle)
writeLines(x,”temp”)
x=read.table(“temp”,skip=19,colClasses=rep(“numeric”,5) )
names(x)=c(“year”,”anom”,”unc”,”smooth”,”sm_uncert”)
berk=ts(x[,2:5],start=x[1,1])
ts.plot(berk[,”anom”])
Would be curious to see if this is indeed the source of the monthly data.

Editor
October 22, 2011 6:57 pm

Chase Stoudt says:
Let’s hope your MATLAB and data analysis skills are better than your reading comprehension.

Roger Andrews
October 22, 2011 6:57 pm

Willis:
Would it be possible for you to post your digitized monthly data here? I could certainly use it, and I’m sure others could too. Thank you.

October 22, 2011 7:01 pm

Willis:
Let me be the first to say, as a RIGHT WING NEANDERTHAL REPTILE HATED REPUBLICAN, you are completely brilliant. That’s an indirect way of saying my judgements are made on the character of the analysis, not the color of a man’s environmental religion. (I’m still confused on that, are you white, grey, black or polka-dot?)
Keep up the marvelous work. You are the BEST defense anyone could ask for. Or is it the defense against BEST?
Max

R. Shearer
October 22, 2011 7:01 pm

Echo observa, could have been BETTER.

October 22, 2011 7:01 pm

Crosspatch,
The magnitude of the destruction is more a function of the growth in population in bad areas (like New Orleans), rather than the destructive power of the storm. Basically modern science has enabled people to live where they couldn’t before, and so the reasons we didn’t live there earlier – below sea level, flooding in the case of N.O. – we’re seeing more people affected. It’s not the weather – it’s our modern engineering capability, and the fact it is limited.

Jeff D
October 22, 2011 7:03 pm

Chase Stoudt says:
October 22, 2011 at 6:44 pm
Dear Mr. Watts, if Matlab “does you no good at all” you probably are not qualified to talk about data analysis.
____________________________
Hate to rain on your ever so clever jab at Anthony. This article was written by Willis. Soon as you get that downloaded and working let me know, I have a copy of ” See Spot Run ” laying around from when my son was 3.

Joel Shore
October 22, 2011 7:05 pm

PS—The world is warming. It has been for centuries. Rather than saying anything about anthropogenic global warming, all the BEST dataset does is confirms that. How that’s gotten twisted into some supposed “victory” for the AGW crowd escapes me.

Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.

Mike D in AB
October 22, 2011 7:06 pm

Robert – Let’s give the benefit of the doubt. Should Willis take Chase up on the invitation, we’ll gladly ignore the faux pas of naming the wrong author. Anthony ultimately has editorial control (and responsibility so far as his reputation is behind all posts), and although it is poor data analysis to not note who scribed a particular posting, but I don’t recall seeing Chase post here before and so I’m withholding condemnation until I see more of what Chase has to offer. (My condemnation or approval would mean little, of course, since I’m a long-time lurker and seldom post.)

HaroldW
October 22, 2011 7:07 pm

Willis,
Look here. The file “Full_Database_Average_complete.txt” has the monthly global averages.

Theo Goodwin
October 22, 2011 7:09 pm

Very well said, Willis. You raised some important ethical points. I would like to remind everyone that BEST played Bait and Switch with Anthony by switching to a 60 year analysis when he wanted a 30 year analysis beginning in 1979. Only the 30 year record includes siting data so BEST chose not to address what Anthony considered most important, questions about siting.

Septic Matthew
October 22, 2011 7:10 pm

Here it is:
P. Roebuck. matlab: MATLAB emulation package. http://CRAN.
R-project.org/package=matlab, 2008. R package version 0.8-2.

HaroldW
October 22, 2011 7:12 pm

Just to clarify, the link I posted above is to the data underlying your figure 1, not your figure 2.

Theo Goodwin
October 22, 2011 7:13 pm

Joel Shore says:
October 22, 2011 at 7:05 pm
“Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.”
BEST chose not to address the 30 years for which siting data is available but switched to a 60 year record and did no siting analysis. If you do not address the siting issues, as BEST does not, then you cannot criticize those who have argued that there are important siting issues. Total fail for BEST and your comment.
By the way, BEST played Bait and Switch with Anthony. They might make good used car salesmen.

Editor
October 22, 2011 7:15 pm

Mike D in AB says: October 22, 2011 at 7:06 pm
Mike, it’s like this: I’ve been doing computers for over 40 years and MATLAB doesn’t do me any good either. Somehow I don’t think command of MATLAB is the best basis for judging someone’s data analysis ability… but the ability to comprehend the written word is. Young Mr. Chase is just another drive-by.

Theo Goodwin
October 22, 2011 7:15 pm

Chase Stoudt says:
October 22, 2011 at 6:44 pm
“Dear Mr. Watts, if Matlab “does you no good at all” you probably are not qualified to talk about data analysis.”
Well, what a sweetheart you are! Here to make friends, right?

Jack O'Fall
October 22, 2011 7:17 pm

While I have not seen the actual data set with their 1.6 Billion data points from their 39,000+ observations points, I do have a calculator and am troubled by the small number of data points from each station. If their data goes back to 1940 (that seems about where they claim to have mostly eliminated their margin of error), that means there are 70 years of data, 840 months, multiplied by 39,000 stations equals 32.7 Million, with in the end means there are less than 50 temperature records for each station each month. Is that enough? I’m sure that means many stations don’t have anywhere near that many data points per month, or aren’t complete for that period.
Considering that they probably have data going back a lot further than 70 years, that again raised the question about how much those numbers actually mean. Are they just there to confuse everyone into thinking it is a massive comprehensive list?

Mike D in AB
October 22, 2011 7:18 pm

Joel Shore says:
October 22, 2011 at 7:05 pm
Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.

The rate of change is important. Poor siting masks or overstates the severity of the problem. If you’re doing 55 mph in a 55 zone, you’re ok. If you’re doing 150 in the 55 zone, you’re in trouble. This skeptic is of the opinion that we’re doing 30 in a 55 zone, and that the official speed of 100 mph is way out to lunch. Those of us who disbelieve the treemometer record know that we’re recovering from the little ice age, and expect temperatures to be increasing because temperatures don’t stay static. Where I live in Calgary is often under 2 miles of ice. I’m glad to be in an inter-glacial, and will do all that I can to help us stay here, but adopting alarmism is too much for me.

Frank Kotler
October 22, 2011 7:20 pm

There is, IMO, a subtle difference between “the world has warmed” and “the world is warming”. The latter implies facts not in evidence.

Mike D in AB
October 22, 2011 7:24 pm

Robert E. Phelan says:
October 22, 2011 at 7:15 pm

Point taken. I’ve studied 6 computer languages over the year (but claim fluency in none of them), so I see where you’re coming from. I failed a stats course twice because I refused to buy the proprietary software that statistics department insisted I use, and instead did the homework in a spreadsheet (which was immediately rejected because it wasn’t in the “right format”. The work should stand on its own, and placing it inside a limited framework (or one not amenable to common usage) is obfuscation rather than transparency.
I expect you’re right about the drive-by, but we do often get sincere but misguided folks here and it behooves us to not drive away the few who might actually be open minded enough to learn.

Jeff D
October 22, 2011 7:30 pm

Found this listed on Bishop Hill. Seems suited to the conversation.
Another review of BEST’s math, This one is more in depth.
http://wmbriggs.com/blog/?p=4530

1 2 3 9