A preliminary assessment of BEST's decline

Guest Post by Willis Eschenbach

With altogether far too much fanfare for my taste, the Berkeley Earth Surface Temperature (BEST) project has not released its preliminary results.

Or at least I can’t find them. I just wanted the month-by-month data that their hotrod new computer program spits out at the end of its run. The results they’re all hot and bothered about.

But despite releasing a massive database, 39,000 stations, along with the code in Matlab (which does me no good at all), I can’t find anywhere their freakin’ results. You know, the actual results of their work? The monthly average global temperature, the stuff that they mangled to produce things like their PR graph shown in Figure 1:

Figure 1. Purports to show that the BEST temperature record, and all the others as well, are all going “in one direction”, nowhere but up. I’m sure you remember the Climategate mantra, “hide the decline”? Keep that in mind as we proceed.

So … that will show those shifty skeptics, even BEST says it’s warming nonstop, evidence is right there before your eyes.

What’s not to like? How can you argue with that? The science is in.

Since I couldn’t get their results, I did the next best thing, and digitized their results. Even then, I was frunstrated. As far as I could tell, they never showed their actual results. The closest I found is in Figure 1 of their paper here:

Figure 2. Figure 1 of BEST’s “Decadal Variations” paper. Everybody’s going up, up, up, although you can’t really see what anyone is doing.

I blew that Figure up, and digitized it. Sixty years, 720 data points, boooring. Plus I hate it that they’ve smoothed the data, that makes it useless for statistical work. But it could have given us an idea of what’s going on in each of the records … if they hadn’t printed them atop one another in confusing colors. Enough of the spaghetti graphs already, you mad scientist persons, they show nothing! Figure 3 shows the BEST dataset along with the other datasets, this time displaced from each other so that we can actually see what’s happening:

Figure 3. The BEST land-only temperature record, compared to other surface and satellite land-only temperature records. 12 month moving average data, sadly. Note the decline.

[UPDATE: An alert reader noticed what I did not, that this is a subset of the BEST dataset that does not contain the stations used by the other groups (NOAA, etc). He points out that the full dataset is again different, in that it in fact rises more than the partial dataset. I have updated the figure and struck out some text to include that.

However, this doesn’t fix the questions. The post 1998 record from all of the BEST data is much more poorly correlated with the current records (~0.65) than prior to 1998 (~0.90). So this does not verify or validate the current groups datasets.

Hmmm … that gives a very different picture than Figure 1. Even with the bizarre 12-month moving average, the BEST record is clearly the outlier since 1998. You would think that in the modern era, the BEST would agree more closely with the other records. And indeed, from about 1975 to 1998 they were moving in something like lockstep.

But both before and after that time period, the BEST results are a clear outlier. And since 1998, BEST has been in a slow decline … funny how that didn’t show up in Figure 1. Yes, I know, a ten-year moving average shouldn’t show anything within five years from the end of the dataset. And I’m sure folks will argue that it’s just coincidence that they chose that exact smoothing length, and that it was the chance selection of colors that jumbled up the spaghetti graph so it’s unreadable … but y’know, after a while “coincidence” wears thin. I’m going with a more nuanced explanation, that it was a “deliberately unconscious choice to hide the decline”, although certainly you are welcome to stick to the story that it’s all just an unfortunate chain of events  …

CONCLUSIONS:

Conclusion 1. It is extremely sneaky to send a truncated, smoothed result like Figure 1 out to the media to announce your results. That’s advocacy disguised as science. They did it to make it look like the temperature was headed for the sky and that BEST agreed. Instead, BEST actually disagrees with the other datasets by claiming that over the last decade, land temperatures are dropping, not staying stable or rising as per the other datasets. Using a graph that didn’t show that is … curious. As Gollum would say … “Oooooh, tricksy”. Including you, Judith. Figure 1 was nothing but “hide the decline” PR spin. Bad scientists, no cookies.

Conclusion 1. The correlation between the old data points used by the current groups, and the new data used only by BEST, is quite poor after 1998. This is visible in the plots of both the partial and full BEST datasets. The reasons for this are not clear, but it provides no support for the current datasets.

Conclusion 2. First point. The raw terror point, the thought that has the AGW alarmists changing their shorts, is the dreaded 2°C rise that is forecast from CO2. That is supposed to be the mythical “tipping point”. Second point. If we look at the 10-year smoothed data in Figure 1, BEST says that in the last two centuries, the temperature has risen about two degrees.

Let me note that over that two-century time period there have been:

a) No known increase in extreme weather events.

b) No known increase in catastrophes (other than from increased populations and property in vulnerable areas).

c) No major costs, deaths or damage from sea level rise. And don’t bother me with Katrina. A Category 3 hurricane took down ancient poorly maintained levees on a city below sea level. Absent that, no problem.

d) No climate-related spread of various infectious diseases.

e) No known increase in droughts or floods.

f) No loss of Tuvalu or other coral atolls.

g) Actually, none of the horrendous outcomes or biblical plagues of frogs and the like which are supposed to accompany the Thermageddon™ of a two degree temperature rise occurred over the last two centuries. To the contrary, the increased warming seems to have been a net gain for most humans, animals, and plants. Nobody likes freezing their asterisk off, after all, and the warming has mostly been in extra-tropical winter nights. That’s the theory, at least, although the BEST data should be able to tell us more.

Conclusion 3. BEST has done the world a huge service by collating and collecting all the data in one place, and deserves credit for that.

But they have done the world a huge disservice by becoming media whores, by putting out a shabby imitation of science in Figure 1, and by making a host of claims before peer review is complete.

This last one astounds me, that they’ve done it before peer review is finished. Doug Keenan and William Briggs have both raised separate and cogent arguments that the BEST analysis contains flaws. That would make me nervous, they’re kinda heavyweights, although any man can be wrong … but no, the BEST folks are making a host of claims as though their paper has already passed peer review. It’s the same publicity circus that Muller put on for Congress. And what three-ring media circus would be complete without their own brand new personalized “hide the decline” poster?

Since they have held out for extreme transparency, or at least given lip-service to the idea, I would be very interested to find out the names of the reviewers.

Because certainly, one possible explanation of their brazen trumpeting of their results before the peer review process is finished is that the fix is in. Why else the confidence that the reviewers will not find fault with their work? It is extreme hubris at a minimum, which is reputed historically to have unpleasant sequelae involving wax and feathers …

w.

PS—The world is warming. It has been for centuries. Rather than saying anything about anthropogenic global warming, all the BEST dataset does is confirms that. How that’s gotten twisted into some supposed “victory” for the AGW crowd escapes me.

Advertisements

  Subscribe  
newest oldest most voted
Notify of
Septic Matthew

FWIW there is a Matlab interpreter for R. I’ll find it for you in a day or two.
Matthew

crosspatch

A Category 3 hurricane took down ancient poorly maintained levees on a city below sea level. Absent that, no problem.

That is the part of the problem here. While you are correct that we really haven’t seen an increase in weather related calamities, the ones we are seeing are huge. Andrew, for example. That is simply because there are more people living where these things have the potential to strike. But as for Katrina, what many people are not aware of (because the news really didn’t cover it) is that the communities East of New Orleans in Mississippi were extremely hard hit. Some towns were basically scraped off the face of the earth. Those were mostly poor black and Viet Namese communities not associated with any major metro area. I had friends that went to the area around Biloxi, Gulf Port, and Long Beach Mississippi and were there for months.
http://en.wikipedia.org/wiki/File:Hurricane_katrina_damage_gulfport_mississippi.jpg
There were no levees at that location, that was all wind and storm surge.

observa

Is that the BEST they can do?

Dear Mr. Watts, if Matlab “does you no good at all” you probably are not qualified to talk about data analysis. Stand by while I download and run the code and included datasets, thereafter posting my results. I wish you the best of luck in downloading Octave (an open source version of Matlab), also known as GNU Octave and concurrently running the Berkeley code and data. If you have any questions, do not hesitate to email me for help. Your’s truly Chase Stoudt
REPLY: Here’s what I sent via email.

Dear Mr. Stoudt,
Thanks for your kind offer of assistance. Can you please help me identify who the author of the essay is? I may not be qualified to read either.
Best regards
Anthony Watts

John Tofflemire

Steve McIntyre provides the following script in order to plot the BEST (presumably) monthly data:
download.file(“http://www.berkeleyearth.org/downloads/analysis-data.zip”,”temp.zip”,mode=”wb”)
handle=unz(“temp.zip”,”Full_Database_Average_summary.txt”)
x=scan(handle,sep=”\n”,what=””)
close(handle)
writeLines(x,”temp”)
x=read.table(“temp”,skip=19,colClasses=rep(“numeric”,5) )
names(x)=c(“year”,”anom”,”unc”,”smooth”,”sm_uncert”)
berk=ts(x[,2:5],start=x[1,1])
ts.plot(berk[,”anom”])
Would be curious to see if this is indeed the source of the monthly data.

Chase Stoudt says:
Let’s hope your MATLAB and data analysis skills are better than your reading comprehension.

Roger Andrews

Willis:
Would it be possible for you to post your digitized monthly data here? I could certainly use it, and I’m sure others could too. Thank you.

Willis:
Let me be the first to say, as a RIGHT WING NEANDERTHAL REPTILE HATED REPUBLICAN, you are completely brilliant. That’s an indirect way of saying my judgements are made on the character of the analysis, not the color of a man’s environmental religion. (I’m still confused on that, are you white, grey, black or polka-dot?)
Keep up the marvelous work. You are the BEST defense anyone could ask for. Or is it the defense against BEST?
Max

R. Shearer

Echo observa, could have been BETTER.

Crosspatch,
The magnitude of the destruction is more a function of the growth in population in bad areas (like New Orleans), rather than the destructive power of the storm. Basically modern science has enabled people to live where they couldn’t before, and so the reasons we didn’t live there earlier – below sea level, flooding in the case of N.O. – we’re seeing more people affected. It’s not the weather – it’s our modern engineering capability, and the fact it is limited.

Jeff D

Chase Stoudt says:
October 22, 2011 at 6:44 pm
Dear Mr. Watts, if Matlab “does you no good at all” you probably are not qualified to talk about data analysis.
____________________________
Hate to rain on your ever so clever jab at Anthony. This article was written by Willis. Soon as you get that downloaded and working let me know, I have a copy of ” See Spot Run ” laying around from when my son was 3.

Joel Shore

PS—The world is warming. It has been for centuries. Rather than saying anything about anthropogenic global warming, all the BEST dataset does is confirms that. How that’s gotten twisted into some supposed “victory” for the AGW crowd escapes me.

Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.

Mike D in AB

Robert – Let’s give the benefit of the doubt. Should Willis take Chase up on the invitation, we’ll gladly ignore the faux pas of naming the wrong author. Anthony ultimately has editorial control (and responsibility so far as his reputation is behind all posts), and although it is poor data analysis to not note who scribed a particular posting, but I don’t recall seeing Chase post here before and so I’m withholding condemnation until I see more of what Chase has to offer. (My condemnation or approval would mean little, of course, since I’m a long-time lurker and seldom post.)

HaroldW

Willis,
Look here. The file “Full_Database_Average_complete.txt” has the monthly global averages.

Theo Goodwin

Very well said, Willis. You raised some important ethical points. I would like to remind everyone that BEST played Bait and Switch with Anthony by switching to a 60 year analysis when he wanted a 30 year analysis beginning in 1979. Only the 30 year record includes siting data so BEST chose not to address what Anthony considered most important, questions about siting.

Septic Matthew

Here it is:
P. Roebuck. matlab: MATLAB emulation package. http://CRAN.
R-project.org/package=matlab, 2008. R package version 0.8-2.

HaroldW

Just to clarify, the link I posted above is to the data underlying your figure 1, not your figure 2.

Theo Goodwin

Joel Shore says:
October 22, 2011 at 7:05 pm
“Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.”
BEST chose not to address the 30 years for which siting data is available but switched to a 60 year record and did no siting analysis. If you do not address the siting issues, as BEST does not, then you cannot criticize those who have argued that there are important siting issues. Total fail for BEST and your comment.
By the way, BEST played Bait and Switch with Anthony. They might make good used car salesmen.

Mike D in AB says: October 22, 2011 at 7:06 pm
Mike, it’s like this: I’ve been doing computers for over 40 years and MATLAB doesn’t do me any good either. Somehow I don’t think command of MATLAB is the best basis for judging someone’s data analysis ability… but the ability to comprehend the written word is. Young Mr. Chase is just another drive-by.

Theo Goodwin

Chase Stoudt says:
October 22, 2011 at 6:44 pm
“Dear Mr. Watts, if Matlab “does you no good at all” you probably are not qualified to talk about data analysis.”
Well, what a sweetheart you are! Here to make friends, right?

Jack O'Fall

While I have not seen the actual data set with their 1.6 Billion data points from their 39,000+ observations points, I do have a calculator and am troubled by the small number of data points from each station. If their data goes back to 1940 (that seems about where they claim to have mostly eliminated their margin of error), that means there are 70 years of data, 840 months, multiplied by 39,000 stations equals 32.7 Million, with in the end means there are less than 50 temperature records for each station each month. Is that enough? I’m sure that means many stations don’t have anywhere near that many data points per month, or aren’t complete for that period.
Considering that they probably have data going back a lot further than 70 years, that again raised the question about how much those numbers actually mean. Are they just there to confuse everyone into thinking it is a massive comprehensive list?

Mike D in AB

Joel Shore says:
October 22, 2011 at 7:05 pm
Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.

The rate of change is important. Poor siting masks or overstates the severity of the problem. If you’re doing 55 mph in a 55 zone, you’re ok. If you’re doing 150 in the 55 zone, you’re in trouble. This skeptic is of the opinion that we’re doing 30 in a 55 zone, and that the official speed of 100 mph is way out to lunch. Those of us who disbelieve the treemometer record know that we’re recovering from the little ice age, and expect temperatures to be increasing because temperatures don’t stay static. Where I live in Calgary is often under 2 miles of ice. I’m glad to be in an inter-glacial, and will do all that I can to help us stay here, but adopting alarmism is too much for me.

Frank Kotler

There is, IMO, a subtle difference between “the world has warmed” and “the world is warming”. The latter implies facts not in evidence.

Mike D in AB

Robert E. Phelan says:
October 22, 2011 at 7:15 pm

Point taken. I’ve studied 6 computer languages over the year (but claim fluency in none of them), so I see where you’re coming from. I failed a stats course twice because I refused to buy the proprietary software that statistics department insisted I use, and instead did the homework in a spreadsheet (which was immediately rejected because it wasn’t in the “right format”. The work should stand on its own, and placing it inside a limited framework (or one not amenable to common usage) is obfuscation rather than transparency.
I expect you’re right about the drive-by, but we do often get sincere but misguided folks here and it behooves us to not drive away the few who might actually be open minded enough to learn.

Jeff D

Found this listed on Bishop Hill. Seems suited to the conversation.
Another review of BEST’s math, This one is more in depth.
http://wmbriggs.com/blog/?p=4530

Werner Brozek

Regarding Figure 1, I have seen this “game” played before to show there warming where there really isn’t. Let us presume that from 1990 to 2000, there is an even warming. Then let us presume that from 2000 to 2010, there is absolutely no warming (nor cooling) whatsoever. What these people do is take the average from 1990 to 2000, then from 1991 to 2001, and so on until 2000 to 2010. By this “trick”, they can prove to you that it has been warming X degrees per decade from 2000 to 2005 even though the temperature may not have gone up a hundredth of a degree from 2000 to 2005.

Joel Shore

Theo Goodwin says:

BEST chose not to address the 30 years for which siting data is available but switched to a 60 year record and did no siting analysis. If you do not address the siting issues, as BEST does not, then you cannot criticize those who have argued that there are important siting issues. Total fail for BEST and your comment.

They did address these issues, just not in exactly the precise way Anthony would have liked them to. In any such analysis, there will be literally hundreds of decisions that have to be made. One can always find something to criticize if one does not want to accept the results of the analysis! I would suggest you might want to practice a little bit of skepticism here.
They also have apparently released their data, so others can re-run it making various changes…but I doubt there will be any rush on the part of the critiquers to do this.

Joel Shore

Mike D in AB says:

This skeptic is of the opinion that we’re doing 30 in a 55 zone, and that the official speed of 100 mph is way out to lunch.

Well, that may be your opinion but it is certainly not the conclusion that BEST reached.

Mike D in AB
“…it behooves us to not drive away the few who might actually be open minded enough to learn…”
Do a google search on “Chase Stoudt”. The young gentleman may be sincere and is certainly misguided, but he is not here to learn. He’s reached the stage in his life where he knows everything and needs to let everyone know just how superior his intellect is.

If Willis’ suppositions hold up,
has BEST gone BUST?

toto

Willis, you really outdid yourself this time.
The figure you plotted is not the actual Berkeley reconstruction of land temperatures. As explained in the article and in the very caption of the figure that you are showing, it is a partial reconstruction made using only data that was not used by the other records, for the specific purpose of this particular paper (analysing the effects of the AMO).
The real, full Berkeley reconstruction is plotted in Figures 5 and 8 of their first paper on their website – the one that actually describes their methods and results. Which apparently you have utterly failed to read.
As it turns out, the actual Berkeley reconstruction shows the exact opposite of what you are ranting about. Namely, the Berkeley reconstruction runs hotter than both GISS and HadCrut (but closely follows NOAA) in the last decade.
Obviously that didn’t prevent you from throwing copious accusations of dishonesty, based on nothing else than your own misunderstanding of papers that you either read casually or did not read at all.
I encourage all WUWT reader to check the papers by themselves and draw their own conclusions. Keep that in mind for the next time you see one of Willis’ rants.

What is the status of the global curve (as opposed to land only)?

Werner Brozek

“Even with the bizarre 12-month moving average, the BEST record is clearly the outlier since 1998.”
From a different post:
“Bill Illis says:
October 22, 2011 at 9:37 am
(It also looks like BEST has an error in their database for April, 2010 which should be +1.035C rather than -1.035C – it is such an outlier compared to the trend and to other datasets – that means all their moving averages have to recalculated as well).”
I compared BEST for March , April, and May for 2010 and the values were 0.859, -1.035 and 1.098.
Hadcrut3, for the same period, had 0.583, 0.571 and 0.516.
GISS, for the same period, had 1.06, 0.87 and 0.87.
While I do not expect identical results, at least the numbers should be in the ball park. I agree with Bill that the BEST April reading should probably have been +1.035. With this correction, BEST does not look quite as different, however it seems clear that BEST will not support GISS that 2010 was the hottest year on record when all numbers are in.

Tim Minchin

Shouldn’t we be more concerned with hourly data to see whether a day has heated or cooled in general – We should be measuring temperature in an analogue continuous line and then calculating the average temperate for the day based on that rather than the daily maximum and minimums. Who is to say one day is hotter than another day just because one day had a higher maximum – what if taht maximum fell rapidly in the late afternoon – compared to a lower maximum that lasts long into the night – the total heat for a day othe average heat per hour or minute should be the basis of all discussion.

I would be very interested to find out the names of the reviewers.
And more importantly: the text of the reviews.

sorepaw

When we began our study, we felt that skeptics had raised legitimate issues, and we didn’t know what we’d find. Our results turned out to be close to those published by prior groups. We think that means that those groups had truly been very careful in their work, despite their inability to convince some skeptics of that. They managed to avoid bias in their data selection, homogenization and other corrections.
This is from Muller’s Wall Street Journal piece, dated October 21.
The distinct impression I get from the piece is that Muller (a) has an inflated ego and (b) is simultaneously attempting to mollify the NOAA, CRU, GISS, and other Warmista groups and grab a piece of their action.
It will be most interesting to see what the actual journal publications look like.

Steve McIntyre

Willis, their global average is online. I posted the following retrieval script at A and Cl Etc.

download.file(“http://www.berkeleyearth.org/downloads/analysis-data.zip”,”temp.zip”,mode=”wb”)
handle=unz(“temp.zip”,”Full_Database_Average_summary.txt”)
#handle=unz(“d:/climate/data/berkeley/analysis-data.zip”,”Full_Database_Average_summary.txt”)
x=scan(handle,sep=”\n”,what=””)
close(handle)
writeLines(x,”temp”)
x=read.table(“temp”,skip=19,colClasses=rep(“numeric”,5) )
names(x)=c(“year”,”anom”,”unc”,”smooth”,”sm_uncert”)
berk=ts(x[,2:5],start=x[1,1])
ts.plot(berk[,”anom”])

Thanx Willis, compelling read as always.
I notice after the end of the BEST data, the others drop about 0.1 to 0.3 Deg. (just eyeballing)
Assuming BEST follows that, it would mean the current BEST temperature would be the same as about the late 70s early 80s?

Theo Goodwin

Joel Shore says:
October 22, 2011 at 7:31 pm
“Theo Goodwin says:
BEST chose not to address the 30 years for which siting data is available but switched to a 60 year record and did no siting analysis. If you do not address the siting issues, as BEST does not, then you cannot criticize those who have argued that there are important siting issues. Total fail for BEST and your comment.”
“They did address these issues, just not in exactly the precise way Anthony would have liked them to. In any such analysis, there will be literally hundreds of decisions that have to be made. One can always find something to criticize if one does not want to accept the results of the analysis! I would suggest you might want to practice a little bit of skepticism here.”
They did address these issues but you cannot say how? Is that your non-response? It is a non-response. Also, did they or did they not pull a Bait and Switch on Anthony? Did they or did they not switch a 60 year record for Anthony’s 30 year record with siting data? If you want to correspond with me, answer the question.
“They also have apparently released their data, so others can re-run it making various changes…but I doubt there will be any rush on the part of the critiquers to do this.”
Their data is worthless because they used a 60 year period. They owe it to Anthony to do the 30 year analysis that he expected. They also owe him a huge apology. They also owe the media and the scientific community for going to the media with non-peer-reviewed work.

Jimmy Haigh

What’s this? “Climate scientists” hiding a decline? Who would have thunk it?

Willis Eschenbach

Chase Stoudt says:
October 22, 2011 at 6:44 pm

Dear Mr. Watts, if Matlab “does you no good at all” you probably are not qualified to talk about data analysis. Stand by while I download and run the code and included datasets, thereafter posting my results. I wish you the best of luck in downloading Octave (an open source version of Matlab), also known as GNU Octave and concurrently running the Berkeley code and data. If you have any questions, do not hesitate to email me for help. Your’s truly Chase Stoudt

Anthony Watts is not the author of the post, I am. And if you think being good at Matlab is a requirement for talking about data analysis, you’re a computer language snob. I speak half a dozen computer languages. I have no more interest in learning yours than you have in learning mine.
w.

Willis Eschenbach

John Tofflemire says:
October 22, 2011 at 6:56 pm

Steve McIntyre provides the following script in order to plot the BEST (presumably) monthly data:
download.file(“http://www.berkeleyearth.org/downloads/analysis-data.zip”,”temp.zip”,mode=”wb”)
handle=unz(“temp.zip”,”Full_Database_Average_summary.txt”)
x=scan(handle,sep=”\n”,what=””)
close(handle)
writeLines(x,”temp”)
x=read.table(“temp”,skip=19,colClasses=rep(“numeric”,5) )
names(x)=c(“year”,”anom”,”unc”,”smooth”,”sm_uncert”)
berk=ts(x[,2:5],start=x[1,1])
ts.plot(berk[,”anom”])
Would be curious to see if this is indeed the source of the monthly data.

Indeed, that is the source of the monthly data. Many thanks.
w.

Willis Eschenbach

Joel Shore says:
October 22, 2011 at 7:05 pm

PS—The world is warming. It has been for centuries. Rather than saying anything about anthropogenic global warming, all the BEST dataset does is confirms that. How that’s gotten twisted into some supposed “victory” for the AGW crowd escapes me.

Well, maybe if the “skeptic” crowd hadn’t spent so much time talking about how poor the siting of the stations was and making all these grandiose charges of data manipulation on the part of CRU and GISS, and so forth, it would not be viewed as such a big victory. One reaps what one sows.

Thanks, Joel. If the BEST results had verified and agreed with e.g. the post 1998 GHCN/GISS/CRUTEM results, then you could crow.
Since those BEST results do not resemble the others, and are a clear outlier, I’d say that the earlier results and methods have not been verified or validated as you seem to believe.
w.

Tilo Reber

Willis:
The monthly data you are looking for is here. Don’t feel bad, other people couldn’t find it either. And they certainly didn’t put it under their data section.
http://www.berkeleyearth.org/analysis.php
Go to the bottom of the page and click on “analysis chart data”.
I’ve already plotted it. The trend since 98 is strongly positive. Oddly enough, the chart they created for the decadal analysis paper using 2000 previously unused records looks like it has a negative trend since 98. They don’t bother to explain the divergence. And I can’t find the data for that 2000 record example.

Steve McIntyre

Willis, I’m puzzled why you place such weight on “peer review” as presently undertaken in climate journals. The articles will get and are getting far more effective review on the blogs than journals can provide.
In fields other than climate, circulation of working papers for comment is common practice. I wish that we’d had the opportunity to do this with our article on Steig et al.

Willis Eschenbach

toto says:
October 22, 2011 at 7:44 pm

Willis, you really outdid yourself this time.
The figure you plotted is not the actual Berkeley reconstruction of land temperatures. As explained in the article and in the very caption of the figure that you are showing, it is a partial reconstruction made using only data that was not used by the other records, for the specific purpose of this particular paper (analysing the effects of the AMO).
The real, full Berkeley reconstruction is plotted in Figures 5 and 8 of their first paper on their website – the one that actually describes their methods and results. Which apparently you have utterly failed to read.
As it turns out, the actual Berkeley reconstruction shows the exact opposite of what you are ranting about. Namely, the Berkeley reconstruction runs hotter than both GISS and HadCrut (but closely follows NOAA) in the last decade.
Obviously that didn’t prevent you from throwing copious accusations of dishonesty, based on nothing else than your own misunderstanding of papers that you either read casually or did not read at all.
I encourage all WUWT reader to check the papers by themselves and draw their own conclusions. Keep that in mind for the next time you see one of Willis’ rants.

Thanks, toto. I encourage readers to do the same, nice catch. You are right, I was wrong … but the point is still valid. What they are showing is the new, never before seen BEST data.
So despite my acknowledged error, the question remains. Why do the stations we’ve never seen show a different result than the stations we have seen?
w.

Septic Matthew

Willis wrote: Conclusion 1. It is extremely sneaky to send a truncated, smoothed result like Figure 1 out to the media to announce your results. That’s advocacy disguised as science. They did it to make it look like the temperature was headed for the sky and that BEST agreed. Instead, BEST actually disagrees with the other datasets by claiming that over the last decade, land temperatures are dropping, not staying stable or rising as per the other datasets. Using a graph that didn’t show that is … curious. As Gollum would say … “Oooooh, tricksy”. Including you, Judith. Figure 1 was nothing but “hide the decline” PR spin. Bad scientists, no cookies.
I disagree with that conclusion. They posted their drafts, their data, and their code. Anybody can check their work. the BEST downturn toward 2010 has been noted by others. What the BEST team might do is comment on exactly how their methodology produced that discrepancy from the other, as well as discrepancies at other years.

Septic Matthew

toto says:
October 22, 2011 at 7:44 pm
Oh shucks. I have seen that graph numerous times without reading the caption all the way through. The differences still should be commented on, I think, but the sampling is obviously the source of the difference.

0.5.) Open Matlab/Download Octave (These directions are only written for Matlab since I have work tomorrow, and will mess with the Octave directions then)
1.) Download the code/data from
http://berkeleyearth.org/our-code.php
http://berkeleyearth.org/data.php (the .mat for data in this case)
respectively
1.) Read the read me!!!!
2.) open the script temperatureStartup.m after unpacking the .zips. It is located in “/AnalysisCode/Export/Code/temperatureStartup.m” (without the quotes)
3.) add these 3 variables to the scripts like this in Matlab/Octave depending on your setup windows or mac (mac way shown)
psep = (‘/’)
temperature_software_dir = ‘/mycomputer/AnalysisCode/Export/Code’
temperature_data_dir = ‘/mycomputer/PreliminaryMatlabDataset’
temperature_scratch_dir = ‘/mycomputer/BEST/out’
4.) The BEST people forgot to include one .mfile in the .zip.
type “edit /mycomputer/BEST/temperatureGlobals”
(I’m emailing them tonight about this)
then edit the script to read
“load /BEST/Complete_Prelim_Dataset.mat”
save it and close. Whenever you run this .m, watch out it’s a lot of data and will take a long time to load.
5.) Run temperatureGlobals.m that you created(just type it into the command window no .m needed). Like mentioned above it will take awhile to load.
6.) type the command “results = BerkeleyAverage(se,sites,’quick’);”
7.) Have fun boys and girls, remember the help command is your friend. Will post all the graphs they include in the toolbox tomorrow. I need to sleep before work. And I apologize for not giving you credit to this post Eschenbach. I’m willing to learn any new computer language, the question is are you?

David Falkner

Anthony: Mea Culpa for my coffee joke on the other thread. I thought it was light-hearted and funny. I am capable of mean that would make the Devil blush, I guess. It is funny though.
Willis: The BEST also seems to show a much greater variability through out the data set. Of course, it’s hard to tell without the data to find the deviation. And eyeballing is tricky with all those graphs next to each other, but with some visual aiding (an index card) it looks like it is so. Pea and thimble? Does changing the amount of variation in the data increase the uncertainty in the attributions? I wouldn’t see how it could be avoided.