Guest post by Jeff Id 
I will leave this alone for another week or two while I wait for a reply to my emails to the BEST group, but there are three primary problems with the Berkeley temperature trends which must be addressed if the result is to be taken seriously. Now by seriously, I don’t mean by the IPCC which takes all alarmist information seriously, but by the thinking person.
Here’s the points:
1 – Chopping of data is excessive. They detect steps in the data, chop the series at the steps and reassemble them. These steps wouldn’t be so problematic if we weren’t worrying about detecting hundredths of a degree of temperature change per year. Considering that a balanced elimination of up and down steps in any algorithm I know of would always detect more steps in the opposite direction of trend, it seems impossible that they haven’t added an additional amount of trend to the result through these methods.
Steve McIntyre discusses this here. At the very least, an examination of the bias this process could have on the result is required.
2 – UHI effect. The Berkeley study not only failed to determine the magnitude of UHI, a known effect on city temperatures that even kids can detect, it failed to detect UHI at all. Instead of treating their own methods with skepticism, they simply claimed that UHI was not detectable using MODIS and therefore not a relevent effect.
This is not statistically consistent with prior estimates, but it does verify that the effect is very small, and almost insignificant on the scale of the observed warming (1.9 ± 0.1 °C/100yr since 1950 in the land average from figure 5A).
This is in direct opposition to Anthony Watts surfacestation project which through greater detail was very much able to detect the ‘insignificant’ effect.
Summary and Discussion
The classification of 82.5% of USHCNv2 stations based on CRN criteria provides a unique opportunity for investigating the impacts of different types of station exposure on temperature trends, allowing us to extend the work initiated in Watts [2009] and Menne et al. [2010].
The comparison of time series of annual temperature records from good and poor exposure sites shows that differences do exist between temperatures and trends calculated from USHCNv2 stations with different exposure characteristics. 550 Unlike Menne et al. [2010], who grouped all USHCNv2 stations into two classes and found that “the unadjusted CONUS minimum temperature trend from good and poor exposure sites … show only slight differences in the unadjusted data”, we found the raw (unadjusted) minimum temperature trend to be significantly larger when estimated from the sites with the poorest exposure sites relative to the sites with the best exposure. These trend differences were present over both the recent NARR overlap period (1979-2008) and the period of record (1895-2009). We find that the partial cancellation Menne et al. [2010] reported between the effects of time of observation bias adjustment and other adjustments on minimum temperature trends is present in CRN 3 and CRN 4 stations but not CRN 5 stations. Conversely, and in agreement with Menne et al. [2010], maximum temperature trends were lower with poor exposure sites than with good exposure sites, and the differences in
trends compared to CRN 1&2 stations were statistically significant for all groups of poorly sited stations except for the CRN 5 stations alone. The magnitudes of the significant trend differences exceeded 0.1°C/decade for the period 1979-2008 and, for minimum temperatures, 0.7°C per century for the period 1895-2009.
The non-detection of UHI by Berkeley is NOT a sign of a good quality result considering the amazing detail that went into Surfacestations by so many people. A skeptical scientist would be naturally concerned by this and it leaves a bad taste in my mouth to say the least that the authors aren’t more concerned with the Berkeley methods. Either surfacestations very detailed, very public results are flat wrong or Berkeley’s black box literal “characterization from space” results are.
Someone needs to show me the middle ground here because I can’t find it.
I sent this in an email to Dr. Curry:
Non-detection of UHI is a sign of problems in method. If I had the time, I would compare the urban/rural BEST sorting with the completed surfacestations project. My guess is that the comparison of methods would result in a non-significant relationship.
3 – Confidence intervals.
The confidence intervals were calculated in this method by eliminating a portion of the temperature stations and looking at the noise that the elimination created. Lubos Motl described the method accurately as intentionally ‘damaging’ the dataset. It is a clever method to identify the sensitivity of the method and result to noise. The problem is that the amount of damage assumed is equal to the percentage of temperature stations which were eliminated. Unfortunately the high variance stations are de-weighted by intent in the processes such that the elimination of 1/8 of the stations is absolutely no guarantee of damaging 1/8 of the noise. The ratio of eliminated noise to change in final result is assumed to be 1/8 and despite some vague discussion of Monte-Carlo verifications, no discussion of this non-linearity was even attempted in the paper.
Prayer to the AGW gods.
All that said, I don’t believe that warming is undetectable or that temperatures haven’t risen this century. I believe that CO2 helps warming along as the most basic physics proves. My objection has always been to the magnitude caused by man, the danger and the literally crazy “solutions”. Despite all of that, this temperature series is statistically speaking, the least impressive on the market. Hopefully, the group will address my confidence interval critiques, McIntyre’s very valid breakpoint detection issues and a more in depth UHI study.
Holding of breath is not advised.
Robin Hewitt,
Sorry for the spelling. I’ve corrected the post at tAV.
malagaview says:
November 4, 2011 at 2:25 am
AT LAST – an intelligent analysis of a temperature data set!…
_________________________________________
Agreed. Sort of blows holes in all the “Official” data sets does it not?
It also blows holes in the CO2 has cause the warming propaganda.
I would like to see the info posted here too.
Spen says:
November 4, 2011 at 8:51 am
I am still puzzled by confidence levels/error range. I assume the accuracy of the older temperature measurements was no better than +/- 0.5 deg C. Shouldn’t that degree of accuracy apply to the anomaly?
_________________________________
You might want to look at AJ Strata’s article about error in the temp record.
http://strata-sphere.com/blog/index.php/archives/11420
Jeff, in relation to your #1 here is a comparison of FFT of Berkeley-est and HadcruT3 (land and sea).
http://tinypic.com/view.php?pic=24qu049&s=5
Up to around 20y period (5 on the /century scale) seem possible but the longer frequencies seem to have been decimated by the scalpel.
As you noted , they need to asses the effects of this processing and report it in the paper.
Hopefully this will address this lacuna before it gets published.
well done Verity for resisting the passive aggressive teenager that Lord Stephen Mosher is trying to become – he is obviously becoming more and more opposed to the idea of truth as he grows ever younger and more immature
Why is it that UHI effect always get upped by the models when they either should stay the same or rather be simmered down due to the mitigating effect that we create as well by planting tress, creating reservoirs/lakes, irrigation of dried out land, and so on and so forth.
Why is weather always chaotic but climate is always simplified beyond disbelief to a über linearity?
Why is the temperature said to be rising when nobody are even certain of what the temperature was before it supposedly started to rise?
@Septic Matthew says: They cut and splice where there is a jump discontinuity in the data. It is the act that produced the jump discontinuity (which may have been relocating the thermostat, or putting an asphalt runway near it) that perturbed the low frequency signal. Cutting and slicing restores the low frequency signal that the jump discontinuity perverted.
It is a legitimate intent to preserve the low frequency (GW and UHI) part of the spectrum. But does the suture do that? Have I made the point clear enough that the splices cannot contain the low frequencies? The low frequencies must be part of the gluing process. And just what data do they use to tune the glue? Hmmm?
Let me give you an example. One of the discontinuities you cite is a station move. Now station moves can be for many different reasons, but I’ll wager that a disproportionate number are positively related to urban pressure. At a station that used to be a Class 1, building modifications, parking lot paving, build up of neighbors, the station class has grown to 4. We can’t have that! So we move the station to a new Class 1 location. The right thing to do, but…. Discontinuity! Over the course of a century, this happens 4 times, once per 20 year span. We have a temperature record that is a bit saw-toothed with a UHI rise in each tooth. Deft use of the scalpel slices this 100 year record with 4 discontinuities into 5 “clean” pieces.
They are glued together… HOW? Eliminate the saw-tooth and make a ramp? If “the trend is the only thing you can trust” how can they do anything else?
I say the use of the scalpel and suture has made a bad data situation much worse. In the example I presented above, which of these is closer to the truth of the real regional temperature record? A) To remove the discontinuities? OR B) LEAVE THEM IN? I say use the entire record, uncut, unaltered. Leave the low frequency in. Isn’t the very act of moving the station an exemplary way to account and correct for a UHI problem? The ‘discontinuities” are not noise. They are an important part of the signal and must not be removed. The discontinuities are themselves the removal of a great deal of UHI from the record.
If you use one long saw-toothed record, you will have a strong low frequency signal that will be a combination of GW and UHI. Yes, UHI will contaminate the signal. But the UHI component is in the teeth of the wave, the gradual buildup between moves. By moving the station, the base temperature ought to drop back down due to lower UHI effects to the GW region trend. The overall century-long trend in the station might be (GW + ½ UHI). At least that is an upper bound on GW. But if you cut at discontinuities, treat the trend as your friend, and discount the absolute temperatures, then after splicing out the discontinuities your long term reconstructed signal will probably be (GW + 3.5 UHI), vastly overestimating the real GW trend.
I have used this station-move-because-of-UHI example to make an illustration of my point. Is it contrived? I think not. I think it represents many discontinuity problems, but not all. Be that as it may, my main point is that the scalpel destroys the very data we seek in the GW argument. The suture, the glue, is no guarantee the original signal is preserved. Indeed, the glue can be a major source of corruption or counterfeiting.
“old engineer says:
November 4, 2011 at 12:01 pm
Now assume that station B starts to be engulfed in urbanization from a nearby city. Station B temperature starts being greater than A because of the urban heat island.”
This is an excellent point. I assume we know how much hotter an urban station is from a rural station. What we need to know now is how many stations went from rural to urban during the time involved. Or perhaps we should even eliminate all readings where there was a rural to urban change to get the true change in global temperature?
Mike Bromley the Kurd,
“People who frequent “preprint libraries”, although vast in number, are not qualified to critique methodology.”
People who submit to those libraries apparently are often not qualified to critique their own methods either!
diogenes,
Mosher does have one glaring personality flaw. He is impatient with the stubbornly stupid. Stings, don’t it. You can’t carry Mosher’s jock strap. Your lamp has gone out.
Stephen Rasey says:
November 4, 2011 at 4:52 pm
Let me give you an example. One of the discontinuities you cite is a station move. Now station moves can be for many different reasons, but I’ll wager that a disproportionate number are positively related to urban pressure. At a station that used to be a Class 1, building modifications, parking lot paving, build up of neighbors, the station class has grown to 4. We can’t have that! So we move the station to a new Class 1 location. The right thing to do, but…. Discontinuity! Over the course of a century, this happens 4 times, once per 20 year span. We have a temperature record that is a bit saw-toothed with a UHI rise in each tooth. Deft use of the scalpel slices this 100 year record with 4 discontinuities into 5 “clean” pieces.
They are glued together… HOW? Eliminate the saw-tooth and make a ramp? If “the trend is the only thing you can trust” how can they do anything else?
I say the use of the scalpel and suture has made a bad data situation much worse.
Stephen, could not agree more with what you have said. Well put. That is pure common sense, not even needing deep statistics, and you know this is happening, the trends are being manufactured to some degree, intentionally or not from such manipulation of the data.
So many people have misread what the UHI effect actually is. They think it about energy USED in the cities warming the Earth causing a global trend and that is totally wrong.
UHI has to do with trends of growth from tiny cooler undeveloped towns to warmer cities over many decades and the trends the temperature stations create out of solar heat absorbed by manmade structures that heats the thermometers locally more than when the structure didn’t even exist years before, not warming the Earth itself.
BEST is right that cities only account for a tiny fraction of the world’s area and the energy they use is insignificant but BEST is also so wrong for it is each individual local thermometer that is being affected giving the illusion of widespread warming when it is just a warmer thermometer caused by UHI growth over the decades.
The splicing you speak of above just magnifies this illusion and you explained how this happens so clear.
Stephen Rasey, one more thing along your line of thought. When Anthony gets some time I would be very curious if the surfacestation metadata shows any significant number of stations being moved from grassy knolls outside of cities into the core of the cities atop fire stations or in proximity to air-con exhausts.
My guess is no. The opposite should have happened as stations are moved to improve the quality of the readings out of cities, BUT, this creates exactly the scenario you lay out above. I just keep seeing the chart from NOAA in my mind of the difference between adjusted temperatures and raw temperatures as it stair-steps upward and upward like a machine. This could very well be caused by the splicing and adjustments being performed on the temperature data.
Bet many people would like to know that.
Stephen Rasey: Have I made the point clear enough that the splices cannot contain the low frequencies?
Your point was clear, but I think you are mistaken for the most part. That is, there may be some records for which the technique masks low frequency variation and introduces spurious low frequency variation. Your example shows that it is possible. It is always possible after the fact to create a single time series, or a few time series, that defeat a particular analysis technique. That’s one of the reasons that so many analysis techniques have been invented. So what you wrote may apply to some of the records. It’s something that the authors might be able to check on.
More likely, in these r records, there is some variability related to natural oscillations, some to trend ( which may include UHI). For example, a small airport near a tiny town that grew from 1950 – 2010, may have had it’s runway first paved and two new buildings added in about 1957, and the thermometer reconditioned. the resultant (presumptive) jump in temperature would pervert the trend and any low frequency signal, but cutting and splicing would restore the trend and the low frequency signal to something more like their true values.
So the question works out to a question about the preponderance of cases as you describe (sawtooths) vs. the preponderance of cases such as I describe (trend line plus sinusoid, with a few random jumps.) When you think of the sources of the low frequency signal (solar, AMO, PDO,) and the otherwise flat or monotonic trend in most temperature records, and reflect that there are 39,000 station records, I think that the cases like I describe predominate. Eventually I’ll ask the authors about this, in a professional meeting of some sort.
More points: The ‘discontinuities” are not noise.
to the degree that they pervert estimating the trend, they are noise.
The discontinuities are themselves the removal of a great deal of UHI from the record.
That’s desirable in this case, like “partialling out” the effect of a covariate.
The suture, the glue, is no guarantee the original signal is preserved.
There are no guarantees, so this method is one more method that is no guarantee.
I think that your sawtooth example is interesting. It is the first I have read of the possibility that a station may have been repeatedly located away from a growing heat source. I think there is no way of estimating UHI or eliminating its effect without some kind of systematic attempt to study the “cooling” and “warming” thermometers. Anthony Watts has tried to lead an effort, but there are too many to be examined in sufficient detail, and the sampling is biased. So a random sample instead of a census might be required.
Jeff Id says:
November 4, 2011 at 3:22 pm
I addressed this in my response to Stephen Rasey. I doubt that such cases predominate, but I hope for a systematic study, full histories, of samples of the thermometers. This is for sure a reason to think that their error bounds are too optimistic.
Like you, Steve McIntyre has impact. I hope that the Berkeley team is attending to his critiques.
My last word on the subject: you and Mr. Rasey might be right.
diogenes: my view from a number of blogs is that Steven Mosher is about 14 years old
That is most inaccurate and unfair. Steven Mosher very knowledgeable about the topics that posts on, and his posts are always worth reading and considering. If he is sometimes mistaken (who isn’t?) he’s never written a post as stupid as that one from you. (n.b. that is a criticism of the post, not of the person who wrote it. For all I know you are a great and knowledgeable person who just goofed.)
Stephen Rasey says: “They are glued together… HOW? ”
You have made a lot of comments about the segments being glue back together or sutured. While I don’t disagree with your general arguments, I don’t think this suturing is what is done in B-est. The little bits remain little bits. and are viewed as zillions of short records that are then statistically analysed. If they were glued back together long term signals would likely be preserved. Study the methods paper again, I see nothing indicating such a reassembly of the shreds.
Your point about longer frequencies seems to be born out by an FFT frequency analysis. Here’s a comparison of Berkeley-est and HadcruT3 (land and sea):
http://tinypic.com/view.php?pic=24qu049&s=5
A significant issue is how this method will handle a volcanic event. A rapid cooling followed by decade long recovery. All this emphasises the need for BEST to study what their method is actually doing .
Their paper simply states the effect of splicing “should be trend neutral”. This seems to be a clear and honest declaration that they have not even looked.
It must be remembered that this is UNREVIEWED and UNPUBLISHED at this stage, so I would expect this issue to be addressed during the review period.
@ur momisugly P.Solar. The word “Suture” is not mine. It comes from BEST. Somehow they are taking trends of a temperature segment (some sort of average first derivatives of a function), throwing away everything else, and then making the pieces fit into a 200 year temperature record at the end.
Glue is a good a word as any. And if it implies a source of contamination, it is a better word than most.
The situation becomes even worse when we consider that the downstep is more mathematically detectable than the upstep due to the general uptrend of the average. You are more likely to wipe out the re-sighted temp stations.
So the case where station has urban pressure, it is moved to an out of the way position and a downstep is produced is more likely to be chopped than a station which experiences the installation of a building next to it and is not moved.
Matt says:
In the end, it is true that atmospheric gasses reflect some light back into space. This is a component of the earth’s “albedo” and it does have a slight cooling affect. However, ice and clouds are much more important to the earth’s albedo than atmospheric gasses.
Henry@Matt
Matt you are arguing that somehow the cooling effect of CO2 is already “counted” in earth’s albedo.
I heard this point made before. That is of course a very stupid argument. Because as CO2 increases, so must its cooling effect (by deflecting sun’s energy) and its warming effect (by deflecting earth’s energy). The question is and was: what exactly is the net effect of an increase in CO2?
I again urge you to carefully read the footnote on the bottom here:
http://www.letterdash.com/HenryP/the-greenhouse-effect-and-the-principle-of-re-radiation-11-Aug-2011
I am particularly worried about the the “absorption” of CO2 in the 4-5 um band because this is where the sun is emitting “hot” radiation which as CO2 increases bounces off the earth.
I am glad you are not teaching anymore, because I don’t know what you want to teach until you understand by simple observation what is actually happening.