Posted by Jeff Id on January 5, 2010
So I’ve learned a great deal playing around with the GHCN data, I think this is a reasonably significant post. Ya know, it’s hard to know anything until you try it yourself and I hope more of the readers here will. Again, there was a problem in my last CRU post, however, the more I look for it, the more avenues there are to explore. The issues have been corrected by avoiding the remaining possibilities in this post.
Of all the details worked out over the last two days, one is a decent gridded average of temperature data. Unfortunately for us skeptics it looks like Figure 1 which is pretty similar to the CRU plot.
Figure 1 – all data gridded
Yes, there is warming according to our temp stations, but I don’t think comrade Phil Climategate Jones would like this curve, because the warming in this curve happens entirely after 1975.
It’s nice to see a good quality CRU similar curve after the previous effort, but that’s how things happen when you do your work in public. The plot above uses all the data with each 5 digit temp station code averaged together individually, as my first post did. Anomaly is calculated over the entire series length.
The concern which was explored in some detail, regarded the hypothesis that the loss of stations in recent years created or biased the trend. It came about since so many stations are lost in recent years as Ken Fritsch pointed out in the recent CRU #3 thread.
Figure 2 – Stations per year GHCN per Ken Fritsch from KNMI database
I’ve run dozens of plots over the last several days, some of which contained an error in them created from data selection or a code problem in my previous post. Using the algorithm which averages together individual station ID numbers, I get very consistent CRUesque patterns. the warming is common to a variety of data sorting processes. This methods avoids the issues of data selection or code problems in the other methods and I’m confident in the accuracy of these results, but you should check them.
Several methods were employed to test the consistencey of result, including sorting for Rural and Urban, and sorting for several different time lengths of station data. All varieties so far produced very similar same results. There are, however, interesting revelations from examination of the slight differences.
Figure 3 – 100% Urban Data
Figure 3 is a plot is the urban data only. Of note is that the warming starts at 1978 with only slight warming beforehand and launches up about 1.2 C with no end in sight. Also, 1982 isn’t much reduced from around 1940 which is different from the global average in Figure 1. So the next thing I did was to plot the rural data.
Figure 4 – Rural station data.
That looks a great deal more like the satellite data. The temp rose and fell again prior to 1978 and rose again since 1978 is maybe 0.5C total. I tend to ignore data prior to 1900 due to the very small number of stations. I don’t think the drop in temps to 1900 levels in the early 70’s is the kind of curve that supports the high CO2 sensitivity claimed by climate science. Does anyone remember the snow storms of the early 70’s? Yeah, yeah just weather, I know.
One of the other avenues explored at great length , yet still isn’t finished, was how station starts and stops affect the trend in recent years. To explore that, one of the several methods I used was to sort data according to number of available data points. Below, I presented the gridded global average for all stations with at least 100 years (1200 points) of available data, since many of the stations in Figure 2 were started in 1950.
Figure 5 – Urban gridded data from stations with at least 1200 points
The urban data only in Figure 5 has an even steeper curve, you would expect this from longer series in this type of analyis. The temp rise since 1978 is about 1.2C. The rural 100 year curve is below.
Figure 6 Rural temperature stations with at least 1200 points
So the Rural stations show about 0.7C of warming since 1978. Visibly less warming than the urban stations by themselves. Also note the slight downtrend in recent years. Since the industrial revolution occurred a hundred years ago, it’s hard to imagine this curve is created by CO2. Still I’m not denying the heat capturing ability of CO2, just that the curves here don’t show a continuous warming but rather a short term recent spike.
So of course we should look at the difference between urban and rural stations.
Figure 7 – Difference between urban and rural data as identified by GHCN
Figure 8 – Difference between urban and rural data from GHCN stations with at least 100 yrs of data (1200 monthly points)
Look at that curve! Despite the crudeness of the categorization of thermometers, there is a clear warming bias for big city data. The curve in Figure 8 ends at 0.6C difference. What’s more, the trend between the two looks statistically significant. If Phil Climategate Jones and Michael Marx Mann can choose which data they want to show and hide the rest, I think it’s only fair to choose to look at trends only from Figure 8 since 1978 (even though it won’t make much difference). After all, one hundred percent of global warming has apparently occurred since that time. Let’s do a simple significance test.
Figure 9 – Statistical significance between rural and urban stations
Woah, it’s not even close, a trend of 0.12 and a no trend null hypothesis limit of +0.04. The difference between urban and rural warming is as great as the entire trend in UAH data over the same timeperiod.
Just how much trend do the ground stations show.
Figure 10 – All urban stations different Y scale from 11
Figure 11 – All rural stations different Y scale from 10
Even Figure 11 is still greater than UAH and RSS satellite data but it’s one heck of a lot less than the urban stations. Of course we would be remiss to not mention that WUWT has taught us what rural stations often look like.
What could go wrong with sophisticated technology like that?
The R code for this post is here.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Phil Jones knew what he was talking about. If he had released the data, people would have found something wrong with it.
That should read “and one not biased by very localized heat sources,”
Atmospheric Nuclear testing was stopped in 1963.
“…Prof. Julia Slingo, of the proof that CO2 causes global warming. Her response comes down to this:
“We are now rapidly approaching 390 parts per million which means it’s been a 40% increase. Most of that increase has happened in the last 50 years. And if we know that carbon dioxide is a greenhouse gas, it’s hard to believe that if you increase it by 40% you’re not going to do something to the temperature of the planet.”
———-
Reply: The operative word for Julia is “if”, as in “if we know”. All that does is assume because there is no more spirited argument currently underway than HOW MUCH is CO2 a greenhouse gas? Does it trump water? Does the modeled forcing factor assigned to it make any sense?
So many questions and so few definitive answers, yet her final analysis is based on “belief” and “something”. That wouldn’t cut it where I work. I also “believe something” is going on, but to make a guess like Julia does wouldn’t be science and it certainly isn’t definitive enough to plunge the world into chaos because some non-scientific politicians and traders glom onto it and use it to their personal benefit.
The entire work that has been done by a couple of generations of climatologists since 1960 has apparently been “corrected” by a single blogger after two days of work. This is good news indeed! Let’s hope Jeff puts his remarkable talent to use in other fields as well – maybe we could have a cure for cancer by the end of this month and a working fusion reactor by the end of February. I would also recommend that all particle physicists send their data to Jeff without delay – who needs the LHC when there is not a single problem that cannot be solved by diligent bloggers?
I think this is land-only temperature and should not be compared to or mixed with land+ocean temperatures…?
“I tend to ignore data prior to 1900 due to the very small number of stations.”
For a Global Average, fine. But don’t ignore them altogether. They have quite a story to tell:
http://i49.tinypic.com/rc93fa.jpg
Juravj V:
Not if you average over 5°x5* grids before combining them to make the global mean.
The real problem you run into is if you have more stations in urban areas than in rural, if you just do an unweighted average over all stations, that 5°x5° grid point, you’ll end up with a net urban-heating bias in that 5°x5° grid.
OT
A Guardian newspaper exclusive:
http://www.guardian.co.uk/business/2010/jan/07/gas-rationing-national-grid-factories
Yes, most of the warming since 1988, which was the first year climate changed in most parts of Norway, has occured on the Nothern hemisphere. The data below show annual mean temperatures since the beginning of instrumental observation in capital of Norway, Oslo. I think everyone sees the steep and sudden warming trend. The data below is not homogenizied and are retrieved from various stations in Oslo area due to change in observation positions in the early years. But it should cover the trend pretty well.
Most of Norway, in particular not-costal areas, have already got their two degree warming forecasted for the globe by the IPCC. We live fine with that!
We now see a weak decline in temperatures for 2009 which gives a set back to 2001. Entering a new year we have so far the coldest weather since 1987; the last cold year below mean.
Talking about Arctic: Spitsbergen (Svalbard, governed by Norway), has experienced even warmer weather and a collapse due to lack of sea ice. Some believe this is caused by deviation in circulation pattern and change og wind direction.
I have know scientific background, but as lay man studied these things for 30 years or so. But I have to say that this religious thinking of CO2 as the devil must be wrong. I still though wonder why NAO + is stronger when it occurs. In ten years we will know?
Decade Mean
1820-29 5,5
1830-39 5,2
1840-49 5,1
1850-59 5,4
1860-69 5,1
1870-79 5,0
1880-89 5,4
1890-99 5,6
1900-09 5,5
1910-19 5,7
1920-29 5,8
1930-39 6,7
1940-49 5,7
1950-59 5,7
1960-69 5,3
1970-79 5,9
1980-89 5,7
1990-99 6,4
2000-09 7,1
There is a companion graph that is more instructive as it shows the corrections step-by-step.
I got hold of the “peer-reviewed” papers behind all those station adjustments on USCHN over the Christmas break and read them. You can find my thoughts on this at this link. However, let me give you a couple of highlights.
1. The time of observation bias provides a large correction, because the day to day temperature variation is quite large at many sites in the U.S. It is a stochastic correction because it depends on the temperature difference from a day behind to the current day. What the form of the correction over time suggests is that lots of COOP station observers were changing their observation schedule from midnight to late in the day from about 1900 to 1930 or so, and then switching to a morning observation time from 1930 to the present. There were some large changes apparently just after WWI and just before 1950. It looks like the correction is based solely on a study of 1958-1964 data, and I wonder if it is appropriate to all time.
2. The homogenization step simply compares lots of “nearby” stations with one another. It is meant mainly to look for step discontinuities, which means that slow drifts like UHI or degradation of station site will not be removed, but rather spread through stations uniformly most likely. Karl’s paper specifically suggests doing this process last, but you will notice that NCDC does this correction before removing UHI and that is out of order according to the boss. I think this presents a problem, but I can get no one else to render an opinion right now. Anthony, of course, has pointed out the smearing effect of this adjustment.
3. The correction for UHI effect is based on a study of paired rural-urban stations. I don’t know that the pairing was particularly effective because paired stations differ by plus or minus 9 degrees, and the correction is a regression based on this noisiness and looks almost insignificant, and is dominated by the largest stations (because the regression is weighted by population).
One interesting thing that strikes me as suspicious about the validity of these corrections is that the correction arrived at through homogenization is just about the mirror image of that for UHI. This does not seem likely to me, but rather indicates that doing corrections out of order has fouled them up. Moreover, the raw station differences in the UHI correction are not used, but rather these are weighted by size of climatic zone. So, the weighting factors are different for the different corrections, and I wonder what potential there is for strange interactions here..
The last peasouper in London occured during the winter of 1962-3. We remember it well. My mother had just returned to England after many years living in the Middle East. The week’s fog, where for several days you could not see the person who was walking alongside you in the street, drove her to go and stay with my sister in the West Country over Christmas. The snow was so deep in Somerset that she was housebound for nearly six weeks, but at least there was no fog there.
We did had thick fogs in at least some London suburbs in the 1970s but without the acrid yellow infusions which used to turn the fogs into peasoupers.
Can I repeat the question posed by lichanos. What is the protocol for defining stations? Has anyone ascertained how many “urban” stations were previously “rural”? It is all very well only selecting stations with 100-year records but do we know that each of these was always urban or have some of them been reclassified during the period? And, if so, when?
Interesting article, though I could do without the ad hom labeling.
Anthony;
Regarding the ‘truly rural’ and properly installed stations you have found through surfacestations.org, could you please supply the list of those stations to JeffId to run through the kind of processing he has done here?
I seem to recall having heard someone (GS?) on one of those ‘climategate debates’ making the (uncontested) claim that the bottomline result of the surfacestations effort was identical to the CRU result, and that the results had been ignored or buried. This last assertion really bothered me.
I realize it is a mostly-NA effort at this point, but the time seems ripe to offer JeffId a shot at plotting the net results of the good rural stations separately, even if they be a smallish group.
I will keep asking this question until someone can give me a reasonable and believable answer: If you have dozens of stations which show little or no warming and many which show cooling since 1895, how is it called global warming. In my simple mind, if some stations show cooling, then the globe is actually cooling and warming is only local. That is the physics of heat transfer as I understand it! For example, if one station in California shows warming (Petaluma) and another station nearby (Santa Rosa) shows cooling, what is local and what is globe? I used the public web interface for USHCN and looked at the Monthly Mean Temperature vs years, at the following link:
http://cdiac.ornl.gov/epubs/ndp/ushcn/ushcn_map_interface.html
Stephen
[REPLY – This much I can say: US (USHCN) stations, equally weighted, raw data, show 0.14C average warming per century. For adjusted temperatures (FILNET+SHAP+MMTS – outliers), it comes to 0.59C warming per century. Areas designated by NCDC as Urban (9% of USHCN stations) warm around 0.5C per century faster than non-urban. ~ Evan]
Carrick (09:45:02) : “…While I agree on the need to be cautious, urban islands are real sources of heat. Eventually, that isn’t a bias in temperature, but a source of actual heating of the local climate.”
Except that UHI-influenced temperature in essentially every case is used as the value for a much larger area than the UHI itself.
I’m a fan of sci-fi author Larry Niven. He co-wrote “Fallen Angels” which was published in 1991. Good story if you care to check it out.
He also wrote in another story that “heat is a by-product of civilisation”. I can’t find the source (I think it was a Ringworld story).
Is there not a grain of truth in that statement?
assuming a linear trend, the increases after 1978 can be modelled by:
temp(urban) = m(urban) * x = (m1(urban) + m2) * x (1)
temp(rural) = m(rural) * x = (m1(rural) + m2) * x (2)
where
x is he time after 1978
temp is the temperature increase after 1978
m1 is the slope due to UHI
m2 is the slope due to other influences such as ocean cycle, land use change GG
m is m1+m2
With an overall slope ration of
m(urban)/m(rural) = 2,
the equations (1,2) can be solved, if we make an assumption for the ratio
m1(urban)/m1(rural).
2 cases:
1. given there is no UHI in rural stations, i.e. m1(rural) = 0.
it follows then, that m1(urban) = m2. UHI and other factor contribute equally to the urban increase. the rural increase in only due to the other factors.
2. given rural station have half of urban UHI increase, i.e. m1(urban)/m1(rural)=2:
satisifying equations (1,2) then requires, that m2 = 0. that means all increase, both urban and rural was only due to UHI
Carrick (09:45:02) :
” Eventually, that isn’t a bias in temperature, but a source of actual heating of the local climate.”
I disagree a bit with this point and the context of it. While there are city sized chunks of ground air which show warming, the sections are very thin in relation to the lower atmosphere. Therefore, it represents a very very small fraction of the atmosphere. IMHO, trends in this data are therefore almost purely erroneous for determination of heat in the atmosphere and comparison to climate models.
Tom P (09:37:27) :
“Also agreed, and why UHI is compensated for by both CRU and GISS in deriving their gridded temperatures.”
You and I could sit in a room and disagree about the color of the walls. It would not be my fault either. You can’t be serious in claiming gridding data compensates for bias in warming? Somehow, I’m sure you are though.
Finally, you again have assigned contrarian belief to me I’m very tired of arguing this point. CO2 does capture heat better than some other gasses so we don’t disagree (I hope). I don’t believe warming is dangerous, nor do I believe it’s as large as these curves show. Urban warming effects also apply to many rural stations. I also don’t believe in the flatly ignorant solutions advocates are so fond of. So with that said, I just plot the data and let it take me where it will, unlike Michael Mann, and Phil Jones, I don’t have a choice in the matter.
Two points for Id/Skywalker…a third source of error would be from error associated with different types of instruments at the location. It would be hard to detect since it is mixed into the UHI effect.
The good news is that UHI effects for most locations now may have reached their max since all the various urbanization effects around the obervation sites are probably not going to get much worse. I have noticed that Phoenix and Tucson have not shown any change in temperature since the mid 90s…
@KevinKilty et al;
I haven’t noticed the topic of ‘Inversion Bubbles’ being discussed, although this may reflect my own myopia.
After spending collegiate years in the LA air basin, I was surprised to recognize the same ‘Smoggy Horizon’ back home in Oklahoma City,
, you know,
‘where the wind comes sweeping down the plain…’
My mom said; Well, it is because most cities are started near rivers, and they are thus built in valleys. When the warm wind is in the right direction, it can blow over the top of the valley and make a local inversion layer.
My realization of the day is; Within that layer, the UHI effect will tend to be trapped, just like the smog.
Hopefully this is valuable insight to more than just me.
TIA
ACakaTLakaRR
DirkH (08:38:40) :
“Veronica (08:29:31) :
[…]
Also surely particulate emissions are growing in developing countries with all the industry and forest clearing that is going on now. Wouldn’t that cancel out the clean air acts in developed countries?”
Not necessarily. A lot of that industrialization happens with very modern technology, for instance german or american companies building car production lines in say Mexico or India. China produces solar cells, they use the most modern equipment from germany (Roth+Rau machines for instance). Often developing countries can skip the bad solutions we had in the past.
Get a grip on reality. Been to china lately ??????????????? ( I have )
smoking chimneys almost wherever you look……..
Remember the Olympics, they had to shut down most industry over a large area prior so that the athletes could breath…..
Wee sheds at the side of the roads ( and beside the irrigation ditches ) in which there are smoldering rubbish fires going as a method of rubbish disposal……
We are exporting our pollution BIG time to china, as they apparently have no environmental controls.
Juraj V. (09:23:41) :
“Since almost all of met stations are on the Northern hemisphere, the result is skewed to warmer trend. Tropics, Antarctic and Southern globe have not definite trends as N extratropics. If every station occupies 10km2 on the whole surface, then plain average would do fine.
Until Arctic (where the CO2 rise in dry cold air should manifest mostly) shows 40ties as warm as present and falling again, there is no much sense in the whole CO2 warming theory.”
Reply: You’re correct that doing a simple average using such a small number of data points will not provide an accurate measure. If my maths are correct then based on 100km^2 grid this means 1,487,700 of the data points are extrapolated?
Land area of earth = 148,940,000km^2
Number of 10km X 10km gridded squares = 1,489,400
Current number of thermometer data points = 1,700
Extrapolated data points = 1,487,700
Even allowing for the fact that it is only the anomaly which is required, a crystal ball would probably give as good a result.
Then we have to think about how many data points are needed to cover the 361,132,000km² of water.
Sticking to the satellite data is a better bet, despite the issues regarding calibration, sensor degredation e.t.c.
“…Prof. Julia Slingo, of the proof that CO2 causes global warming”
Some further gems from the good professors article:
You mentioned models in your last answer, and people have asked whether we can really rely on models to tell us about the future of our climate?
“It’s a very good question, but of course we have to remember they are the only thing we have to tell us about the future. We are trying to look into the future to predict what’s going to happen based on the best science and our best understanding of how the climate system works. The only way to do that is through using these models.
I think what people find difficult to understand is what is this thing that we call a model? Well, it’s a huge computer code and it’s about solving the very fundamental equations of physics which describe the motion of the atmosphere, the motion of the oceans, how clouds form, how the land interacts with the sun’s rays, how it interacts with rainfall and so on and so on.
So what these models are is hundreds and thousands of lines of code which capture and represent our best understanding of how the climate system works. So they are not in a sense tuned to give the right answer, what they are representing is how weather, winds blow, rain forms and so forth, absolutely freely based on the fundamental laws of physics.”
So thats alright then! There is no “tuning” from Met Office Scientists and the models are completely impartial.
BTW Just seen the BBC evening news featuring a Profit Of Doom from the Met Office who again told us that despite the evidence of ones own eyes Global Warming is still a real phenomenon and in fact other parts of the world are experiencing unusually warm weather for the time of year.
So just when you thought it was safe to relax we are in fact still all doomed.
I new it, it’s a trick. The global warmist are taking all our money and building a big space ship to take them to a warmer universe……Cape and Trade is a space ship tax!!!!!