Would You Like Your Temperature Data Homogenized, or Pasteurized?

I’ll issue a gripe here. Annualizing the temperatures. Why? There are four seasons every year. If you want to make the data less choppy, fine, but at least keep it in its natural rhythm. Do the Grand Poobahs have a reason for imposing the calendar’s will on the data?

Wait, is that what you did in the graph with sd Basil? I read fast and assumed the standard deviation.

Basil, I felt a very slight breeze above me as read this. Definitely above my level of understanding. Hopefully we have some other readers who understand your data torture better than I do. I do however relate to your closing remarks about the level of our collective ignorance regarding attribution.
Thanks . . . I think.

You guys are on the right track, but what you have not done is gone into the scientific literature and looked at exactly how the adjustments are made… you will, i guarrantee, find this whole temperature monitoring and reporting bizzarre

There is something wrong with our local reported temps. Is there a reasonable explanation of why the WeatherChannel and Weather Underground are reporting temps well above actual temps in my town? Our GISS station has been shut down, though it was well located and well manned. Today I have noticed that the temps reported at our airport are well above the real temps. Not just a little but over ten degrees difference. We supposedly reached over 42F but we never saw above 26F. I wonder because if our GISS is shut down, where are our temps being reported and who is recording them? Do I sound paranoid? Yes, because the politicians are relying on a warming earth.
[REPLY – No surprise here. Airports typically show a much faster warming over the last 30 years than non-APs, for a variety of reasons: Increased air traffic, urban encroachment, severe HO-83 equipment issues, etc. Even the best-sited systems in rural APs are seriously affected. ~ Evan]

A relevant hit from the late great Benny Hill
She said she’d like to bathe in milk
He said alright sweetheart
And when he finished work one night
He loaded up the cart
He said you wanted pasturised
Coz pasturised is best
She says Ernie I’ll be happy
If it comes up to me chest
And that tickled old Ernie (Ernie)
And he drove the fastest milkcart in the west

“I think we have to accept the depth of our ignorance, for now, and admit that we do not really have a clue… ”
That is a proper attitude toward what is without a doubt one of the most complex natural systems one could choose to study. Had that attitude been maintained by all of the scientists interested in GCC, or at least those instrumental in setting the terms of the debate, it would have maintained the reputation of climatology as a science properly humble in the face of awesome forces and bewilderingly complex interactions.
Now, because of plainly revealed human error–to apply a generous characterization to plainly revealed behavior–this most vital of natural systems seems likely to revert to mean: a scientific backwater, ‘though as ever an immensely interesting one, whilst a reliable and universally accepted record is accumulated, commensurate with the immense periods and geography of global climate and with the importance of the immense human consequences of modifying its dynamics.

Anyone that can make sense of this let me know:
I was going through the “raw” from GISS for State College Pa. when I came across something I can’t see how they cam up with the figure. First background for those that don’t know when you get the data from GISS there is 2 ways to get it, one by a Postscript file the other by looking at their txt file and pasting it over.
From there you see the data strung out with headers for each month followed by the four seasonal means starting with Dec of the previous year and going forward. So you take the three months and add them together then divide by 3 and you got the seasonal mean. From the four means they add them together and divide by 4 and get the annual mean. Now I decided to check to see how they computed the Seasonal mean when they had no data for a month and this is where things went into the twilight zone.
So there I am looking at 1973 and 1974, both of which had no data for the month of November. 1973 had 17.9C for Sept and 12.9C for Oct and a S-O-N of 12.3 . So then I go and look at 1974 and for Sept they had 15.9C and Oct 9.5C and a S-O-N of 9.6C . Now can anyone explain how you come up with a seasonal average lower then all inputs and another only .1C above all inputs?
To my poor tires brain it should be an average of 15.4C for ’73 and 12.7C for ’74.
Then in 1975 they don’t have data for Oct and I used the Sept and Nov data provided and what I calculated matches the S-O-N listed of 11.9C . However when I go to 1984 they don’t have data for Sept and when I calculate the S-O-N it doesn’t match the data file. My Calc: 8.5C what they got is 11.6C.
To me it looks like the USHCN v2 has a bug in it that when the data is missing from the center of the Seasonal mean calculation it gives a funking reading but when it is on either side (ie the S or the N in S-O-N) it works fine.

“So what would appear to be a general cooling trend over the past ~130 years at this location when using the unadjusted HadCRUT3 data, becomes a warming trend when the homogeneity adjustment is supplied.”
This is not correct.
The HadCRUT3 data you are using are not raw data. They are homogenized data.
The GISS homogenization was not applied to HadCRUT3 to arrive at the GISS homogenized series.
If you want to ascertain the effect of the GISS homogenization, you need to compare to raw data, or back it out from the GISS code.

What really bothers me is that Tiger Woods gets crucified for liking sex and women, while the UN and governments of the world get off scot free perpetuating this massive AGW swindle that will cost taxpayers trillions of dollars, even when blatant examples of their fraud are being uncovered daily. So much for journalism. Where is the investigative reporting into this serious matter that will affect millions of lives? Where is the peer-review in the scientific process of climate change? No, instead Tiger Woods gets front page coverage in the News, Sports AND Entertainment sections of all the mainstream media channels. Something is very wrong; a revolution may be required to get rid of these climate change hucksters.

Great work!

This post, Willis Eschenbach’s “Smoking Gun”, and Anthony’s on raw/partially cooked/stewed data got me thinking.
Just as with the old “SETI at Home” project, couldn’t we all engage in some “distributed computing” using raw data, if any still exists, and then having it compiled in a central location?
I’m thinking along the lines of Anthony’s surfacestations.org project.
How about if each of us who was a surfacestations surveyor would take the “raw” data from the stations we surveyed, crank it through formulas like Willis and Basil did in their posts, and then upload the results to supplement our surfacestations.org surveys?
Now, I’m not educated in the right field to figure out how to devise the proper analytical tools (I’m a retired lawyer, not a scientist or statistician), but I bet someone could come up with an Excel script (or something) that each one of us could download that would create the right kind of spreadsheet for us to plug the raw data into. We then could let our computers do the work, draw the graphs, and then upload it for all to see.
It seems that there’s a lot of interest and energy out here in WUWT reader land. Analyzing all the HCN stations would be too big a project for folks like Willis Eschenbach, Basil Copeland, Anthony, et al., to take on all by themselves. But if analysis was handed off to all the readers of WUWT, surfacestation.org surveyors, and the like, there might be quite the artillery battery of smoking guns.
Am I making sense here? Is this in any way feasible?
Regards,
Mike

Basil and others
When examining this sort of data please bear in mind that in many parts of the world we can go furher back in time and see ‘modern’ temperatures in much better context where this is the latest in a series of peaks and troughs
http://climatereason.com/LittleIceAgeThermometers/
Clearly there is a considerable UHI effect in many stations which is not properly accounted for as the locations have become increasingly urbanised over the years. Some of the earliest temp recording stations have seen the area around them grow 100 fold.
Tonyb

Ot but people, especially young ones, seem to need an agenda.. There was socialism (which failed even the ol ruskis admit that now…LOL), then there was the hippie movement… then “globalization”, which was “terrible” but now apparently is a “good thing” LOL. No wonder climate change has become so popular. Fortunately everybody is gonna become real bored because it ain’t gonna change over their lifetimes LOL. They will soon find some other agenda don’t you worry about that…

Thanks for this.
When I first read this story (as well as the Darwin Zero entry), I thought it was a tragedy for science. Surely it is a sad day when the amateurs (blog readers, “non-climate scientists”, “non-academic professionals”) are unpicking the errors of the professionals – at a terrifying rate. Perhaps, in a sense, it is. The professionals long ago gave up the pretence of caring about the truth. Tax-payer funds are now going exclusively to large institutions who fudge data to keep the tax-payer funds rolling in.
But in the end, I think this situation is better for science and for humanity. Science doesn’t care who finds the truth. It could be a worker in a patent office in Bern. It could be a blogger in the tropical town of Darwin. Websites like this one, and a dozen others, have allowed us all to get involved with the science – to go out and find our nearest temperature station, and check the records (as just one example). Websites like this have forced the great institutes to throw open their doors and make their data (or some of it) public. It is us ‘amateurs’ who are now checking their work, finding the mistakes, and correcting them.
If this situation continues beyond the (fortress) walls of climate science and into other areas, one wonders whether we’re on the cusp of another scientific revolution. How many seemingly-intractable problems might be solved by an army of blog-readers who have an unbiased passion for the subject?

Jerome, yes there has been warming, but then all too often graphs and data are started from the end of the Little Ice Age (LIA), so of course there’s been warming! The golden question to the statement, “The world has warmed” is “From when?”. You have to pick a time period. Warmists always pick from the end of the LIA, and hardened sceptics pick from 1998. Neither are good at explaining what, if anything, is going on. We should at least go back 1,000 years. When you go back 11,000 years you get a true picture. http://www.uigi.com/Temperature_swings_11000_yrs.jpg

I would prefer global series made from several hundred hi quality stations with long records, which are positioned on the same rural place. Take Arctic with several dozens of stations: no net warming between 40ties and 2000s:
http://ff.org/centers/csspp/library/co2weekly/2005-04-21/what_files/image014.gif
My favorite station is Irish Armagh Observatory: 2000s are barely above 1940s:
http://www.junkscience.com/MSU_Temps/CETvsArmagh_long.html
CET is very long record, but guess who adjusted its data for UHI – Cheers, Phil.

I think this is interesting. I’ve made the point before that another way of looking at the data is to use a CUSUM chart. This is the cumulative sum of differences from the mean. It is a standard quality control plot in analytical and engineering laboratories and is very sensitive to small differences in trends and steps in data. I think Bill’s approach is probably very similar.

An interesting approach to dealing with step changes. But I think your final section, labelled ‘An Important Caveat’, is misleading. To see if there has been warming recently, you should choose a period (say the last thirty years) and simply look at the average of the seasonal differences over that period. Your approach of fitting a line to the graph of seasonal differences is in fact measuring any acceleration in warming, a very different thing. There again, I may have misunderstood.

MattB (02:11:30) :
Nigel S – I worry that many readers are American and will not know Benny Hill and will also not get the best joke in the song simply because US humour is so different than the English…
Just because we’re rough, untutored frontiersmen doesn’t mean we don’t know the man who single-handedly turned “Yakkety-Sax” from a minor hit on pop radio into an iconic chase scene theme.
And “US humour” isn’t so different — just omit the superfluous “u”…

My thanks to the various dedicated researchers/posters and to the organisers of this site for much personal enlightenment.
Having assumed until the ‘Climategate’ emails were made public that the measurement and collection of environmental data was subject to scientifically established and internationally-agreed standards, I have only recently become interested (fascinated) by the absolutely chaotic and messy, probably unacceptable methods that pertain to ground-level temperature data collection, seemingly on a world-wide basis, which is open to considerable fantastical interpretations, in my opinion.
A few days ago, in search of enlightenment, I posed what I thought were 4 quite reasonable and rational questions about this on a Guardian (UK) newspaper-run blog written by George Monbiot and was immediately accused of trolling by a virulent AGW poster! To say that I was dismayed is an understatement, but I have rapidly come to realise that questioning anything about AGW provokes a huge reaction from the AGW faithful. In a recent UK employment court case the Judge ruled that AGW was held by the individual (who brought the action against his employers for firing him for promoting his AGW beliefs to the detriment of the company’s performance) as a religious set of beliefs.
While I have little actual scientific background and no science quals whatsoever, like a significant section of the population, I am literate, numerate and have a reasonably accurate nose for bullshit. I spent a number of years serving as elected chair of a community committee in New Zealand, set up as part of the permit process under the first NZ Resource Management act to have a critical oversight of the establishment and operation of a soon to be built ‘sanitary landfill’ and to liaise, on behalf of the community, with the landfill operators, the engineering and technical peer review group, various inspectors from local and national bodies , etc. During this decade I garnered considerable environmental information, plus a good insight into the methodolgy of on-site environmental measurement and data collection. I was also apalled by the absolute BS promoted by some of the more hysterical members of the local green movement.
Sorry to be a tad long-winded, but it is refreshing to read information put together honestly and using ‘proper’ scientific methadology.
Thanks again,
Alexander

I very much disagree that “less cooling” means warming.
If the trend is still negative there’s no way to call it warming.
It’s the same as “less than expected job-losses” not meaning a recovery – it’s still getting worse.
cheers, acob

I just checked the data for London Gatwick Airport.
‘Good’ data only available from 1961 to midway through 1998 !!
So how are they getting their global temperature data?
Which stations are they using?
Are they cherry picking again?
Gatwick started life under the Air Ministry in 1934, does CRU expect us to believe that they were not competent enough to make temperature measurements? Why stop at 1998, which just happened to be one of our warmest years.

As each new bit of data pops up and shows that it’s been tampered with, I’m increasingly concerned at the scale of what’s going on.
I thought it was primarily about clique behaviour and group-think acceptance of an argument which had gained popular traction. I really didn’t think that ANYONE would fiddle with the actual temperature records.
Making up models to suit their PowerPoint slides and selective use of start points is one thing – but to actually rewrite history is simply stunning. And so stupid – how did they expect to get away with this?
They do say that if you’re going to tell a lie, tell a big one – but this?!?
It’s like Dr Shipman and his hundreds of victims or 9/11 – no one would have believed it was real.

Admittedly slightly OT, but you’ll love this editorial in the Calgary Herald.
http://www.calgaryherald.com/technology/Real+scientists+should+care+more+about+fraud/2333666/story.html

“But even that is far from certain, for before we can conclude that, we have to rule out natural climate variability on centennial time scales. And we simply cannot do that with the instrumental temperature record, because it isn’t long enough.”
I’d like to see this derivative analysis (“trend of a trend”) for the dozen or two continuous records that do go back to the 1700s or even 1600s (Central England), a century or more longer than the 1881 GISS cut-off date. What equation do I use in Excel to calculate it?
Fahrenheit’s scale of 1724 made thermometers fairly accurate. He set salted ice water at 0° and healthy body temperature (of a horse) at 100°. When the variation in the boiling point of water with altitude was understood, the scale was altered so that water boiled at 212° (half the degrees in a full circle above ice water at 32°). Thus by 1800, or even 1750, thermometers were fairly good. Even if absolute values were off for a given thermometer, even the original body temperature scale should be quite good at measuring changes.
From Wikipedia: “An individual’s body temperature typically changes by about 0.5 °C (0.9 °F) between its highest and lowest points each day.”
My temperature with a drugstore digital thermometer under my tongue came out as 98.1° F instead of the standard value of 98.6° F. Body temperature isn’t good enough for climatology. But if I’m off by 1° F in the high point of my 0-100° F calibration then the length of my degree is only off by 1% so I can still accurately measure temperature changes quite accurately.
For those interested, these two plotting sites allow listings in terms of how old records are:
http://rimfrost.no
http://www.appinsys.com/GlobalWarming/climate.aspx
And this site has collected many plots as images: http://climatereason.com
Most old records are quite linear and show no clear AGW signal even with urban heating not subtracted out. A few do (especially De Bilt and Uppsala) show a recent upswing. And a few (e.g. Paris) show linear trends that alter at some point to a different slope).
I’ll post my own long-record plots, made merely as appealing illustrations rather than analytical figures but which are in fact accurate aside from a couple of sadly truncated records having had 2-5 years of the very latest data tacked on from GISS after adjusting it to match the last available year in the super-long data set (New York is one and Minneapolis one I’d still like to treat thus):
http://i47.tinypic.com/2zgt4ly.jpg
http://i45.tinypic.com/125rs3m.jpg

MattB (02:11:30)
Glad you liked it but things are pretty secure humo(u)rwise over there. Have you ever watched the Simpsons? Possibly the most brilliant thing on TV ever.

There is no way to make heads or tails of the temperature record. and with each passing day, isn’t that the point? They have bent, spindled and mutilated the surface temperature data so they can present any fairy tale they choose. It’s really disgusting that this would then show up as science.
It’s going to take years to try and sort it all out, if it’s even possible to do so.
We need first a world version of SurfaceStations.org to even get started.
And then some agreed upon rules.

The HadCRUT3 data set is not the raw data. Here is how the Met Office describe the data.
http://www.metoffice.gov.uk/climatechange/science/monitoring/subsets.html
“The data that we are providing is the database used to produce the global temperature series. Some of these data are the original underlying observations and some are observations adjusted to account for non climatic influences, for example changes in observations methods”
What changes have been made?

It’s a Climategate Christmas:

Hahaha!

“Would You Like Your Temperature Data Homogenized, or Pasteurized?” ??????
Just before I clicked here I was reading the emails…I searched and did not find that the following had ever been posted here:
“One way would be to note that the temperature amplitude (1000 – 1950)
for each is ~1.5°C. Thus you could conclude that hemispheric/global
climate varied by over a degree Celcius (although with regional
differences)
Another way would be to average the records. The resulting temperature
amplitude would be smaller because extremes would cancel since
variability is large and each region’s extremes occur at different
times.
Thus, if people simply looked at several records they would get the
impression that temperature variations were large, ~1.5°C. Imagine
their surprise when they see that the ensemble averages you publish
have much smaller amplitude. ”
http://climate-gate.org/email.php?eid=219&keyword=their%20surprise%20when%20they%20see%20that%20the%20ensemble%20averages%20you%20publish
……..
hmmm… Imagine their surprise!!!!

I have a question:
Why study the temperature at all? Temperature is a weather phenomenon – to begin with. It flickers and swings too much and thus requires mathematical ‘handling’ to make sense of. The warmists will always set up filters that warm up things because there is no baseline climate measurement paradigm to begin with. Using temperature seems to have its roots in the use of the phrase ‘global warming’.
Why not study only proxies? Arriving at a new compromise proxy that all agree on shouldn’t be contentious and it will have the added advantage of both camps not knowing what way things will trend (for both past and future) when the analysis is performed. Drastic political action should be taken only if the proxy satisfies predetermined conditions (which must still be subject to revision with improving scientific understanding).
Mann chopped off Briffa’s post-1960 part because it trended downwards w.r.t to the temperatures. In reality, it is a basic fallacy to do so, given the fact that tree ring reconstructions are the more long-term secular trend between the two. Questioning the longer tree record (with all its inherent problems) against a much shorter thermometer record (which has its own problems no end) doesn’t sound right.
Throw the temperature blade away and see if the secular proxy still bends only once to form a hockey stick. If it doesn’t – no unprecedented warming.

From another thread thiss seems an extremely important analysis:
http://www.gilestro.tk/2009/lots-of-smoke-hardly-any-gun-do-climatologists-falsify-data/
The adjustments over the whole GHCN series add up to
NO BIAS
A selection of random long series stations GISS and GISS homogenised compared
http://img11.imageshack.us/img11/7440/gissrawtemps.jpg

Mike Fox (23:35:01) :
Mike, take a look at the plots at these two stations in the surfacestations database:
http://gallery.surfacestations.org/main.php?g2_itemId=1557
http://gallery.surfacestations.org/main.php?g2_itemId=27418&g2_imageViewsIndex=1

Doesn’t Pasteurization require applying heat?

Basil,
“I didn’t say HadCRUT3 was “raw.” I referred to it as “unadjusted.””
The HadCrut3 data you used was not raw data. It is adjusted data. Specifically, it is homogenized data. The Met office tells you that on the page that you linked to above. This:
Some of these data are the original underlying observations and some are observations adjusted to account for non climatic influences, for example changes in observations methods.
means homogenized. Homogenized means ‘adjusted to account for non climatic influences’.
“Now what I meant by that is that it should be the same as GISS before GISS applies its “homogeneity” adjustment.”
That is not true. The HadCrut3 is not raw data. It is homogenized data. The Met tells you that on the page that you linked to above. It is not the same as GISS before before GISS applies its homogeneity adjustment.
You are not comparing GISS adjusted data with unadjusted data, as you claim. You are comparing two different adjusted datasets. What you say about that comparison is false. Please correct your error.
“For its purpose, I think what I’ve done is defensible.”
It isnt. You dont understand what you have done, and are describing both the datasets and the comparison you made with them incorrectly.
“If you go to the GISTemp web site, you get the option of downloading its “pseudo-raw” version of the data, i.e. the data before it applies its “homogeneity” adjustment. I could have used that, instead of HadCRUT3, and I believe that the results would have been similar, if not the same.”
[snip] You have the option to actually get the unhomogenized data from GISS, but instead you got homogenized HadCRUT3 data, and pretend that it is the same? Why would you do that? Honestly, why? That makes absolutely no sense. If you could have used unhomogenized GISS data where your intended analysis called for unhomogenized GISS data, why on earth didnt you?
“So it is relevant, and interesting, to explore how they can take the same data, and come up with such different results. Which is all I did.”
No it isnt. What you did was compare two homogenized datasets, and claim that one was homogenized and the other wasnt. And you did this, even though you admit to having access to what you knew to be the unhomogenized data you claim you were using.
Why on earth would you do this?

Basil, more of a question. You are using the entire year’s data. What happens when you look just at the daily temps for the two ends of the climate, Jan and July? That is, what is the trend for all the Jan months for the series and same with the July months?
I have long speculated that what we are seeing is not a true increase in temps, but a narrowing of varation below the maximum temps, which will tend to increase the average daily temps, but no real physical increase is occuring. That is, what we are seeing are shorter warmer winters, with no change in summer temps, which gives an increase in average temps over time.
Also, I’m speculating that spring can come earlier and fall come later, which one can see with the temp changes in those transition months.
This could be the killer of AGW if this is the case because there is no real increase in temps, just less variation below max temps for each month. The alarmism of a catastrophic future dissapears then doesn’t it.
I’m going to test this speculation myself as I’m currently downloading, from Environment Canada’s website, all the daily temps for all 1600 Canadian stations from 1900 to the present. It will take about 10 days to complete that download. Then I’ll import it into Access or SQL Server to do the analysis.

If it has not already been put up here and I missed it, this is interesting:
CRU: Artificial Corrections, Fudge Factor

A programing file called ‘briffa_sept98_e.pro’ from the somewhat overlooked documents section of the CRU ‘FOI2009’ collection: [here there is a very interesting graph]
;
; PLOTS ‘ALL’ REGION MXD timeseries from age banded and from hugershoff
; standardised datasets.
; Reads Harry’s regional timeseries and outputs the 1600-1992 portion
; with missing values set appropriately. Uses mxd, and just the
; “all band” timeseries
;****** APPLIES A VERY ARTIFICIAL CORRECTION FOR DECLINE*********
…

Here is some rough excel code. It works.
NOTE that the excel sheet must be saved as a .XLSM macro enabled worksheet and macros will have to be enabled.
excel7
click developer tab
type a new macro name
e. g. YearPerRowToClmn
Assign to keybord if you like
click [create]
then between:
Sub YearPerRowToClmn()
End Sub
paste this (but not the “———“:
‘——————————
ActiveCell.Offset(1, 0).Range(“A1:A11”).Select
Application.CutCopyMode = False
Selection.EntireRow.Insert , CopyOrigin:=xlFormatFromLeftOrAbove
ActiveCell.Offset(-1, 2).Range(“A1:K1”).Select
Selection.Copy
ActiveCell.Offset(1, -1).Range(“A1”).Select
Selection.PasteSpecial Paste:=xlPasteAll, Operation:=xlNone, SkipBlanks:= _
False, Transpose:=True
ActiveCell.Offset(11, -1).Range(“A1”).Select
‘ must “hide the incline”
‘ beware the ides of march
repeattranspose:
If Len(ActiveCell.Text) < 2 Then GoTo stopp
ActiveCell.Offset(1, 0).Range("A1:x11").Select
Application.CutCopyMode = False
Selection.EntireRow.Insert , CopyOrigin:=xlFormatFromLeftOrAbove
ActiveCell.Offset(-1, 2).Range("A1:K1").Select
Selection.Copy
ActiveCell.Offset(1, -1).Range("A1").Select
Selection.PasteSpecial Paste:=xlPasteAll, Operation:=xlNone, SkipBlanks:= _
False, Transpose:=True
ActiveCell.Offset(11, -1).Range("A1").Select
GoTo repeattranspose
stopp:
'———————–
to use:
get data in text form with 1 year of 12 months + other stuff
Select data [ctrl]+a selects the lot
Copy data [ctrl+c
open blank sheet in the workbook containing the macro
paste the data at cell a1
You now have a single column of data one year per row. If your excel is set up differently it may convert the data to columns automatically, ifnot
select the first to last year:
click first – scroll to last and click the last year whilst holding shift
select [data] tab
click text to colums
click delimited if you KNOW that there is always a certain character (space, comma etc) between monthly data
or click fixed width
click next (select delimiter character if necessary)
check the colums are correctly selected – move, delete or add. If station number is in data this usually has the date attached without space. If so add a separator to separate the date from the station.
If station number is in first column click next and select station number to be text click finish
else click finish
You should now have the data separated into columns.
Click the cell containing the first date (or the first cell to the left of january’s temperature)
Save the workbook as the next step is not undo-able.
Run the macro above (use keyboard shortcut or go to the developer tab and double click the macro name.
The macro should stop on the last line of data (looks for empty cell). However if it does not press [ctrl]+[break] a number of times. select end or debug from the options according to taste.
No guarantee is given with this software!!!!!

Basil,
Turning to the balance of your analysis, you can save yourself considerable effort next time when calculating your ‘mean seasonal difference’ statistic. It is not necessary to create a difference series and tally all of its values to do that. Mathematically, your ‘mean seasonal difference’ statistic simplifies to the following equation:
(Te – Ts)/Y
Where:
Ts= Temperature at the start of the analysis period.
Te= Temperature at the end of the analysis period.
Y = number of years in the analysis period.
It should be much faster for you to calc that statistic next time. This does raise the following questions, however.
You have 130 years of data at the Nashville site, which amounts to 1,560 monthly average temperatures. First, you throw out 92% of that data by choosing only to look at October. Then you throw out 98% of that October data, by using a ‘mean seasonal difference’ statistic that is determined by the values of only two of the October monthly averages – the endpoints of the period of analysis.
Why is using only 0.13 % of the data to calculate a temperature trend superior to calculating a temperature trend from a larger portion of the data? Like, say, all of it?
Given that your ‘mean seasonal difference’ statistic only uses two datapoints (the endpoints of your seasonal series) it should be apparent that the choice of those two points is … pretty important. Just moving one of the endpoints of your analysis period forward or backward by one year could dramatically change the ‘mean seasonal difference’ trend that you calaculate.
In fact, on a dataset with essentially zero trend (such as the homogenized GISS dataset for Nashville that shows much less than 0.1C warming over a century) you could completely flop the trend from warming to cooling and back with only tiny moves of the endpoints. And that warming or cooling trend could pretty easily be an order of magnitude larger than the miniscule warming trend calculated from all of the data.
Is that what you mean when you say this method is ‘robust’?

Basil, I think we need to be more certain about the raw data. You suggest that you do not think using raw data is reasonable and that it is not necessary, if I read you correctly.
Basil 6:28:57 “For all the talk about going back to the “raw” data, I don’t think that is where the problem begins. From my work with US data (I do some consulting work where I have occasion to look at the truly “raw” data occasionally), NOAA does some “quality” control right off the bat in reading from the hand written daily records. I doubt that any systematic “warming bias” is introduced at that point.”
I just can’t agree. NOAA’s “‘quality’ control right off the bat” must be checked by truthful citizens/scientists — perhaps at a sample of stations to verify that the raw data is not already cooked. Why would you “doubt” that any systematic “warming bias” is/has been introduced when this is the most significant scientific scandal of our time plus one that already has cost us billions, if not trillions — perhaps even quadrillions — of dollars during the last 10 years? Do you think these people are going to let this go easily?

“From another thread thiss seems an extremely important analysis:
http://www.gilestro.tk/2009/lots-of-smoke-hardly-any-gun-do-climatologists-falsify-data/
The adjustments over the whole GHCN series add up to
NO BIAS” – bill
An interesting analysis, worrying for our “smoking guns” if true. But is he really analysing “raw” data, as I thought they were only releasing the adjusted stuff? If the data was this available, why have they been working so hard to avoid releasing it? I’m also unclear how you analyse an adjustment in terms of its effect on trend.

I prefer my data shaken, not stirred.

on above post of mine
The code does not translate to wordpress very well
Any red marked line:
single quote, replace with single quote from keyboard – its a comment so you can delete!
question mark replace with double quote from keyboard
double quote replace with double quote from keyboard
If you ever end up with a correctly formatted column of monthly data you will need to remove all error indicators.
Mark the data column
on [data] tab select filter
on first line of column click the down arrow
deselect all (click the ticked select all box)
look through offered numbers for error indicators “-” “-9999.9” etc.
click the boxes associated with the error indicators
press[ok]
only the data in error is now shown
mark it all and press be careful of the first box as this is patially obscured by the arrow [delete]
Turn off filter by clicking filter in the ribbon again
Data is now clean
I data shows temp*10
the adjacent (right) to the first temp type
= [click temperature to left to enter cell]/10
mark all column including this cell to last row containing temperature
from [home] tab click fill then select fill down
This column now contains correctly scaled temperature but the cell contents are actually formulae.
with column marked copy column [ctrl]+c
right click first cell of this column and select paste special
select values only then ok it
The column now contains actual values and can therfore be copied to another sheet with dates in decimal format i.e. year+(month-1)/12. Note that excel does not like true dates before jan 1st 1900

What do you make of this?
http://www.informationisbeautiful.net/2009/the-climate-deniers-vs-the-consensus/

Isn’t the HadCRUT3 data already “adjusted”? If so, it would certainly be interesting to do a similar comparison to data that hasn’t even been touched.

Quite brilliant fun and useful post – build your own climate change model
http://iowahawk.typepad.com/iowahawk/2009/12/fables-of-the-reconstruction.html

I live in a small UK village, population about 250, two days ago I drove my mini cooper early evening from our village, the warning alarm came on showing 3 degrees C, 15 minutes later I entered the small town of Mansfield, population 100,000, the temp reading now was 5 degree C, seems the UHI effect is quite significant.

Graphs at
http://www.akk.me.uk/Climate_Change.htm

Seth (07:42:33)
Does anyone know if there has been a large scale attempt to collect all ships logged weather data?
Yes, NOAAs’ National Oceanographic Data Center has the data. see http://www.nodc.noaa.gov/General/NODC-Archive/bt.html about bathythermograph data, specifically.
and http://www.nodc.noaa.gov/General/getdata.html for a list of what data is available (online anyways)

Thom (09:05:19) :
Also precious is this assertion:
“And we haven’t even gotten to the stage where the oceans warm up.”
10,000ish years is not long enough to warm the oceans? Instead it is going to happen in the next 100? Huh? Are we in the magical 1% of the interglacial where previous natural processes do not apply?
You can fool 1% of the people 100% of the time, 100% of the people 1% of the time, but not 100% of the people 100% of the time.

Let me get this straight: HadCRUT3 is based on real temperature measurements (with likely UHI effects), but has “corrections” and “homogenizations” in it already — probably including extension of measured data into unmeasured areas of the earth, correct?
Does GISTemp use HadCRUT3 and make more corrections to it? Or do GISTemp and HadCRUT share a root data set?
I would sure like a map to all of this. Maybe a little flow chart showing the Global Warming creation process.

If one wishes to test the hypothesis that UHI has affected the trends, then follow the lead of epidemiology and make a dose-response “curve”. For instance, sort the raw data into strata based on population growth (probably local energy usage is important also) and calculate average $\Delta t$ for each stratum. If one wishes to detect whether siting issues are relevent, then a dose response curve against station ranking 1-5, and so forth.
Just using rural stations may be probative in this UHI regard, but Evan has said that rural stations have worse siting problems, and probably have worse maintenance problems, than do urban stations.
Now if one wishes to quantify the UHI effect the dose-response curve is not much use in my opinion, and the UHI project that Anthony has on his “Projects” page is pertinent. It allows one to calculate an apparent partial of T with respect to time (i.e. an apparent global warming value) from a “v dot partial of T with respect to x.”

boballab (23:07:48) :
Anyone that can make sense of this let me know:
Missing data is difficult to deal with when it occurs in a time-series with a strong trend or cycle. So, for instance, in the S-O-N season the N value will always be colder (Northern latitudes here) than the average of S, and O, so you cannot figure a seasonal average from S and O alone, but have to have some algorithm for replacing the missing N. Apparently, from what you describe, a missing O is no big deal as they just average S and N, and this makes some sense, as O is generally intermediate to S and N in temperature. I have no idea how they handle missing terms at the front or back end of these seasons. I have lots of my inquiries to NOAA and related agencies go unanswered, but you might ask them what it is they do.
By the way, missing data are called “censored” values in the world of statistics, and isn’t that an ironic term considering the last month or so?

ScottR (10:31:14) :
…
I would sure like a map to all of this. Maybe a little flow chart showing the Global Warming creation process.

This game has become so complicated that a “program” would be helpful. Unfortunately the flow-chart might end up looking like those diagrams that Mort Saul made of the Kennedy Administration, and only good for comic effect.

Phil A,
You can expect that anyone who calls people “deniers” probably isn’t a disinterested neutral party. I think the gentleman in question overreaches with his summation that this analysis effectively debunks any concerns about adjustments in the surface temperature record.
This may show that, in total, the CRU is relatively even-handed with their adjustments once they get the data. It does not end to end verify anything and, maybe someone can confirm/deny, is the CRU raw really raw?

Let me see if I get this correct. Based on what I think you did with the data. If I were to sum up all the data points for Jan over the 100 or so years then I would see how much the temperature rose in the month of January over the 100 or so years. That might be as instructive as just knowing the deltas from year to year.

TonyB (08:36:05) :
Basil 6 44 43
A bigger problem is the stations that have been dropped from the data

Scott R,
“Let me get this straight: HadCRUT3 is based on real temperature measurements (with likely UHI effects), but has “corrections” and “homogenizations” in it already ”
Yes. The HadCrut3 data that have been released are homogenized. We dont know by what method(s), because Phlim Phlam Phil wont tell anyone. But they are homogenized. They likely do not include any significant correction for UHI, because Phil doesnt believe in it.
“— probably including extension of measured data into unmeasured areas of the earth, correct?”
No. Extending measured data into unmeasured areas occurs during the gridding procees. The homogenization process may extend measured data into measured areas, i.e. adjust some station’s data based on other stations’ data.
“Does GISTemp use HadCRUT3 and make more corrections to it?”
No. Despite what you may have read above, that is not how GISTemp is calculated.
“Or do GISTemp and HadCRUT share a root data set?”
Good question. It now appears that CRU draws heavily from GHCN, as does GISS. We dont know what other elfin magic CRU adds, though. And even if both draw from GHCN, they undoubtedly use a different mix of stations and a different mix of the data from those stations. Until the Motley CRU release their station list and detailed methods we wont know, but it is unlikely that the two start from exactly the same ‘root data set’.
“I would sure like a map to all of this. Maybe a little flow chart showing the Global Warming creation process.”
I agree. A cogent explanation of the two primary ‘global warming’ temperature estimates (GISS and CRU) and how they are derived from GHCN and/or other sources would be useful. It would document the interrelatedness of these ‘independant’ measurments, as well as catalog the holes in the public side of the process that the FOIA requests have been trying to fill.
One of these guys with a website ought to do this. Perhaps someone already has …

seems to me that if it’s “global” warming then i should be able to find a rural site with a long history that is well sited and pull it’s data and see the trend. If it’s truly global then the trend should always show up. Take one from each hemsiphere if you really feel a need to multiple samples.

ScottR (10:31:14) :
“Let me get this straight: HadCRUT3 is based on real temperature measurements (with likely UHI effects), but has “corrections” and “homogenizations” in it already — probably including extension of measured data into unmeasured areas of the earth, correct?
Does GISTemp use HadCRUT3 and make more corrections to it? Or do GISTemp and HadCRUT share a root data set?
I would sure like a map to all of this. Maybe a little flow chart showing the Global Warming creation process.”
No. I know its confusing but here is the situation.
GISS: use GHCN “raw” data. They make adjustments to Urban stations
based on nearby Rural stations. These “adjustments” are all over the map, sometimes warming urban stations, sometimes cooling them. St. Mac has covered this. After adjustment Hansen claims his data shows the same trends as a RURAL ONLY. Problem? Rural isnt Rural.. se surfacestations project.
CRU: CRU ( it would appear) Also use GHCN ( plus others) They make NO ADJUSTMENT. They argue, citing peterson 2003 and Parker’s paper) That the Impact of UHI is SLIGHT. Jones ( see climategate mails in the Jan 07 time period ) clarifies this for Susan solomon prior to her Paris briefing. According to Jones ( 1990) the urban effect is on the order of .05C per century. CRU handle this NOT BY ADJUSTING, but by INCREASING the error bands around the dataline.
Clear? I thought not.

Anthony, we here in Wallowa County are as rural as you can get. And we don’t appreciate you degrading our BBQ’s and silver upturned boats next to the temperature house. Plus, it is DAMNED cold in the winter and a lil’ heat by the Stevenson screen helps us navigate the daily thermometer check. Afterall, 15 minutes in this windchill and we freeze our ears and fingers. However, I will admit that the mug of hot Tom and Jerry helps a bit. I will remind the folks that monitor the two stations we have here to take the mug’o T&J with them instead of firing up the BBQ or using the blowtorch so’s they can see the frosted glass.
Now, for the really important stuff. My Jeep windshield frosted on the INSIDE (not fogged…it fricken froze!) this morning. Anyone know of an ice scraper shaped for the inside of the window?

My own feeling, after looking at some of the actual raw data station records and reading the Climategate documents, is that the real problem CRU/GISS have is that the raw thermometer data they gathered wasn’t of sufficiently good quality for them to be able to produce meaningful anomaly graphs, and they had to ‘fudge’ it as best they could. There were also too many problems with dissonance in the various temperature proxy series for this to be much use either.
Satellite data is much better in terms of continuity and spacial coverage, but even after it had been ‘calibrated’ to the obviously flawed thermometer data, it didn’t show the trend they believed was happening. So perhaps we shouldn’t be surprised at seeing station deletions and adjustments made to the pre-satellite data, needed to provide the +ve global temperatures needed to get continue support for the CAGW myth as our cooling Earth failed to co-operate.
Once you start going down the slippery slope, it’s impossible to stop.

Thom (09:05:19) :
What do you make of this?
http://www.informationisbeautiful.net/2009/the-climate-deniers-vs-the-consensus/
All you have to do is look at the “climate deniers vs the consensus” title to see that the author likely has no idea what s/he is talking about, because both sides would likely be wrong, or at least would most likely have positions that have nothing to do with science. But, then, what are “climate deniers” anyway? I haven’t even heard of that one!
Going to the site, the author admits having no knowledge at all about climate science, praises Gavin Schmit and realclimate – where openness is absent, and doesn’t understand the basic argument of many sceptics – that the ipcc and its elite Climate Scientists are not doing real Science, where you have to first prove or back up your hypotheses by following the Scientific Method, which involves ‘showing your work’ by making it – code, data, ‘materials and methods’ – very accessable to anyone interested and in a timely way.
Otherwise, there are no scientific conclusions to either defend, promote, or criticize in the first place. It gets even worse from there for the “consensus”, pro-AGW position, which has been demonstrated well by Steve McIntyre, Richard Lindzen and many others, completely apart from what the leaked emails reveal.
I’m afraid that the author of the linked blog has no idea of what science is. S/he’s trying, but completely off the mark.

Basil (05:26:33) :
YYYY xxxx xxxx …. xxxx
where YYYY is the year, and the xxxx are monthly figures (except for GISTemp, which has a bunch of averages after the December figure. In either case, the object is to parse the line into “words” and write the 2nd through 13th word out line by line to a new file. (Well, I’d need to probably write out the first YYYY, maybe as a comment, at the start of the new file, since the data do not all start at the same year.) I realize this is a very trivial exercise…to anyone who codes daily.
So the bleg was just to ask for some ideas about how to do this. Lots of bright and capable people read this blog. Paul Clark (WoodForTrees) has to do this to read the GISS data for his web site, so he probably has some C++ code that does it,

Some ideas for doing such things in C++ are here
http://arnholm.org/tmp/basil.htm

Pamela,
Up here in the Great White North ice on the inside of a windshield is a frequent winter hazard. The only recourse that I am familiar with is to use an old credit card to scrape the glass, as it will bend to conform to the shape of the window. Points cards and other useless customer loyalty cards work as well. I use an old Tim Horton’s card. But since I am dealing with a minivan, I often find it hard to reach the furthest recesses, and often fantasize about someone inventing an inside scraper. It’s been frigid here, too; in fact, record-breaking out west.

Having become more than a little disenchanted with the low level of statistical inquiry (and data handling) undertaken by GISS and CRU, I can see no other choice other than for individuals to enter into the fray and provide disinterested, non-partisan analysis and make this analysis available to review by their peers.
To this end, I am setting up a Bayesian inference engine (using MCMC) which will decompose monthly or daily temperature records for a single (or multiple locations) into the following:
1) A within year annual cycle – using a Bayesian form of Fourier decomposition taking, say, the first 12 terms – (i.e. 1 year cycle, 1/2 year cycle, 1/3 year cycle, and so on). [Purpose is to systematically account for the strong seasonal signal].
2) A decadal cyclical component – (i.e. time span of data series, time span of data series/2, and so on). [Purpose is to identify influences such as PDO and other long cyclical behaviour – and to place error bounds on that signal].
3) A random walk component – possibly fitted as an AR(n) process.
4) A linear trend (with offset) and error bounds. This is the long-run ‘climatic’ trend component and is, ultimately, the primary statistical object of interest.
5) Measurement noise. All measurement processes are noisy. Some of the noise may be attributed to human-induced variations; other parts may be attributed to short-term weather variations.
Does anyone have any thoughts as to the utility of this endeavour?

Nick Stokes (5:5:39), and Basil (7:31:56), this needs more analysis.
http://www.gilestro.tk/2009/lots-of-smoke-hardly-any-gun-do-climatologists-
falsify-data/
He is a Neurobiologist and very confident that the adjustment bias is zero. I was about to put in my two bits worth when I scrolled down to a comment that echoed my thoughts, by SG. SG said that if the early adjustment is down and later is up then a warming slope is produced from a neutral
The point is, are the adjustments random on a time basis ?. My reading of comments here (Spain,Darwin) is that they are not. Early years are adjusted down and later years are up.
If you look at this on a swing (around zero) basis, plus one and the minus one cancel out but the time series result is a positive warming trend.

Basil,
“Let me see if I understand. The GISS adjustment is to make Nashville “rural.” And it does this by making it appear that Nashville has warmed more than it actually has?”
Has the GISS homogenization adjustment made Nashville appear warmer? The only way you would know that would be to compare to the non-homogenized data, and you have not done that. See my post above on this topic.
“Shouldn’t the adjustment be doing just the opposite?”
Perhaps it does. You wont know until you compare to the non-homogenized data.
“If the “unadjusted” Nashville trend was already sloping downward,”
Was it? Have you looked at the non-homogenized data yet?

nominal (09:58:14)
Thank You for the links. Would this information be useful for a complete global surface temperature reconstruction (being at the sea surface, not in the water) since land stations cover such a small percentage of the globe? Or has this been attempted. It’s funny how everybody ends up just using surface stations. Also there is an enormous amount of data on paper lying around never reported to NOAA, although this would require more effort than the CRU seems to have.

Basil,
“There are other reasons for doing what I’m doing. I’ll come back to that. But for now, I’m trying to follow you, and not having much luck.”
I believe I have figured out our disconnect.
I had understood that you were only using the October data for your ‘mean seasonal difference’, using it as a representative annual figure (much like we often use the annual water year minimum for certain trend analyses, which coincidently also uses October values). Under that assumption, my previous post was correct.
I gather from your subsequent comments that you are actually using the data from all months of the year. If that is the case, I amend my previous post as follows (changes in italics):
***
Turning to the balance of your analysis, you can save yourself considerable effort next time when calculating your ‘mean seasonal difference’ statistic. It is not necessary to create a difference series and tally all of its values to do that. Mathematically, your ‘mean seasonal difference’ statistic simplifies to the following equation:
(Te – Ts)/(Y-1)
Where:
Ts= Temperature at the start of the analysis period.
Te= Temperature at the end of the analysis period.
Y = number of years in the analysis period.
Simply apply this equation for each month you wish to include in your ‘mean seasonal difference’, and average the results.
***
The balance of my previous comments on your method apply, with the minor correction that your method only uses 24 out of the 1,548 temperature measurements available to you to define your ‘mean seasonal difference/Trend’. You are actually using 1.5% of the data to derive your trend, not 0.13% as I had claimed earlier.
The previous questions remain:
Why do you consider a trend that only reflects 1.5% of the data to be superior to a trend calculated from a larger percentage of the data, such as the standard trend line that uses all of the data?
Given that your ‘mean seasonal difference’ statistic only uses 24 datapoints (the monthly endpoints of your seasonal series) it should be apparent that the choice of those 24 points is … pretty important. Just moving one of the endpoints of your analysis period forward or backward by one year could dramatically change the ‘mean seasonal difference’ trend that you calaculate.
In fact, on a dataset with essentially zero trend (such as the homogenized GISS dataset for Nashville that shows much less than 0.1C warming over a century) you could completely flop the trend from warming to cooling and back with only tiny moves of the endpoints.
To quantify, I have checked these assertions against the Nashville data.
Applying your method to these data, I arrive at a ‘mean seasonal difference/trend’ (MSD) for 1881-2009 of -0.0126. This matches what you report here.
Applying my simplified method to the same data, I arrive at an MSD for 1881-2009 of -0.0126. The simplified method works.
Moving the start of the analysis period up only five years, the MSD flops sign from cooling to warming. MSD for 1885-2009 = 0.007.
Move up the start of the analysis period up one additional year and quit two years sooner, and you get an MSD for 1886-2007 of 0.025. By shifting one endpoint of the analysis by six years and the other 2 years, your method turns a cooling trend into a warming trend double the size.
Is this robust?
Incidently, I calc’d a standard linear regression trend for the GISS homogenized data 1881-2009. I did it a couple of different ways, and end up with a cooling trend each time. What method did you use to calculate trend that shows a warming in the GISS data over that period?

Re: Paul Vaughan (00:29:52) :
Thanks for the note of caution wrt randomness. Correct me if I am wrong, but this caution refers to assumption #3?
Input wrt model assumptions always appreciated.
In any event, it will take me a few days to set up and debug the statistical model: I have to work through the algebra, write the Mathematica code, and debug – all in my spare time. But with Christmas looming, results may not be forthcoming quickly.
I will start with a couple of neighbouring sites – a la Peter (and Dad) – to begin with. If there is any merit in continuing, I will expand to a larger data set.
Insights, if any, will be shared – as will data, assumptions and code.

Seth (22:06:12)
yeah, I’d say it’s not only useful, but a requirement that this data be used. And you’re right, it is only the sea -surface- temperature (SST). i think the CRU used the HADSST2 dataset for the IPCCar4, which is composed of the:
“…International Comprehensive Ocean-Atmosphere Data Set, ICOADS, from 1850 to 1997 and from the NCEP-GTS from 1998 to present.”
HadSST2 is produced by taking in-situ measurements of SST from ships and buoys…”
http://badc.nerc.ac.uk/data/hadsst2/
list of available marine datasets here: http://www.marineclimatology.net/wiki/index.php?title=Datasets
IMO, the more data used, the more accurate the models… of course, there is the homogenization, “quality control” and gridding manipulation issues that effect accuracy…

Late yes. Fun though!
http://iowahawk.typepad.com/iowahawk/2009/12/fables-of-the-reconstruction.html

EM. When I started looking a this years ago the daily data was a real eye opener. Primarily because in California there were other sources of daily data that were pristine: daily data from the Agriculture department. Stations in the middle of crop lands. It

E.M.Smith (01:50:48) :
“Jeff (01:31:25) :
I’m curious if the GISTemp data has both raw and homoginized data ?
Because if that is the case I have seen several stepladder adjustments from raw to adjusted in Pa alone …
adjustments that can’t be justified by station moves or UHI …”
A rather complex question to answer that OUGHT to be simple. But, IMHO, the complexity shows where the “issue” starts…
GIStemp takes in what is called “raw” data. Everyone studiously uses that label. It uses “raw” GHCN and USHCN. But when you go to NOAA / NCDC you find that the “raw” GHCN and USHCN are not raw. There is some ill defined “QA” and “homogenization” applied. …
Thank you, sincerely, for that informative and entertaining post. However, you say “ill-define ‘QA'”. Yesterday someone on the thread about pasteurized data pointed to an NOAA web-page that explains the process steps, and includes references. Whether or not the QA is ill-defined must probably now await a look at those research papers behind the adjustments and render some judgement about them. They are pretty old papers, but NOAA appears to have just gotten around to applying them — am I wrong here?
In my opinion the over-all result of this QA, shown in a graphic on the web page, looks a little suspicious. For example, the net effect of all this QA is to produce an adjustment that is a hockey stick of about 0.4C magnitude, warms nearby decades, and cools the 1930s. Therefore truly raw observations with no trend, will end up after correction with an accelerated warming up to the present time. I have several projects lined-up for my X-mas break, but I may add this to the list. Perhaps a few other people can do similarly and we can compare notes.

Roger Knights (19:31:16) :
Richard Wakefield:
“I have long speculated that what we are seeing is not a true increase in temps, but a narrowing of variation below the maximum temps, which will tend to increase the average daily temps, but no real physical increase is occurring. That is, what we are seeing are shorter warmer winters, with no change in summer temps, which gives an increase in average temps over time.
Also, I’m speculating that spring can come earlier and fall come later, which one can see with the temp changes in those transition months.
This could be the killer of AGW if this is the case because there is no real increase in temps, just less variation below max temps for each month. The alarmism of a catastrophic future disappears then doesn’t it.”
Amazing that mainstream science hasn’t looked into this possibility as a way of testing its hypothesis. (Actually, it’s not amazing.)

Actually mainstream science was saying exactly this in the late 1990s. There was widespread agreement that the range of daily maxima and minima had narrowed, with minima increasing. Tom Karl, i think it was, pointed this out in a paper I read in Science Magazine at the time, and also pointed out that almost all of the warming had occurred in western North America and Siberia.

Kevin Kilty (11:56:40) : , Your point about narrowing of maxima/minima concerns the problems of inland land temperature measurement with Stevenson screens reading an air layer that may be convenient for humans but inappropriate for consistent long term records.
In another thread it was stated that the problem with climate science is that is deals with averages and not absolutes. If the interest is in warming (that certainly seems to be the bias), why not select maxima only, this would stop the dispute about urban/rural sites and UHI. But that might spoil the game.
On another issue why not look at temperature records (maxima) on small islands like Malta and Rapanui ?,my reading of these is that they are remarkably stable and measure for the most part, sea breezes.

Basil,
“As I said earlier, all these changes are well within the range of error because of the large standard deviation. We could argue all day and night about the ‘right’ way to compute a trend, because there is no clear cut answer. ‘It depends.'”
Your articles here have put the issue of choosing trend calcs in the center of the table, both by decrying the use of other trends as being sensitive to endpoint choice, and by offering this alternate method to allegedly correct that deficiency. And yet that method is demonstrably more susceptible to such effects. Unlike other trend calcs, such as standard linear regression, your method only uses the endpoints of the data. You need to be prepared to address the issues you raise.
“The method I’m using has its usefulness in showing how temperature varies because of natural climate variation.”
That remains an unsupported assertion. And unquantified.
“The variation you see in the final figure can be quantified in other ways, as well, such as with wavelet transforms, and spectral analysis.”
Yet more obtuse analyses are not the answer, when at issue is a method that, while much simpler, seems to have successfully hidden a two-point trend.
“At this point, our p***ing match…”
I do not consider this a p!$$ing match. It is worrying that you do. Teamish. Your article contains a couple of large, obvious errors, and a few untested methods and concepts that require examination. You should be far more receptive to constructive criticism and peer review than you are, IMHO. I am trying to help.
“… has taken us away from the main point of the post, and has done nothing to advance the discussion.”
Nonsense. This discussion has been about your sd method, and that method was fully two thirds of your post.
The other third of your post was your mistaken interpretation of GISS vs Hadcrut3. Why have you not addressed the fact that, contrary to the assumptions and claims of your post, HadCrut3 is not the antecdent to GISS, and HadCrut3 is homogenized data?
“However you slice and dice it, the GISS homogeneity adjustment increases the trend compared to HadCRUT.”
The use of the word ‘increases’ here is misleading. It implies that GISS starts with HadCrut3 and operates to change a cooling trend to a waming trend. That is not true.
First, GISS does not start with HadCrut3 data. GISS starts with unadjusted data. Those data are available to you. Use them.
Second, it is not clear that the homogenized GISS data in fact show a warming trend. I have fit linear regressions to the homogenized GISS Nashville data a couple of different ways (monthly and yearly), and it shows a small cooling trend both ways.
Earlier, I asked you how you calculated the trend in homogenized GISS data (your second figure), and your response was to post stats for your transformed SD data. How did you arrive at a warming trend in non-sd transfomed GISS data?
“If you want to add substantively to the discussion,…”
Snarkiness is not necessary. Nor is it supported by your position.
“… maybe you could offer your take on my response to a comment by Steven Mosher.”
OK:
“Let me see if I understand. The GISS adjustment is to make Nashville “rural.” And it does this by making it appear that Nashville has warmed more than it actually has?”
No.
In point of fact, the GISS homogeneity adjustments for Nashville change a slight warming trend in the unhomogenized data into a slight cooling trend. I learned this by downloading the freely available unadjusted GISS temp data and running a comparison. It took all of six or seven minutes.
“Any thoughts?”
Yes.
My thought is that you would have known that the GISS homogeneity adjustments for Nashville induce a cooling trend, had you bothered to download the unhomogenized GISS data and use it for your comparison, instead of using HadCrut3 data and pretending that it was unadjusted data.
You need to correct your article.

Basil,
“Where do you get the idea that my statement “implies that GISS starts with HadCRUT3?””
It is implicit in the logic of the discussion of your article, and explicit in your further comments. Quoting you from above:
“I didn’t say HadCRUT3 was “raw.” I referred to it as “unadjusted.” Now what I meant by that is that it should be the same as GISS before GISS applies its “homogeneity” adjustment.)”
Yet you now say:
“Nowhere did I say, nor do I think I implied, that HadCRUT3 is the same as GISS before applying the homogeneity adjustment.”
Are you channeling Michael Mann? Is Phil Jones feeding you these lines? You are currently acting like a Team player.
Your article has several demonstrable errors, and you now appear to be in full denial mode.
If you will not correct your article, Anthony should pull it before you cause further embarassment to yourself and this site.

When looking at various ways of analyzing data derived from observation and experimentation, it would be helpful to present a synthetic data model (a known signal with specified noise) to see how each analysis technique can tease out information from within the noise.
The observation data you are using is from a turbulent system that has fluctuations over timescales ranging from below your sampling interval to longer than the available data. If you use synthetic data with properties similar to your observational data you can identify the strengths and weaknesses of the competing analysis procedures. Specify a temperature series with short and long timescale fluctuations with an underlying linear trend and see if either of the analysis techniques can pick out the trend accurately. Then apply each technique to the observational data using what you know from the synthetic analysis for interpretation.