Warwick Hughes shows how Jones selections put bias in Australian Temperatures

Jones et al 1986 methodical insertion of warming bias

by Warwick Hughes

Jones et al 1986 looked at 86 Australian stations and rejected 46 (25 Short term – 21 long term). Of the 40 they used 27 were short term and 13 long term. Of the long term there were 5 large cities.

The 27 short term stations were mostly only quoted from 1951 onward – regardless of what data was available. It just so happens that the years just post WWII were not prominently warm in Australia so an “automatic” warming trend was reinforced into the CRU Australian component.

Here are 11 examples where Jones et al systematically truncated pre-1951 data or ignored more rural data around many small town Australian stations. These graphics and text have been extracted from a 1992 vintage Word doc that somehow survived the decades and how many HDD’s.

Port Hedland

The aerodrome records 1951-80 shows a clear warming trend. Marble Bar, 150 kms south east, shows a similar trend over that period but a flat trend over 80 odd years.

Port Hedland

Longreach

For the period 1951-80 this trend is sharply upward, yet if the Longreach Post Office record is spliced to the aerodrome record (post 1940s) the trend becomes markedly flatter. When Longreach is compared to Isisford, a much smaller rural centre 80 kms south, the trend is closer to neutral over about 70 years.

Longreach

Mackay

The A.M.O. record 1951-80 shows a clearly steeper warming trend for Mackay when compared with St Lawrence and Pine Islet Lighthouse.

Mackay

Rockhampton

Compared to Bustard Head Lighthouse and St. Lawrence, Rockhampton shows a warming trend of about 0.5°C over 70 years. Rockhampton data was used only for the period 1951-1970. The two nearby more rural sites show a similar temperature pattern but a negligible temperature change over 70 years.

Rockhampton

Meekathara

The Aerodrome record 1951-80 shows a clear strong warming trend. The small centre of Cue, 120 kms south west however has a flat trend over 90 years.

Meekathara

Charleville

The trend for this station 1951-80 also shows a strong warming trend. Cunnamulla, a smaller centre approximately 170 kms south, shows a much flatter trend over about 80 years.

Charleville

Kalgoorlie

The aerodrome record 1941-80 shows a well defined warming. However, when Post Office records are spliced on, the trend is much closer to zero over 90 years. Looking at Southern Cross, a continuous Post Office record, approximately 200 kms west, the trend is very similar, flat over some 90 years.

Kalgoorlie

www.warwickhughes.com/cru86/tr027/kalgoorlie.gif

Forrest

This station 1951-80, shows a steeper warming trend than the nearby Rawlinna, where records go back to 1926.

Forrest

Ceduna

A warming trend is seen over the 1951-80 period, yet the longer term and more remote Streaky Bay, where records are available back to 1925, shows a flatter trend.

Ceduna

Woomera

Once again, this record shows the 1951-80 warming. Broken Hill, the nearest long term station in a similar climate, shows a cooling trend over a hundred year time span.

Woomera

Mt. Gambier

From 1951-80 this aerodrome station shows a strong warming trend. When the Post Office records are spliced on the trend 1860s to 1990, it is close to neutral.

Mount Gambier

It gets better, in Warwick’s blog comments, Warwick points this out:

The situation for Jones 1986 Sth Hem compilation – is that Sydney and Melbourne aside, there is not one station, long or short term, between Brisbane and Mt. Gambier. This area includes all of NSW and Victoria and contains the greatest concentration of long term recording stations in Australia. Must be one of the great and complete exclusions in the history of science.

He adds in another comment:

I have never been able to discover which stations contribute to their gridded data.

Maybe that will change now.

0 0 votes
Article Rating
80 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
David L. Hagen
November 26, 2009 8:26 am

Jeff C comments under “The Code” at Bishop Hill’s blog:

There is a PDF called “idl_cruts3_2005_vs_2008b.pdf” in the documents folder that has plots of seasonal temps for 154 stations from around the world. Each plot has two traces overlaid:
19012005ann_seasons_regcountrymeans____.climgen (black)
19012008ann_seasons_regcountrymeans____.climgen (magenta)
Not surprisingly, if you zoom in, you can see the black trace ends in 2005 and the magenta trace ends in 2008. The really interesting thing is that in the overwhelming majority of the cases, the magenta trace has a cooler trend than the black trace. The magenta trace often shows a warmer early part of the 20th century and a cooler late part of the century than the black trace. Some of the differences are quite large, over half a degree C.
It seems clear that these two versions were calculated using different algorithms or possibly different input data sets. The author of the Harry read me talks of his inability to replicate earlier data and his revising sections of the code that didn’t seem to make sense. Just a thought, but perhaps this could be a comparison of the earlier version and his results? Regardless, it does make one wonder why the later version shows less warming.

There appear to be some “adjustments” or variations in the algorithms for both Australia and New Zealand – at least in this “unpublished” CRU code!

November 26, 2009 8:27 am

Man, this is like Christmas in November.

Al
November 26, 2009 8:30 am

Add this to E. M. Smith’s analysis of station dropouts. here.
He’s basically pointing out that you can manufacture warming with perfectly flat temperatures if you are creative with your station openings and closings.

November 26, 2009 8:32 am

Looks like the perfect “How to” lesson on cherry-picking.
Keep it coming.

November 26, 2009 8:33 am

I’m getting to the point of getting on top of my house and shouting:
“AVERAGE TEMPERATURE IS A FICTION! IT IS MEANINGLESS! IT’S B.S.”….
It’s energy in, energy out.
Temperture is INTENSIVE. Not an extensive variable. BY DEFINITION you cannot average temperatures from different locations.
YOU CAN ONLY TALK ABOUT “MEAN” Temperatures for a PLACE and then only in a SEASONAL CONTEXT.
The fact that these yo-yo’s have been “averaging” temperatures from divergent locations TELLS YOU THEY ARE NOT SCIENTISTS! Only political hacks.
Aside from the other “weak kneed” aspect that finally dawned on me. The “big guns” don’t do their own programming. They have a “graduate student slave” do that. (Thus the “comments” on the impossibility of doing various data manipulations IN THE CODEs.)
The “big guns are 45 to 65 years old. With probably a mean age (valid) of 54 or 55.
A LOT of folks my age have NOT kept up with the rapid changes in computing technology. It’s a GREAT temptation to LET SOMEONE ELSE (i.e.,
younger, eager, needed a stipend, etc.) to do that work.
Making the “big guns” all the more fraudulent.
Sad, very sad..

David Jones
November 26, 2009 8:33 am

Seems that CRU has a policy of being paricularly selective in their choice of data!!

Back2Bat
November 26, 2009 8:35 am

Since government funding seems to turn scientists into whores, I suggest that it either be eliminated OR provided to opposing viewpoints as well. Truth will eventually prevail even against all odds but should it be severely handicapped? In the legal system, we have open ADVOCATES of both guilt and innocence, no pretense of objectivity is made.
“The first to plead his case seems right,
until another comes and examines him.”
Proverbs 18:17
My thanks to Anthony, the moderators and all patient “examiners.”

Fred from Canuckistan . . .
November 26, 2009 8:35 am

hmmmmmm a new level of contempt from the Club of Ethically Challenged Scientists

Henry chance
November 26, 2009 8:38 am

We see a pattern. There are motives to block the release of data. The CRU “helps” the data along.

Nigel Brereton
November 26, 2009 8:38 am

Surely someone can provide similar analysis for the UK

samuellhall
November 26, 2009 8:40 am

My, my. Things just get better by the day.

November 26, 2009 8:47 am

Oh, excellent. Thank you.
TonyB has done an excellent and beautiful article at Jeff Id’s blog Triplets on the Hudson River which shows rank uncorrected UHI in the GISS records. You might like to put it up here presently.
Also, this reminds me, I think you would do well to post up John Daly’s page What the Stations Say which has a goldmine of global station records; this sheds lots of light on the heavy questions still hanging over GISS. It would also be a fitting tribute.

fafhrd
November 26, 2009 8:51 am

Is there any justification in the paper as to why some sites were used and not others and why available data was truncated? It certainly seems that the paper chooses data points that are misleading and only serve to bolster the desired conclusion, but I would be interested in reasoning, if indeed there is any, behind the choices.
BTW – link did not work for me.

November 26, 2009 8:53 am

I think a number of us have long suspected the alarmist crowd of cooking the books but without any evidence it was always dodgy ground – perhaps that will change now

Stephen Wilde
November 26, 2009 8:53 am

John would have been pleased that his work would be so useful after his death.

November 26, 2009 8:58 am

I am afraid that they don’t know where the stations are and which of them exist themselves – as some of the programmer’s notes in one of the files indicate. The only thing one can safely learn from the CRU hacked documentation is that there is a lot of mess in this enterprise.
By the way, some people still underestimate how much bias one can obtain by simple cherry-picking. Let me offer you the following model.
Imagine that you have N random quantities that are randomly and normally distributed around 0, with the standard deviation (a typical error margin, if you don’t know statistics too well) equal to one. They’re copies of a quantity that should give you zero after averaging.
But imagine that one is allowed to pick a fraction of these quantities. He wants their average to be maximized, so he cherry-picks M largest values, and takes their average. For large enough M,N, only the ratio M/N matters. So let us write M as f times N, where f is between 0 and 1 and will be written as a percentage.
Again, the average of the M entries should be zero if you’re unbiased. But if you take the maximum (positive) bias, i.e. the f.N maximum entries the results will be different. The averages will be
f=10%: 1.76
f=20%: 1.40
f=30%: 1.16
f=40%: 0.97
f=50%: 0.80
f=60%: 0.64
f=70%: 0.50
f=80%: 0.35
f=90%: 0.20
f=100%: 0.00
So if one can throw about 60% of the things away, he can easily get the average which is 1 standard deviation above the right mean value. That’s not “too much” but such cherry-picking in alarmist climate science is being done at several level because the calculation has several steps and the results are progressively substituted to the following step.
So in some counting, the results obtained in this way are K standard deviations above the right value where K is the number of steps. Of course, it’s not hard to elevate proper climate sensitivity, which could be 1.0 plus minus 0.5 degrees Celsius, to 5 degrees Celsius in five biased steps. In proper science, noisy things are pretty sensitive to a fair methodology and ways to avoid bias are pretty much the main criterion of quality in this kind of scientific work.

JMANON
November 26, 2009 9:04 am

Seems to me that a natural target for the warmists will be those stations with aerodromes because of the rate at which activity increases over the years.
It would be nice to see an Oz surface stations.org analysis of these and many other such stations (and if we are to have a repository of global climate records, and it shouldn’t be the University of East Anglia any more, that repository should have a very strictly limited responsibility for collecting, maintaining, collating and verifying the data and nothing esle. They should not be scientists involved in warming or any other research where a conflict of interest could arise).
I was struck that in the US many weather station locations were compromised by a need for 24/7 attendance so were, in water treatment plants and on airfields where temperature anomalies (warmer) could be expected.
It would be easy to suggest that there has been some dereliction amongst those responsible for the siting and maintenace but I suspect they were never expected to be put to such a use, the “detection” of sub decimal temperature changes over decades or even centuries.
Still, I couldn’t help but note how many stations where where aerdoromes were mentioned.

Scouse Pete
November 26, 2009 9:05 am

I think we have yet to see the Icing on the cake concerning Climategate. Google now reporting 6,790,000 Hits, up over 1m in half a day! And when papers such as The Telegraph (hardly a sceptics friend!) show their story from the 20th No 2 for “Today”, 1st place for “The Week” and 1st place for “The Month” of most read – I mean first place for THE MONTH considering it broke on the 20th is staggering in the way the Wild Fire has swept across the globe!

jaypan
November 26, 2009 9:09 am

Increasing temperatures at aerodrome stations? Really? How surprising.
That’s still called “science”?

Jack
November 26, 2009 9:14 am

Tar and feathers.
But I bet that Bob Carter is turning cartwheels, and he should.

November 26, 2009 9:17 am

“Climate Science” is now officially the laughing stock of the scientific community, the new Scientology, the new Phrenology.

November 26, 2009 9:20 am

It gets worse – how about changing the source data points to suit your arguments – like only picking data from the beaches of California for example?
I’m appalled – I really am
http://chiefio.wordpress.com/2009/11/03/ghcn-the-global-analysis/

anon
November 26, 2009 9:24 am

This is slightly off topic, but it seems to me the constant focus on Phil Jones and his shenanigans — though necessary simply to expose the extent — may ultimately be a problem.
The constant calls for his resignation seems to me that the establishment cabal will simply use him as the fall guy, along with one or two other light weights, and then it will be back to usual with the damage control spin being “there were a few rogue scientists and they have been released and so now everything is fine — move along nothing to see here.” There might be a whitewash investigation that will come to the same conclusion.
This is the standard cover-up damage control play in these situations, and has been used very effectively time and time again to appease the public that nothing serious happened and get back to “business as usual” — i.e. nothing really changes: new players/leaders, same game.
I’ve yet to see truth really prevail in any major scandal involving any large entrenched establishment (military, govt, academic, etc.) because too many vested interests are involved, too many people have too much too loose and so whatever the reason ( pay-cheques, reputations, etc.) everyone joins the bandwagon to have a fall guy out of self-interest and preservation.
REPLY: a valid email is required to comment. anon.com is not a valid email, you’ll be blocked unless you provide one. Emails are not automatically published. – Anthony

November 26, 2009 9:32 am

Try this for a damning summary for just the Pacific area
http://chiefio.wordpress.com/2009/11/03/ghcn-the-global-analysis/
“The Pacific Islands and Australia / New Zealand:
http://chiefio.wordpress.com/2009/10/29/ghcn-pacific-basin-lies-statistics-and-australia/
http://chiefio.wordpress.com/2009/10/23/gistemp-aussy-fair-go-and-far-gone/
And one of my favorites where we see how one island can shift the whole region:
http://chiefio.wordpress.com/2009/11/01/new-zealand-polynesian-polarphobia/
The end of it all is that the entire Pacific Basin is substantially flat on temperatures. Hard to have “Global Warming” if the Pacific is not participating. Australia and New Zealand show warming, but only due to thermometer change artifacts. For New Zealand, it is one single cold thermometer: And when that one is deleted from the whole record, not just the last few years, New Zealand has no “Global Warming” either.
Hard to have “Global Warming” when the 1/2 of the planet that is the Pacific Basin is dead flat with only a small “ripple” as the PDO flips state every 30 or so years.”

anon
November 26, 2009 9:34 am

This is slightly off topic, but it seems to me the constant focus on Phil Jones and his shenanigans — though necessary simply to expose the extent — may ultimately be a problem.
The constant calls for his resignation seems to me that the establishment cabal will simply use him as the fall guy, along with one or two other light weights, and then it will be back to usual with the damage control spin being “there were a few rogue scientists and they have been released and so now everything is fine — move along nothing to see here.” There might be a whitewash investigation that will come to the same conclusion.
This is the standard cover-up damage control play in these situations, and has been used very effectively time and time again to appease the public that nothing serious happened and get back to “business as usual” — i.e. nothing really changes: new players/leaders, same game.
I’ve yet to see truth really prevail in any major scandal involving any large entrenched establishment (military, govt, academic, etc.) despite intense heat from the internt because too many vested interests are involved, too many people have too much too loose and so whatever the reason ( pay-cheques, reputations, etc.) everyone joins the bandwagon to have a fall guy out of self-interest and preservation.
Not sure what the solution is.

philincalifornia
November 26, 2009 9:56 am

Scouse Pete (09:05:52) :
Yes, and it’s probably weighted towards the younger generation, some of who may be hearing this for the first time. Continued suicidal tendencies from the (former) mainstream media in ignoring it.

Editor
November 26, 2009 10:00 am

An interesting web site that was posted ages ago on a Joanne Nova thread that I still remember and managed to hunt down says:
Chris Gillham:
April 12th, 2009 at 8:00 pm
If anybody is interested in temperature trends across Western Australia from the 1800s to the present day … http://www.waclimate.net

A lot of the stations show very little warming over their record. Some at that site are also in Warwick Hughes’s post.
Excerpts from the header:
Below is an approximate 100 year comparison of average annual mean minimum and maximum temperatures at 32 different locations across Western Australia, including weather in the capital city of Perth.
The temperature averages have been extracted from the database publicly available at the Australian Bureau of Meteorology website for Western Australia, and from the first ever climate study conducted in the colony of Western Australia before 1900.
This page is designed to provide a simple, convenient tool for people to observe temperature trends since weather records began in Western Australia, particularly in the context of perceived climate change.

joe
November 26, 2009 10:14 am

Frankenstein graph, each segment is made of different data, in some cases make up your own. And viola, you get your creature global warming hockey stick.

Scouse Pete
November 26, 2009 10:40 am

Sorry I can’ t keep up – now 7,220,000 since the past hour! Another half million viewers! :=O 10m by end of today anyone?

Henry chance
November 26, 2009 10:49 am

Pajamas media
In an October 14 email to fellow alarmist Tom Wigley, Trenberth plaintively writes:
“How come you do not agree with a statement that says we are no where close to knowing where energy is going or whether clouds are changing to make the planet brighter. We are not close to balancing the energy budget. The fact that we can not account for what is happening in the climate system makes any consideration of geoengineering quite hopeless as we will never be able to tell if it is successful or not! It is a travesty!

Ken Smith
November 26, 2009 11:06 am

Many thanks for your work over the years, Anthony, et. al. I am deeply grateful for the exchange of information and views this site has made possible!
May I suggest that a needed step among climate non-alarmists in the months and years ahead is to press toward a deeper understanding of the philosophy and history of science. The drama we are witnessing calls for a breadth and depth of analysis of the sort that lay behind Thomas Kuhn’s classic _Structure of Scientific Revolutions_.
We must, for example, develop the means to explain why climate alarmism is both more than a hoax and less than a hoax, and how it has elements of conspiracy but it is not a conspiracy. It is more like what Kuhn called a paradigm, with all the enormous complexity and many-tentacled vastness that Kuhn signified when he used that term.
In my mind it is imperative that we acknowledge the reasons for the appeal of the climate alarmist paradigm. We must admit that it is not a fluke or will ‘o the wisp but a highly evolved system that fulfills (however artificially) a variety of social, psychological, political, and YES, _scientific_ needs.
Intellectual historians often develop deep admiration for systems (often discarded in history’s dustbin) that they do not regard as true in any empirical sense, yet which marvellously captivated the imagination of generations, shaping societies and marking the destiny of countless individuals. Examples are Perry Millers works on the Puritans and Ronald Numbers’ works on creationism.
I fear that in our intense desire to “stop the bastards” (Numbers’ summary of what his academic colleagues falsely thought was the goal of his research), we will resort to oversimplification and we will impoverish our own understanding. Along with whatever polemics we feel called to engage in, we should strive to develop a more sophisticated understanding of this fascinating phenomena we are dealing with. And we need to be reminded that decency and kindness are core values that will ultimately compliment empirical truth.
I am convinced–partly because of what I have learned in the last two years from this website–that in the long run (which could mean a year or a generation), CAGW dogma will not stand. My fondest hope is that its collapse would lead to a more constructive period in scientific thought, and that looking back we could say that the human enterprise has, collectively, been advanced in truly beneficial ways becase of the trauma of this difficult time.
Thanksgiving blessings to you all. You are appreciated.
Ken in North Dakota

November 26, 2009 11:08 am

Anyone knows, which stations are used for CET record and whether they are adjusted for UHI?

Phil Rowlands
November 26, 2009 11:12 am

Hopefully this will not be considered too far off-topic but here’s a copy of a post I made to Paul Hudson’s blog on the BBC earlier today –
“We have seen the difficulties that “Harry” encountered when trying to build a ‘gridded’ dataset from the raw data. We have seen the ‘corrections’ that needed to be made to the raw data before this dataset could be produced and the fact that the raw data itself is riddled with errors and omissions.
We can also see that the calibration of the proxy dataset (tree rings) does not seem to hold for measurements taken in the recent past. One might have expected this to force a review of the methodology and initial calibration of this dataset but, and this is truly unforgiveable, this anomaly seems to have been ‘hidden’.
My question is – does the meteorological community consider these datasets to have authoritative status? Are they something like that platinum bar at NPL from which all other measurements of length are derived? Are they the ‘primary standard’ for global meteorological data? Are there other ‘primary standards’ and do these too have similar issues?
I’m sure it’s obvious where I’m going with this…”
Unsurprisingly this did not generate much of a response and the one chap who did respond (by suggesting that I cite peer reviewed papers (Yawn), and suggesting that I might be “very, very silly”) didn’t find “where I’m going with this” at all obvious.
I’m posting it again here in the hope of a more useful response and I’ll try to offer a clearer description of “where I’m going”.
If we are to believe, as seems increasingly likely, that the dataset derived by CRU has been adjusted, massaged or cherry-picked (opinions do vary!) then are there any serious implications for research subsequently carried out using this dataset?
Given that there are perhaps only three such datasets available in the western world are the algorithms used to generate them truly independant?
Let me be clear that I’m talking about the measurements, the base observations that drive climate research. Everyone starts with the same raw data from the observing stations – does everyone adjust, massage or cherry-pick it in the same way?
I want to get away from the emails (that’s just way too easy) but if the measurements/observations are themselves questionable does that not make a fair amount of the subsequent hard work worthless.

anon
November 26, 2009 11:43 am

To me the most important point about HARRY_READ_ME.txt file is what the file represents.
What we have here is a documented THREE year (2006-2009) effort by a CRU programmer, who had access to all the data, access to all the code, access to all the people who developed the code and the models and still HE could still NOT duplicate CRU’s OWN results. If he can’t it simply means the CRU’s results cannot be reproduced even by themselves and so there is no point anyone else even trying — CRU themselves have proven it’s a waste of time and so they themselves have proven their own results are plain rubbish. That means any “peer reviewed” document CRU produced along with any other papers that cited the CRU papers are based on data the CRU themselves can’t verify.
However, as we’ve seen in the spin from Phil et al in the few statements that have been issued so far, is that the CRU data-set is one of 4 temprature data sets and all 4 are consistent with each other. Given that we now know CRU’s is essentially doctored rubbish, it is self-evident the others must be too as it seems the all the custodians are a very small clique who are directly or indirectly related to the hockey team.
Therefore it seems to me that the data from these other organizations needs to be outed also and unless there are similar FOI requests pending (or stalled — which would indicate something to hide again) at the US institutions, we’re going to find the spin “we’re only one of 4 data-sets and we’re all consistent with each other” win over the public. Because it is a reasonable explanation absent evidence (read leaked data) to the contrary from the other organizations.

November 26, 2009 11:46 am

Luboš Motl (08:58:02) :
So now we know they are doing this sort of thing for sure – how does the scientific community deal with it?
A gutter journalist when found out costs his employer dearly – who holds these people to account and how?

Billyquiz
November 26, 2009 11:50 am

I noticed a few comments wrt the high number of airfields used as weather stations.
I’m not sure what the development has been in the US but in the UK the national weather service was established around 1850 as a service to seamen, accurate weather reports being a requirement for safety at sea.
After the first world war most of the job was officially given to the Air Ministry (now part of the Ministry of Defence) with the Royal Navy retaining a small portion. Due to mission critical data being required for aviation in a timely manner, most weather stations in the UK were relocated within RAF airbases (making access impossible to civilians).
One important point here is that until around 1945-50 airfields used grass landing strips but, with the dawn of the jet age, these were replaced with concrete or tarmac and dimensions were dramatically increased.
Might explain post 1950’s warming trends?

SABR Matt
November 26, 2009 11:55 am

In all of the discussion of the various flaws with the surface temperature averaging, why aren’t more of you focused on the directly observable flaw…
Temperature is quadratically related to kinetic energy. And worse…latent heat energy in moist parcels distort the energy budget of planet Earth even further away from the globally average surface temperature record. We need to *NOT* measure surface temperature, but average surface ENERGY. The vast majority of the warming is occurring in the extreme northern latitudes but the vast VAST majority of the earth’s energy is in the TROPICS…where dewpoints are in the 80s and total energies are an order of magnitude larger than they are at the poles. Notice…the temperatures in the tropics are no warmer now than they’ve ever been. So the total energy budget of the planet cannot be much different.

vigilantfish
November 26, 2009 12:06 pm

Ken Smith:
I recommend that you and anyone else interested in the history of how environmentalism went from being a reasonable cause to becoming a religion to read Alston Chase’s “In A Dark Wood” (1996, 2001). This is an eye-opening account of how the environmental extremists gained the upper hand in the United States both in terms of public morality and politically through litigation. Chase is a former philosophy prof who has turned his attention to environmental matters and his account is very even-handed. Perhaps a little long, but riveting nonetheless, and it will answer your question about how global warming became an “accepted paradigm” by explaining the emergence of the environmental, earth-first agenda.

John Barrett
November 26, 2009 12:17 pm

I’d have thought Woomera would have been a lot
warmer in the 1950s and 60s with all those rocket tests
and the Quatermass Experiment starting there.

November 26, 2009 12:24 pm

Again, I quote Dr. Pierre Latour, in his letter to editor of January 2009 to Hydrocarbon Processing: “I recall HP Editor Les Kane publicly emphasizing the pillar of the HPI (hydrocarbon processing industry) instrumentation industry since the 1970s: if you can’t measure it, you can’t control it. ”
Once again, it is demonstrated that no one knows the global average temperature of the earth at any point in time, nor does anyone know how the average changes over time. Certainly not sufficiently well to attempt to control that average temperature.
see This link

geogrl
November 26, 2009 12:32 pm

Henry chance (08:38:39) :
Maybe Jones and his “CRU” are like ACORN and are, you know, “climate organizers”.

Rich
November 26, 2009 12:41 pm

Here is one columnist who “gets it”.
He is fighting a campaign against the AGW tripe along with a few others here in Canada who are giving it ‘their all’.
http://www.torontosun.com/comment/columnists/lorrie_goldstein/2009/11/26/11929676-sun.html

Rereke Whakaaro
November 26, 2009 12:58 pm

O/T, with apologies
Here is another trend from Australia that is of real concern:
http://joannenova.com.au/2009/05/shock-global-temperatures-driven-by-us-postal-charges/
😉

JonesII
November 26, 2009 1:14 pm

SABR Matt (11:55:25) :”We need to *NOT* measure surface temperature, but average surface ENERGY”.
Right!, and measured before dawn, when the minimum temperature is reached. (“net income” :->)

Bob Aughton
November 26, 2009 2:09 pm

Was the Melbourne weather data one of the 5 large cities included? Normally these roads wear a wall to wall carpet of cars and trucks.
http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=Melbourne+Australia&ie=UTF8&hq=&hnear=Melbourne+VIC,+Australia&ll=-37.807586,144.970082&spn=0.000182,0.000406&t=h&z=21

Nick
November 26, 2009 2:20 pm

How about looking at the paper,its methods and its conclusions before considering speculation about the selection of a small part of its data? Is that reasonable?

TattyMane
November 26, 2009 2:25 pm

I’m from Mt Gambier, and lived there for 30 odd years (some of them decidedly odd). I went back there for a look about a year ago. I can assert first hand that the weather there now is just as lousy as it was in the 50s. No warming at all.

Chris Knight
November 26, 2009 2:30 pm

Juraj V. (11:08:28) :
“Anyone knows, which stations are used for CET record and whether they are adjusted for UHI?”
The Met Office mean HadCET diagnostics page is here:
http://hadobs.metoffice.com/hadcet/cet_info_mean.html
Note the plot of the three supposed reference stations for the CET at the lower right of the page.
The full-size image is available here:
http://hadobs.metoffice.com/hadcet/graphs/stations2mean_2009.gif
Note that all three stations have been flatlined throughout the latter half of the year. I wonder where they actually do get their data from? Could I be forgiven for believing that they make the data up?
I wonder if Anthony could comment?

Dr A Burns
November 26, 2009 2:31 pm

IPCC’s Siberian temperatures show some interesting features in addition to the grids showing more warming than surrounding grids, despite having no weather stations.
http://www.appinsys.com/GlobalWarming/GW_Part2_GlobalTempMeasure.htm
Grid Lat 65-70 Long 110-115 shows strong warming. It contains Zhigansk and Dzardzan stations but neither of these stations show any warming.
http://data.giss.nasa.gov/cgi-bin/gistemp/findstation.py?datatype=gistemp&data_set=1&name=&world_map.x=601&world_map.y=46

SezaGeoff
November 26, 2009 2:38 pm

Bob,
The climate station for Melbourne temperatures is apparently at Laverton Air Force Base http://maps.google.com.au/maps?hl=en&q=Lat%3A%20-37.86%20%20Lon%3A%20144.76&um=1&ie=UTF-8&sa=N&tab=wl somewhere. However, they do quote the Melbourne station in all their records and breathless reports on how hot it is in the middle of a traffic choked intersection.

SezaGeoff
November 26, 2009 2:45 pm

Fie to North American imperialism! That link keeps getting redirected to the US somewhere. If you enter 37.86S 144.76E into Google Maps it takes you to Laverton RAAF base. Note the encroachment of suburbia from the East. The whole place is due to be built in as part of urban development soon, so that should make for interesting UHI affects, I mean Global Warming.

tty
November 26, 2009 3:16 pm

Dr A Burns (14:31:23) :
“Grid Lat 65-70 Long 110-115 shows strong warming. It contains Zhigansk and Dzardzan stations but neither of these stations show any warming.”
The now famous “Harry” encountered a somewhat similar case when temperatures suddenly soared in Guatemala. He finally traced it to an erroneous temperature reading for a mexican station. It was not visible in the mexican data, since Mexico is large and has many weather stations, but it “spilled over” into Guatemala which is a small country with few weather stations, and strongly distorted the result there. Perhaps something similar is at work here?

SABR Matt
November 26, 2009 3:18 pm

What kinds of biases might appear in the NCEP NCAR reanalysis dataset? I could easily download the 1948 through 2008 (daily averaged) surface variables necessary to calculate surface theta-e and see if the net planetary theta-e (area-averaged) has changed as much as the surface temperature.
Theta-e is a better measure of total energy in the system.

Glenn
November 26, 2009 5:12 pm

“Bishop deserts Turnbull as Liberals implode”
“Malcolm Turnbull’s deputy Julie Bishop has urged him to stand down amid open warfare in the Liberal Party this morning.”
http://www.abc.net.au/news/stories/2009/11/27/2755149.htm

Russell
November 26, 2009 5:12 pm

Is there an Australian component of the surface stations project? I know nearly all the locations in the Hughes report, and the grouped locations are VERY close to each other – there is no way to ethically ignore a variance in trends.
I’d love to help survey some stations here!

kse
November 26, 2009 5:57 pm

I’ve been thinking about how to compare the data in the mysterious master.dat.com from CRU to data from NOAA and GISS. There have not been any noticeable success in large scale statistical analysis this far as Harry the Wizard of Coding really f* up the station codes (and locations?).
However, I have been able to do some comparisons per station, like with so called Helskinki/Seutula that has the same WMO code as Helsinki-Vantaa airport. It is somehow perplexing that this mysterious Helsinki/Seutula has data sets dating back to 1828 (CRU and NOAA) or 1881 (GISS) while Finnish Met Office tells that Helsinki-Vantaa went online in 1952.
But any ways – here is a graph of mean temperatures averaged using 10 years sliding window:
http://users.tkk.fi/kse/mtemp-giss-noaa-cru.png
and here is the differenece of the yearly temperature averages between CRU and NOAA:
http://users.tkk.fi/kse/HkiSe_dm.png
Uh? As a mere computer scientist, I cannot say that there is something definitely wrong with CRU data (if that really is the data behind HADCrut) but it seems like the “value” that the people form CRU “add” to data sets seems to be quite odd.
It is quite likely, that CRU people gave a completely new meaning to definition “value added data”, I guess.

Raymond
November 26, 2009 6:27 pm

So, whom are you going to believe? Me or your lying thermometers?

November 26, 2009 6:43 pm

Ric Werme (10:00:50) :
If anybody is interested in temperature trends across Western Australia from the 1800s to the present day … http://www.waclimate.net

Three of the 32 locations I’ve documented in Western Australia with records dating back to or before 1900 are referenced by Warwick …
Marble Bar
1901-1930 (30 years surveyed)
Average mean minimum 19.7 C
Average mean maximum 35.6 C
1971-2000 (30 years)
Average mean minimum 20.4 C
Average mean maximum 35.1 C
12 months from November 2008 – October 2009
Mean annual minimum 20.37 C
Mean annual maximum 35.42 C
Kalgoorlie
1901-1930 at Kalgoorlie Post Office
(30 years)
Average mean minimum 12.0 C
Average mean maximum 25.8 C
1971-2000 (30 years) at Kalgoorlie Airport (4km distance)
Average mean minimum 11.8 C
Average mean maximum 25.2 C
12 months from November 2008 – October 2009
Mean annual minimum 11.98 C
Mean annual maximum 25.43 C
Southern Cross
1895-1920 (25 years)
Average mean minimum 10.3 C
Average mean maximum 25.8 C
1971-2000 (30 years)
Average mean minimum 11.2 C
Average mean maximum 25.1 C
12 months from November 2008 – October 2009
Mean annual minimum 10.11 C
Mean annual maximum 25.35 C
The comparison with averages for the past 12 months is a bit irrelevant because of the brevity of survey, but nevertheless interesting. A warmmonger could argue that the mean has increased by about .2 degrees C in the past decade, despite being cooler than 100 years ago.
Southern Cross (pop 700) and Marble Bar (pop 194) would be struggling for UHI effect but there might arguably be some large town warming in Kalgoorlie (pop 29,000) – although the thermometers were moved 4km out of town to the airport in the mid 1900s. Land clearing is an unlikely influence in Marble Bar and Kalgoorlie, which are in mostly untouched pastoral country, but there could be some influence on Southern Cross, which is more agricultural. All three have lower average max compared to a hundred years ago with two seeing a tiny increase in minima.
In the 12 months to October 2009, all 32 locations dotted across the 2.5 million square kilometres of Western Australia had an averaged maximum .6 degrees C warmer than the early 1900s with averaged minimum .39 degrees C warmer than the early 1900s. These 32 locations include the five major coastal urban cities housing most of the state’s population (including the capital of Perth – 1.65 million out of a total state population of 2.2 million).

November 26, 2009 7:09 pm

Thanks for picking up on this Anthony. Of course these selections, omissions and truncations are only some of the ways Jones et al 1986 inserted warming. They also included 5 of the state capitals.

Rev Dr E Buzz
November 26, 2009 7:14 pm

Carl Sagan pointed out in Cosmos that the time of man is absolutely insignificant compared to the history of the earth, much less the universe…Sagan liked to tell us how insignificant we are.
So we base our ENTIRE understanding of the climate on a 30 year period, with some data removed.
THIS GW STUFF IS SO INCREDIBLY STUPID.
I can surmise that it is deliberately stupid so that it can be thrown out into the mass consciousness. I ask pro GWers “what is their favorite temperature, what do they want?”. I ask them “has the earth ever been too hot for humans?”.
I really dislike Marxists.

Bob Aughton
November 26, 2009 7:51 pm

Geoff (14:38:26) The Bureau’s quoted Max temps for Melbourne and Laverton on 7 Feb 2009 are 46.4 C and 47.5 C respectively. From what I can see the station locations quoted are Melbourne 37.81S 144.97E which google shows to be in the CBD in Lonsdale St. just east of Exhibition St. (about 280M south of where the screens are actually located) and Laverton 37.86S 144.76E which google shows to be a grassed area on the base but no sign of the instruments. The data recording commencement dates quoted are Melbourne 1908 and for Laverton 1941. Anyone have any further insight?

SOYLENT GREEN
November 26, 2009 8:06 pm

On a related note, it appears the ETS may be on the rocks.
http://www.news.com.au/couriermail/story/0,,26406523-952,00.html

November 26, 2009 9:14 pm

Bob Aughton (19:51:27) :
Geoff (14:38:26) The Bureau’s quoted Max temps for Melbourne and Laverton on 7 Feb 2009 are 46.4 C and 47.5 C respectively. From what I can see the station locations quoted are Melbourne 37.81S 144.97E which google shows to be in the CBD in Lonsdale St. just east of Exhibition St. (about 280M south of where the screens are actually located) and Laverton 37.86S 144.76E which google shows to be a grassed area on the base but no sign of the instruments. The data recording commencement dates quoted are Melbourne 1908 and for Laverton 1941. Anyone have any further insight?

http://www.waclimate.net/#laverton

Patrick G
November 26, 2009 9:27 pm

@ Billyquiz (11:50:27)
The transition from grass to asphalt tarmacs and runways is not even the half of it.
What about the fact that 50-60 yrs ago the dominant form of air transport was a piston-engined aircraft with a few dozen seats and a few dozen traffic movements per day. Today we have jet-engined aircraft (what effect jet efflux, anyone?) carrying hundreds of passengers and hundreds of traffic movements per day at these same airfields.
Then there is the expansion of airfield facilities at these airports to take into account. Larger passenger and freight terminals to handle increased air traffic volumes (power/air conditioning requirements, mini UHI, perhaps?) and a commensurate expansion of maintenance and storage hangars.
How about we throw into the mix increased car-parking facilities for all those passenger arrivals/departures.
Has any of this been adjusted for? Has anyone even attempted to model the effects of these changes?

seagull
November 26, 2009 9:39 pm

It is interesting that Phil Jones had good insight into the problem that Warwick Hughes has hightlighted.
Phil Jones, 11 March 2003, 1047474776.txt
“…….even with the instrumental record, the early and late 20th century warmings are only significant locally at between 10 – 20% of grid boxes.” The comment was made by Jones in criticising (ridiculing) the paper by Soon and Baliunas, 2003.
This appears to say that, because of spatial variation, one cannot draw a statistically valid inference of generalised global warming from the CRU grid of temperature anomalies, because the combination of spatial and temporal variance is too great. I think Harry may have said as much, in his own way, trying to replicate CRU graphs.
Lumping together heterogeneous data such as sea surface temperatures and inland continental surface temperatures to create a mean global temperature seems to me not scientifically valid.
Australian temperature reports illustrate the point. Australia has large low rainfall and desert areas where drought is endemic, resulting in large temperature anomalies. This is understandable if there is lack of cloud and rain.
On the other hand, coastal areas, both tropical and temperate, show little change in temperature over 140 years, except for urban sites. Even inland areas show little warming if the long term record can be obtained, as Warwick has shown.
Yet reporting of “warming” is done on a national or regional basis, using 1960-1990 as baseline. The larger low rainfall areas distort the reality of lack of warming on the coast where most people live. This in turn leads to confusion between failure of seasonal rain in a region and “climate change” due to “global warming.”
I suspect that the whole notion of a “global” warming is a statistical artefact arising from a poor concept of spatial temperature distributions and their variability. To put it simplistically, there is no inherent reason to assume that a decade of higher temperatures in Marble Bar is likely to be associated with melting of the Antarctic ice. (Marble Bar is regarded as the hottest place in Australia, I think.)
Phil Jones comment above suggests that Team CRU well understand this but find it inconvenient for their purposes. While the Team control climate science I don’t expect that we can move on.
I like Bob Tisdale’s approach, centred on ocean temperatures, currents and airflows and their wide regional influence.

Bob Aughton
November 26, 2009 10:14 pm

Chris (21:14:48) Thanks, but we were referring to Laverton Victoria (i.e.in SE Aust). I doubt that BOM would try to pass of data from the W.A. desert as being from Melbourne.

November 26, 2009 11:04 pm

Bob Aughton (22:14:13) :
Chris (21:14:48) Thanks, but we were referring to Laverton Victoria (i.e.in SE Aust). I doubt that BOM would try to pass of data from the W.A. desert as being from Melbourne.

Well, you never can tell! I didn’t know you guys had a Laverton over on your side of the country.
Whatever, it’s still interesting to look at the 100 year history of the WA desert variety of Laverton:
1901-1930 at Laverton
(30 years surveyed)
Average mean minimum 13.2
Average mean maximum 27.3
1991-2008 at Laverton Aero
(2.5km distance, 17 years)
Average mean minimum 13.8
Average mean maximum 26.7
12 months from November 2008 – October 2009
Mean annual minimum 13.87
Mean annual maximum 27.01
I figured out a long time ago that global warming isn’t happening in the outback of Western Australia.

Bulldust
November 26, 2009 11:30 pm

You can get a fari whack of info on the Australina BOM web site:
http://www.bom.gov.au/
For Melbourne you get the following measurement stations:
http://www.bom.gov.au/products/IDV60900.shtml
Of which Laverton is one:
http://www.bom.gov.au/products/IDV60901/IDV60901.94865.shtml
A lot of the data is freely available for those willing to drill down to the appropriate stations. For exampl, lonbg term data for Meekatharra:
http://www.bom.gov.au/climate/averages/tables/cw_007045.shtml
Someone with the time and inclination could ferret out some interesting findings I am sure.

Bulldust
November 26, 2009 11:31 pm

Bob Aughton (22:14:13) :
That’s Laverton in Melbourne, not Laverton in WA. See above links.

Mooloo
November 27, 2009 2:51 am

I really dislike Marxists.
So, I suspect, does everyone here.
Accusing someone you don’t like of “Marxism” on the basis you don’t like them makes you look stupid. Not them. It’s Godwin’s Law (http://en.wikipedia.org/wiki/Godwin's_law) except Stalin not Hitler, since you want them to be left wing.

Ripper
November 27, 2009 3:11 am

“I figured out a long time ago that global warming isn’t happening in the outback of Western Australia.”
I thought the same , it may pay to check your figures against the new BOM climate site.
http://www.bom.gov.au/cgi-bin/climate/hqsites/site_networks.cgi?variable=maxT&period=annual&state=wa
Marble bars record of 161 consecutive days over 100 defrees F to 20 April 1924 is under threat from global warming.
This has recently caused the mean Maximum temperature to drop from 35.7 to 34.2 in 1923 and from 36.9 to 35.5 in 1924 .
My server has crashed and I cannot get into my files until the new parts arrive.

lookatthecode
November 27, 2009 9:58 am

Unless you guys can get the horror of this story into the mainstream media in laymans turns, agw will go on.
All the public will see is 2 sets of scientists argueing over stuff they don’t understand, and they will choose the save the polar bear side.
in the uk 99.9% of the population know nothing about this story, the contents of the email, data code and the implications.
They might have heard of a data hack, and maybe a few unimportant scientists being rude to each in private email.
The bbc will not touch it, the emails even show the scientists trying to surpress it with tame bbc sdtaff. 5 WEEKS ago.
No newspapers will touch it, no politicains will touch it. even now the royal soceity is going to invetigate it, with theie cheif global warming evangelicalist doing the work.
How about the following, all verifiable for joe public
Climate Gate – Evangelical eco christian developed some CRU code…?
All information below are in the public domain, and links are verifiable, except for the bio, all the following was simple put together via google searches.
Please do your own research, read it for yourselves, think about it, few dozen phd students, code is going to svae the world, and cost several hundred billion dollars and decide whether newsworthy or not.
To quote one of the programmers:
“Although I have yet to see any evidence that climate change is a sign of Christ’s imminent return, human pollution is clearly another of the birth pangs of creation, as it eagerly awaits being delivered from the bondage of corruption (Romans. 19-22).”
Sounds like a religion to me.
Someone who WANTS to believe that humans are bad, damaging to the environment.
so what you may ask, just another believer…
The author:
Tim Mitchell works at the Climactic Research Unit, UEA, Norwich, and is a member of South Park Evangelical Church.
Yes, this is the guy harry is referring to in that Harry_Read_Me.txt file,
from this article:
http://www.e-n.org.uk/1129-Climate-change-and-the-christian.htm
(for all his stuff there – search tim mitchell on the http://www.e_n.org.uk website)
Not Dr Tim Mitchell yet, he is just a research student developing the computer models for the ‘researchers’ to use in climate change Reasearch unit.
Yes, this is the guy harry (presumably Ian Harris, cru research staff) is referring to in that Harry_Read_Me.txt file, ‘what did tim do’ while trying to make sense of all the code, recreate data, etc,etc,etc.
“On we go.. firstly, examined the spc database.. seems to be in % x10.
Looked at published data.. cloud is in % x10, too.
First problem: there is no program to convert sun percentage to
cloud percentage. I can do sun percentage to cloud oktas or sun hours
to cloud percentage! So what the hell did Tim do?!! As I keep asking.”
Whilst many people of faith are excellent dedicated professional scientists.
I have a few doubts that an evangelical eco christian (my label), that obviously is passionate and committed to the above, may not be perhaps open as they may think they are, perhaps they should be, to both sides of the debate.
http://www.e-n.org.uk/2625-Day-after-tomorrow.htm
“The librarian chooses to rescue an old Bible, not because he believes in God, but because its printing was ‘the dawn of the age of reason’. In this film we see how far we have fallen. Lost, we retreat into a virtual world where disaster becomes entertainment and the unreal seems more real than reality itself. ‘For whom tolls the bell? It tolls for thee.’
Dr. Tim Mitchell,
climate scientist”
Dr Tim now, obviously left CRU around 2004, as published research papers dry up at
this time. Apparentally on a new more spiritual direction.
http://www.e-n.org.uk/searchpage.php?term=mitchell
where is he now?
2004
Dr. Tim Mitchell,
formerly a scientist, now a student at LTS
2006
http://www.e-n.org.uk/3639-Rhythm-of-the-rain.htm
Dr. Tim Mitchell,
Highbury Baptist Church;
formerly of the Tyndall Centre for Climate Change Research
Tim Mitchell is apparently now a student, at LTS:
London Theological Seminary
Evangelical Protestant college for the training of preachers and pastors. Provides degrees up to Masters level. includes course details and resources.
http://www.ltslondon.org
googled following sentence for his bio, link now down/unavailable on cru website:
In 1997 I moved to Norwich to carry out the research for a PhD at the Climatic Research Unit (CRU) of the University of East Anglia. …
Tim Mitchell bio: ( a little bit changed to CAPS by me)
In 1997 I moved to Norwich to carry out the research for a PhD at the Climatic Research Unit (CRU) of the University of East Anglia. My subject was the development of climate scenarios for SUBSEQUENT USE BY RESEARCHERS investigating the impacts of climate change. I was supervised by Mike Hulme and by John Mitchell (Hadley Centre of the UK Meteorological Office). The PhD was awarded in April 2001.
http://www.cru.uea.ac.uk/~timm/personal/index.html – Cached
So basically 12 years ago , a phd student working on thesis, put together some code, which would appear to have evolved into the mess ‘harry’ is trying to sort out make sense of,
Did anyone think, it was teams of professional programmers developing robust code?
testing models, validating data sets, archiving, backing up,etc,etc,etc
After all, haven’t we recently experience a completely financial meltdown, with more highly paid, more intelligent people, with infinitly more resources
phd students, probably self taught in fortran (ancient), followed by subsequent students, with the professors probably knowing absolutely nothing about programming, telling the students they SUPERVISE, what to do and the results they are expecting according to their theories (no pressure there then, directly or indirectly to get your phd)
Maybe we should get a few profesional to seriously lok at and audit the cdode/dataset, before billions are spent, world economies are changed.
Back to the code, and the harry_read_me.txt file. The code and datasets, will destroy this department, and AGW, if not their is truly no hope for real science

E.M.Smith
Editor
November 27, 2009 10:42 am

Phil Rowlands (11:12:06) :
My question is – does the meteorological community consider these datasets to have authoritative status? Are they something like that platinum bar at NPL from which all other measurements of length are derived? Are they the ‘primary standard’ for global meteorological data? Are there other ‘primary standards’ and do these too have similar issues?

I’ve asked a similar question about the calibration of the satellite data series (no, not the sensor calibration, the matching of the satellite data series for microwave reflection to surface temps to get some idea how much mW of reflection is what in degrees) and gotten no answer (other than ‘satellite calibrates sensor on internal standard’ which is nice to know, but does not answer how sensor calibrated mW of microwaves gets mapped to temperatures on the ground…).

Given that there are perhaps only three such datasets available in the western world are the algorithms used to generate them truly independant?

CRU has stated there “lost data” is substantially GHCN. GIStemp uses almost entirely GHCN. GHCN has been “cooked” (see links above to my description at chiefio.wordpress.com ) and so these two series “matching” just tells me they confirm the common selection bias in GHCN. That leaves NCDC. I’m pretty sure they are GHCN as well, but have not done the detail work yet.
The two satellite series are too short for decent comparision, but given that GIStemp leaves the present mostly alone and does it’s major data manipulations on the past, comparison to Satellites is of limited use…

Let me be clear that I’m talking about the measurements, the base observations that drive climate research. Everyone starts with the same raw data from the observing stations – does everyone adjust, massage or cherry-pick it in the same way?

Well, they DON’T start with the “same raw data”. They start with the data AFTER it has been adjusted and “preened” by each country BOM, then adjusted and preened some more by the agencies that creates the GHCN. And that NOAA driven selective process has resulted in what I’ve called “The Great Dying of Thermometers” with about 90% of them deleted. The country BOMs still have most of them, but the get shot at the front gate of GHCN.
After that, CRU, GIStemp, and I suspect NCDC don’t really have any choice in the matter… they are fed the selected highly warm biased thermometer survivors and the rest “passes through” and to the output…

I want to get away from the emails (that’s just way too easy) but if the measurements/observations are themselves questionable does that not make a fair amount of the subsequent hard work worthless.

BINGO!
anon (11:43:48) :
Given that we now know CRU’s is essentially doctored rubbish, it is self-evident the others must be too as it seems the all the custodians are a very small clique who are directly or indirectly related to the hockey team.
Yes. Emphatically yes.

Therefore it seems to me that the data from these other organizations needs to be outed

Well, I’ve been whacking away on the source code and operations of GIStemp (and found “issues”…)
http://chiefio.wordpress.com/gistemp/
NCDC seems to be Number Three (and I’ve not looked at it yet, but given the various affiliations it is most likely just a trivial GHCN variant.)
No idea who they think is #4.
we’re going to find the spin “we’re only one of 4 data-sets and we’re all consistent with each other” win over the public. Because it is a reasonable explanation absent evidence (read leaked data) to the contrary from the other organizations.
I’ve been getting this per GIStemp for about a year now. Mostly they would point at the (hidden) Hadley CRUt and say see! Now that CRUt is “outed” it is amusing to see them point at GIStemp and say “See! We Match!”.
IMHO from this date forward all claims of “See, we match!” need to be taken to mean “See, we collaborated! See, we both HAVE ISSUES!” and even “See! We BOTH need Investigation!”.. Let them hang their accomplices in the public square. Makes my investigation easier…

E.M.Smith
Editor
November 27, 2009 10:46 am

Phil Rowlands (11:12:06) :
My question is – does the meteorological community consider these datasets to have authoritative status? Are they something like that platinum bar at NPL from which all other measurements of length are derived? Are they the ‘primary standard’ for global meteorological data? Are there other ‘primary standards’ and do these too have similar issues?

I’ve asked a similar question about the calibration of the satellite data series (no, not the sensor calibration, the matching of the satellite data series for microwave reflection to surface temps to get some idea how much mW of reflection is what in degrees) and gotten no answer (other than ‘satellite calibrates sensor on internal standard’ which is nice to know, but does not answer how sensor calibrated mW of microwaves gets mapped to temperatures on the ground…).

Given that there are perhaps only three such datasets available in the western world are the algorithms used to generate them truly independant?

CRU has stated there “lost data” is substantially GHCN. GIStemp uses almost entirely GHCN. GHCN has been “cooked” (see links above to my description at chiefio.wordpress.com ) and so these two series “matching” just tells me they confirm the common selection bias in GHCN. That leaves NCDC. I’m pretty sure they are GHCN as well, but have not done the detail work yet.
The two satellite series are too short for decent comparision, but given that GIStemp leaves the present mostly alone and does it’s major data manipulations on the past, comparison to Satellites is of limited use…

Let me be clear that I’m talking about the measurements, the base observations that drive climate research. Everyone starts with the same raw data from the observing stations – does everyone adjust, massage or cherry-pick it in the same way?

Well, they DON’T start with the “same raw data”. They start with the data AFTER it has been adjusted and “preened” by each country BOM, then adjusted and preened some more by the agencies that creates the GHCN. And that NOAA driven selective process has resulted in what I’ve called “The Great Dying of Thermometers” with about 90% of them deleted. The country BOMs still have most of them, but the get shot at the front gate of GHCN.
After that, CRU, GIStemp, and I suspect NCDC don’t really have any choice in the matter… they are fed the selected highly warm biased thermometer survivors and the rest “passes through” and to the output…

I want to get away from the emails (that’s just way too easy) but if the measurements/observations are themselves questionable does that not make a fair amount of the subsequent hard work worthless.

BINGO!
anon (11:43:48) :
Given that we now know CRU’s is essentially doctored rubbish, it is self-evident the others must be too as it seems the all the custodians are a very small clique who are directly or indirectly related to the hockey team.
Yes. Emphatically yes.

Therefore it seems to me that the data from these other organizations needs to be outed

Well, I’ve been whacking away on the source code and operations of GIStemp (and found “issues”…)
http://chiefio.wordpress.com/gistemp/
NCDC seems to be Number Three (and I’ve not looked at it yet, but given the various affiliations it is most likely just a trivial GHCN variant.)
No idea who they think is #4.
we’re going to find the spin “we’re only one of 4 data-sets and we’re all consistent with each other” win over the public. Because it is a reasonable explanation absent evidence (read leaked data) to the contrary from the other organizations.
I’ve been getting this per GIStemp for about a year now. Mostly they would point at the (hidden) Hadley CRUt and say see! Now that CRUt is “outed” it is amusing to see them point at GIStemp and say “See! We Match!”.
IMHO from this date forward all claims of “See, we match!” need to be taken to mean “See, we collaborated! See, we both HAVE ISSUES!” and even “See! We BOTH need Investigation!”.. Let them hang their accomplices in the public square. Makes my investigation easier…

E.M.Smith
Editor
November 27, 2009 11:23 am

Nick (14:20:23) : How about looking at the paper,its methods and its conclusions before considering speculation about the selection of a small part of its data? Is that reasonable?
Pay no attention to the data, just look at the conclusions first. Is that reasonable?

E.M.Smith
Editor
November 27, 2009 11:45 am

Patrick G (21:27:30) : Has any of this been adjusted for? Has anyone even attempted to model the effects of these changes?
It’s worse than that! The airports are used as “rural” to adjust other stations!
IMHO, that is why so many UHI adjustments “go the wrong way” in GIStemp.
http://chiefio.wordpress.com/2009/08/23/gistemp-fixes-uhi-using-airports-as-rural/
http://chiefio.wordpress.com/2009/09/04/most-used-rural-airport-for-uhi-adj/

E.M.Smith
Editor
November 27, 2009 12:10 pm

lookatthecode (09:58:36) : Maybe we should get a few profesional to seriously look at and audit the cdode/dataset, before billions are spent, world economies are changed.
Back to the code, and the harry_read_me.txt file. The code and datasets, will destroy this department, and AGW, if not their is truly no hope for real science

Some of us are kicking around bits of the code here:
http://chiefio.wordpress.com/2009/11/25/crut-fromexcel-f90-program-listing/
Though the discussion started on this page:
http://chiefio.wordpress.com/2009/11/21/hadley-hack-and-cru-crud/
Which has in comments a mr “Steve” claiming to be a professional in “scientific programming” actually trying to defend the indefensible in a bit of the HADcrut code. I show how it could easily be made much more robust, and he wants to claim it’s (variously) a waste of time, not needed, fixed just fine by picking one roach off the top of the stew pot and not looking IN the stew, etc. …
(It would be amusing except I think the guy is A) real and B) serious. If this is the quality of attention to QA in “scientific programming” I’d rather get the guy who does billing systems for WalMart to write the code… FWIW, “Steve” looks to originate from near NCDC based on a casual look at the IP number. Haven’t cared enough to chase it down more than that, though, and could be ‘way off’ since it’s just a “from memory” of what “host” returned). The “ping pong” between him and me is somewhat amusing, though I’m getting a bit tired of giving 100 line exact responses to 10 line vague ‘sillyness’… It’s harder to shovel the “poo” out than for it to be deposited 😉

lookatthecode
November 27, 2009 1:59 pm

None of you guys get it do?
to quote: (and I’m really pleased for you)
‘Some of us are kicking around bits of the code here:
http://chiefio.wordpress.com/2009/11/25/crut-fromexcel-f90-program-listing/
Though the discussion started on this page:
http://chiefio.wordpress.com/2009/11/21/hadley-hack-and-cru-crud/
Which has in comments a mr “Steve” claiming to be a professional in “scientific programming” actually trying to defend the indefensible in a bit of the HADcrut code’
————————————————–
You don’t get it, AGW have the MEDIA, POLITICIANS as true believers of the save the planet mantra, we here are all DENIERS, to be discussed and dismissed in the same vein as ‘holocaust deniers’
mr ‘steve’ is a believer, he is trying to confuse, spin, suck you into trivial (as far as the public see) techy, nerdy geeky debate.
The AGW crowd, control the media, global warming is a religion to them, mere fact, the truth and real science doesn’t stop, then (whilst is needed in court, IF it ever gets that far)
This is a watergate moment, without anyone in the real ie mainstream media running with it… there are no investigative journalists, explaining what is being done to the masses.
in the UK, ‘mann’ has been rolled out on the bbc, to explain, ‘we’ve just worked out why it is cooling a bit at the moment, it’s the oceans’
‘it’s a very complex field, we are learning all the time’
so everthing is ok again’ no mention of the scandal, data emails code…
No one is saying (THAT MATTERS, ie famous) –
Hang about, wasn’t everyone saying last week, the science is settled, 2500 ippcc scientists – SAY SO . now you are saying you just learnt something new, does that mean your previous models are bull, and your computer models are wrong?
NOONE IS ASKING that question in the UK, I’m sure no one in Europe has even heard of the hack.
IT is SETTLED, MAN MADE GLOBAL WARMING IS REAL, you ARE A DENIAR, DENIAR, DENIAR, DENIAR you eat babies, etc,etc
So stop getting sweaty over petty techy arguments in forums, blogs no one (famous, important) cares about has ever heard about. talk to real people. people of influence.
Talk to the union cheif who’s thousand of workers who are going due to CO2 legislation.
Talk to the airline Bosses, unions, about why the con, is going to tax ruin them.
Talk to the auto unions, about CO2 is a total scam , by huge commercial vested interest, that allow aL gore to fly around the world in luxury, becoming a billionaire, why happily buyin carbon offset credits, with his vast new found wealth.
talk to your boss, talk to a ‘real’ person not in an internet chatroom, forum, blog..
AGW, have the media, they have the ‘polar bears, orlando bloom, bono, hollywood, ‘melting icecaps, Hollywood glossy special effects…
what does science have, bunch of techies, bitching about error bars, artificial correction factors, fortran, and emails about scietists beeing rude to each other…
How many comments does this web page have 200 maybe.
6 billion on this planet, less than probably a 1000th% of the planet have even HEARD about the data theft (whistle blowing) and that is probably a huge overestimate…
And all that that tiny, tiny fraction have heards since is spin, lies and media sleight of hand.
tell someone ‘important’ in China, India, Russia third world,
AGW have won.
Apolgies for the rant, I do not mean to offend. I’m off to start saving up to pay for my carbon taxes.

E.M.Smith
Editor
November 27, 2009 3:55 pm

E.M.Smith (10:46:03) : NCDC seems to be Number Three (and I’ve not looked at it yet, but given the various affiliations it is most likely just a trivial GHCN variant.)
NCDC seems to be a synonym for GHCN.
A light slowly dawns…
Starting from a google of NCDC I drilled down their web site and then the ftp links until I ended up in a VERY familiar place… the directory where GIStemp downloads the GHCN data.
So I’ve answered my own question: GHCN is the same as NCDC data set. And I’m already investigating it and finding it bogus.
NCDC, GIStemp, and HadCRUt all agree because they are all GHCN with minor post processing variations… And GHCN has strong biases introduced from deleting thermometers at altitude and in cold latitudes.

Interested reader
November 28, 2009 4:45 am

Letting you know that The Australian National Library has an on going program of digitising historic newspapers from 1803 to 1954 at this link http://newspapers.nla.gov.au/ndp/del/home
A quick search of the word temperature will give you over 92000 hits. Our old newspaper writers were quite interested in land and sea temps.