Joanne Nova and Ken Stewart have uncovered some startling findings about the way Australia’s BOM “High Quality” network data is adjusted. Think USHCN here in America. This is well worth a read. – Anthony
A team of independent auditors, bloggers and scientists went through the the BOM “High Quality” (HQ) dataset and found significant errors, omissions and inexplicable adjustments. The team and Senator Cory Bernardi put in a Parliamentary request to get our Australian National Audit Office to reassess the BOM records. In response, the BOM, clearly afraid of getting audited, and still not providing all the data, code and explanations that were needed, decided to toss out the old so called High Quality (HQ) record, and start again. The old HQ increased the trends by 40% nationally, and 70% in the cities.
So goodbye “HQ”, hello “ACORN”. End result? Much the same.
That meant the ANAO could avoid an audit, since the BOM had changed data-sets, the point of auditing the old set was moot.
For me, this version is so much worse than the previous one. In the HQ data set the errors could have been inadvertent, but now we’ve pointed out the flaws, there can be no excuses for getting it wrong. Instead of fixing the flaws (and thanking the volunteers), it’s almost as if they’ve gone out of their way to not solve them. Instead it’s been complexified, rushed, has many typo’s and gaps, and the point (see below) about the “adjustments having no impact” — when they obviously do — begs to be audited by the Auditor General, the ACCC, Four Corners (ha ha) and 60 Minutes.
To make it all look o-so-convincing, the BOM asked three experts (from NOAA, NZ, and Canada) to look over it all, and score the BOM against its peers. But the peers standards are not too high in the first place: NOAA was caught with 89% of it’s own thermometers in the wrong spots near air conditioners and whatnot, and NZ’s records were so bad, they disowned them themselves. (NZ adjustimongered their temperature trends from 0.06C right up to 0.9C, got caught, and their response under legal pressure was to say but it’s ok, “There is no “official” or formal New Zealand Temperature Record”.)
How useful is it when a team of substandard institutions is asked to evaluate whether the BOM practices are “amongst international best practice” when it is international best practice to ignore concrete, car-parks, tarmac, and lose the data too? We aren’t impressed if the BOM is as bad as the rest of the world, we want open data, transparent methods, and reproducible results. We want high quality to mean, well, high… quality.
So how good is the new ACORN (Australian Climate Observations Reference Network — Surface Air Temperature) set?
Because it covers a vast area, Alice Springs contributes 7-10% of the national signal.ACORN and the BOM claim that since the new results are pretty much the same, really they give more confidence than ever that Australia has warmed since 1960.
Ken Stewart and the independent BOM analysts team have sliced and diced through the ACORN data.
They conclude:
- Like the old HQ series, the Acorn record is also still impossible to replicate.
- The record is much shorter than 100 years for many sites. It’s supposed to be high quality, but it has many gaps and spurious errors. If volunteers can write code on laptops to check for errors — and find, for example, that one 36.8C was accidentally changed to a 26.8C (and there are many) why can’t the Australian BOM?
- Like the old series, Acorn’s trends are very different from what the raw data shows. (Why do we bother with thermometers?)
- Hot and cold extremes have been adjusted, for the most part warming winters and cooling summers, and at some sites new and more extreme records have been set.
Too tricky by half? The BOM tries to hide the effect of adjustments
Here’s a piece of sleight of hand — ACORN, they claim, has a random set of a adjustments of both up and down (which is what we’d expect).
read more here
Looks like Ms. Nova’s site is down.
Nice link
Forbidden
You don’t have permission to access /wp/index.php on this server.
Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.
Don’t know what to tell everyone, the link is valid for me. I’ve double checked and replaced it. Try visiting the main page first, then follow the link from there.
Anthony, received the same error message in both Firefox and Safari. Also tried going to directly to her site. It loaded but the server seems to have the hiccups.
RG
[REPLY: Her main page is OK, but the read more link seems to have a problem. -REP]
http://pindanpost.com/2012/06/18/bom-still-manufacturing-data-homogenized-dud-and-dudder/
Linked OK earlier today when I posted this, but not OK at present. Also a link to Warwick Hughes where BoM fails badly in forecasting too…
Why not just try http://joannenova.com.au/ It is the last post
[REPLY: because you get the error when you click on read more. The main page is OK. -REP]
Still forbidden from Joanne’s e-mail: The news this week (click on the headlines to read the full story):
Threat of ANAO Audit means Australia’s BOM throws out temperature set, starts again, gets same results
Whatever happened to that ANAO audit that we and Senator Cory Bernardi asked for (the one where our auditor general was supposed to go through our Bureau of Meteorology records)? Finally, you can find out…
A team of independent auditors, bloggers and scientists went through the the BOM “High Quality” (HQ) dataset and found significant errors, omissions and inexplicable adjustments. The team and Senator Cory Bernardi put in a Parliamentary request to get our Australian National Audit Office to reassess the BOM records. In response, the BOM, clearly afraid of getting audited, and still not providing all the data, code and explanations that were needed, decided to toss out the old so called High Quality (HQ) record, and start again. The old HQ increased the trends by 40% nationally, and 70% in the cities.
So goodbye “HQ”, hello “ACORN”. End result? Much the same.
That meant the ANAO could avoid an audit, since the BOM had changed data-sets, the point of auditing the old set was moot.
For me, this version is so much worse than the previous one. In the HQ data set the errors could have been inadvertent, but now we’ve pointed out the flaws, there can be no excuses for getting it wrong. Instead of fixing the flaws (and thanking the volunteers), it’s almost as if they’ve gone out of their way to not solve them. Instead it’s been complexified, rushed, has many typo’s and gaps, and the point (see below) about the “adjustments having no impact” — when they obviously do — begs to be audited by the Auditor General, the ACCC, Four Corners (ha ha) and 60 Minutes.
See all the gory details here: especially the part where the BOM tries to argue that there are the same number of warming and cooling adjustments except that the audit team finds the adjustments that increase the trend are larger… Gasp at the graphs.
We are asked to believe that climate models today can accurately predict the future, but that in the first half of the 20th Century simple equipment like thermometers systematically “overstated temperatures”, and that those errors were only just “discovered” recently. Scientists have been using thermometers for 300 years, yet the flaws in most Australian historical thermometers apparently cannot be explained even to people with qualifications in engineering, statistics and maths. Furthermore, historic thermometers are well known to be placed in sites that now are surrounded by concrete, bitumen, and sometimes have electronic heat pumps warming the locale as well (in the form of air conditioners). In some places, tens of thousands of internal combustion engines pass by daily. When common sense suggests that modern thermometers are more likely to have artificial warming sources close by and ought to be adjusted down to compare with historic ones, the BOM adjusts the older historic and better placed thermometers downwards instead. It defies common sense.
‘
I commented there earlier today.
Can’t link now: 10:11 PM Sunday PST
http://joannenova.com.au/2012/06/threat-of-anao-audit-means-australias-bom-throws-out-temperature-set-starts-again-gets-same-results/
Gives me this as others have noted:
The three links to her site all return the same 403 error on the side bar, the here link at end and the Joanne Nova link at the beginning of the article all lead to the index page…?
I have found exactly the same issues with sites in northern Victoria.
In fact for Echuca, the trend line is for cooling since the 1880’s!
The maximum trend line is flat, but the minimum trend line is down.
The other interesting aspect is that the peaks in the maximum temperature correspond to the odd solar cycles, except for the peaks for the last two El Nino events.
The super El Niño of 1998 did not register in Echuca!
It reminds me of GISTemp data. You’ll need to watch both of these videos to see what I mean.
“Does GISTemp change? Part 1”
“Does GISTemp change? Part 2”
I’m in Australia and I’m getting at present when I try to get into Jo’s site the same message as is reported above
You don’t have permission to access /wp/index.php on this server.
Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.
Might be a temporary glitche – I’ve made a couple of comments on her post so I’ll keep an eye on it and report any success here – I’ve also e mailed her
In the meantime you guys could check out Ken Stewart’s blog http://kenskingdom.wordpress.com/2012/05/14/acorn-sat-a-preliminary-assessment/
Jo’s site is back now!
http://cawcr.gov.au/publications/technicalreports/CTR_050.pdf
worth reading
Given the fact NIWA in New Zealand had their temperature data vetted by the BoM and recently had to admit it was “garbage” I am not surprised by this.
hi Anthony,
is it time for the blogosphere to collect data and produce a maximally robust data set ? i.e. us,
i.e. those that read this blog start to collectively collect data to provide an independent set that is non biased -who knows of an accurate weather station with A/D convertors that with a computer and internet connection will auto send data to a site that can collect this info-
i.e. analysed by people from both sides of the warmism divide (and if there are any neutrals), analyse the data set
– there is an excellent chapter in Shaw & Tigg “Applied Mathematica” called “maximum entropy data reconstruction” – that I think would be ideal to find the underlying curve (if there is one) – perhaps there is a skilled operator who can use this technique, will lift the veil that covers the truth
cheers
CM
One has to ask, “Does adjusting the readings modify the climate?” /sarc
Had an email the other day about FROST in Zimbabwe! Crop damage etc. My wife is from a farming family in what was Southern Rhodesia and never experienced frost until I brought her to the UK. Looks like the southern hemisphere is going to have a record winter this year.
If it really is as warm today as they say it is, then where are the previous frost damage records?
‘They’ say it was much colder in the past. Don’t they?
Suspected as much. With the greens and the WWF so deeply entrenched in the sciences here , it is no wonder they produced a biased report. Apparently, the only way to be promoted in BOM is to identify with the warmist ideas. I would gladly be corrected on that.
Their initial report was exposed because they showed a warming trend at Darwin that just did not exist when compared to the raw data.
So it is excellent to have it confirmed.
To repeat Patrick Davis’s point. NIWA seemed unaware that actual thermometer readings were stored in the national library. It had ….ah…disappeared all the readings at NIWA itself. When the NIWA historical temperature results were compared with the actual readings of the 11 or so meteorological stations, they clearly had been falsified.
Unfortunately, the majority won’t know what has been going on because most still rely on the Lame Stream Media as their ‘News’ source and we know the media think their job is to keep the masses oblivious to
the REAL facts. We need new leaders, smaller governments, new teachers and new, honest journalists. Tough to do when our schools are indoctrinating younger generations to believe all of this crap. It’s going to be a TOUGH road ahead.
http://joannenova.com.au/2012/06/threat-of-anao-audit-means-australias-bom-throws-out-temperature-set-starts-again-gets-same-results/
Okay at 5:20 pm Australian time. (EST)
Over at Joanne Nova’s site she makes this important point:
Global warming is man made and Mann made. They are trying to sell us a pig in a poke. “Trust us, the science is clear.”
Nothing good can come of anything with the acronym ACORN…
http://www.aim.org/aim-column/acorn-scandals-grow-grow/
Nearly half of the warming in Australian temperatures over the last 60 years is spurious, resulting from using minimum and maximum temperatures as I explain here.
http://www.bishop-hill.net/blog/2011/11/4/australian-temperatures.html
They persist in using Min/Max temperatures even though much more representative (of the average) temperatures recorded at fixed times are available, because it gives them nearly twice as much warming.
Ian Bryce, can you email me. I have tried your spc email and it bounced. Thanks, Jo
Ken Stewart comments over at Jo Nova’s blog that this is “definitely not world’s best practice”. The depressing thing is that he could actually be wrong.
She moved, stealthy, crossing patches unseen, one determination, one enough
A dagger of gold, a pen of wit, nothing did not she question, everything to be known
Blithely engorged on feats of slather, they grew rump, their song and dance she purloined, for her nest
On a night when the fire burned high, she took her swill, with power and might of will, the dagger did she plunge
Not one of them could defend, not one of them could survive
The truth she held up high, the crown upon her head
They of ill repute, will forever fear
Jonova
I’m an Australian currently in theUK and have no problem accessing any of the links. Perhaps they are now all OK or were the Australians blocking?
Unfortunately although the actions of the BOM which is totally a CAGW supporter, will probably cause indignation and outrage on blogs such as this, the average Aussie will never get to hear of it as the MSM don’t usually report such stories
Hi the link is working fine for me. The MP’s in Canberra need to get a grip on this and I would suggest an immediate Audit of the crooks at the BOM.
From little ACORN’s, mighty oak hockeysticks grow.
If the data does not prove the theory change the data.
A whole new way to do science.
In response, the BOM, clearly afraid of getting audited,
————
Sigh, another nova rant. I feel the little flecks of spittle landing on my face. Why is it when people say “clearly” I know it ain’t so?
We are back again to the ‘lack of trust’ in science and ‘peer reviewed research’. So California Air Resources Board is caught out with faulty research that provides the desired outcome and the whistle blower is sacked from the university post that he was funding. The Australian BOM is caught fiddling the historical figures to provide a desired outcome and immediately dumps them and generates a new set. This smacks of delay until something expected in the future – could that be the passing of the Carbon Tax?
We find from posts here that random noise is more accurate at forecasting the ‘climate’ than the much vaunted GCM ‘clmate models’. These GCM are so much worse than random that one suspects that they are tuned to provide a desired outcome that is not in keeping with the actual climate. Again do not trust flag is raised.
Same with New Zealand figures massaged to provide a desired outcome and when the massaging is publicized they disown the database.
This massaging to provide a desired outcome seems to be prevalent in climate ‘science’ in a way that would be completely unforgivable in any other scientific or engineering area. This is not something minor like a drug trial that would only kill a few thousand people – this is literally affecting the entire human race. Massive amounts of taxpayer money is being wasted paying for these climate scientists to massage data for political and their own funding ends, when children are dying at the rate of one every 5 seconds from hunger. How can these people look at themselves in the mirror?
Hmmmm, GAO could do the same here
11:20 Eastern, I’m getting “Account Suspended” even on the main page. Hoping everything’s ok.
I get this message now when trying to follow the Jo Nova links
“This Account Has Been Suspended”
Obvious enough ploy. They wanted to avoid having the data set audited, the only way to do that was to switch to another data set, now the entire process has to be repeated for this data set. I guess when it gets close enough to being audited, they’ll trot out a new data set. They may have found the perfect dodge.
Jo Nova: This account has been suspended.
Yup, something bad seems to be going on.
http://joannenova.com.au/cgi-sys/suspendedpage.cgi
Perhaps Jo will contact Anthony and let us know what the problem is.
Either Jo Nova “forgot” to pay her ISP bill or the government thugs found a pretext to shut her down. I seriously doubt the former. If the latter, it shows how desperate the thugs are to control the flow of information about their takeover of every aspect of our lives. It also shows how effective Jo has been in exposing them. It appears quite ominous to me but I know nothing beyond the suspension notice.
Does anyone know anything about Jo’s situation beyond “This Account has been suspended.”?
REPLY: It could be something as simple as configuration error that directs to this default message. Until we find out, let’s not speculate about motives. – Anthony
Sigh. Apologies to all concerned. Anthony just let me know about the “suspended” message on my site. This is a second temporary outage today. When we moved the site recently a bit of rouge script deleted 100,000 comments. We’ve just restored them in the last few days. My webmanager said: “”Today your site almost took down one of my servers. Hehe. Not sure what it was, looks like your site got recrawled…. It takes a while to isolate problems given the monsterous size of your site’s log files. You seem to add, on average, 1MB of log files everyday.” I’m guessing that the site traffic plus recaching/recrawling, has taken down the whole server.
Sorry for the inconvenience. I expect things will probably be restored later today. (It’s the early hours of the morning over here).
Thanks to Anthony for letting readers know!
Jo
Lazy Teenager is a well known warmist in Australia. You can tell he/she is totally pro CAGW from statements such as that posted here which is below
“In response, the BOM, clearly afraid of getting audited,
————
Sigh, another nova rant. I feel the little flecks of spittle landing on my face. Why is it when people say “clearly” I know it ain’t so?
This person clearly is afraid of getting to grips with fact. Why is he/she insulting Jo Nova? Does he/she think that the adjustments stated are pure fiction from Joanne Nova? Is he/she short of a rabbit in the paddock? What evidence does he/she have that it “clearly ain’t so” Equally, why is it not clear that the BOM is afraid of getting audited?
The actual process is simply , happy politicians keeps the money flowing in , right now Gillard is right in the sh*t partly because has they become so linked to the AGW scare that there is way out for them. This data manipulation, becasue of the support it offers to the AGW scare, is a god send to her as it provides the fall back of saying ‘its what the science says ‘ This in turn makes them happy which leads in turn to the research cash taps being kept open .
So its smart if not actual good scientific practice, like so much in ‘climate science’ its the politics which are important not the reality of the data .
Tom Harley says
“It defies common sense.”
Actually, this nicely describes the entire AGW religion.
Common-sense shore-dwellers know that sea-level rise is nowhere a problem.
Common-sense metrologists know how bad thermometer siting will destroy all scientific value.
Common-sense software engineers know that even petaflops running on petabytes are thin gruel compared to climatic reality.
Common-sense electrical engineers know that windmills are a grid nightmare.
Common-sense birdwatchers see daily bird-kills by windmills.
Common-sense economists see what a fraud biofuels are.
I could literally go on forever.
AGW and common sense have nonoverlapping Venn diagrams.
Common sense slides off AGW like water off a duck….
Alex Heyworth:
You’re right, I didn’t appreciate the irony in what I wrote!
What a shame Jo’s site went down just when my post was getting some much needed attention!
Ken
Steve Mosher gives the rationale behind the study http://cawcr.gov.au/publications/technicalreports/CTR_050.pdf .
It reveals their thought processes, and, as, discussed above, their bias may well be to match the 20th Century CO2 rise to the (adjusted) temperature record, as a form of confirmation bias. This may sound very strange on this column, but reading the introduction gives revealing insights such as
So, “changes” and “trends” are common terms here, little interest is shown in the finer details of absolute temperatures, only trends. I suspect that they would regard every flat or downtrending temperature as an anomaly, an error of recording that needs to be adjusted. This abhorrent idea to us may very well be their mindset.
Why “trends” and ” changes” ?, well they are easily understood by lay audiences.This argument would also explain why the up and down adjustments are claimed to be random, when upon further dissection they bias early measurements down and later measurements up to produce a positive trend line. This has been discussed before on WUWT.
I suspect that in a Senate Estimates Hearing (similar to Congressional inquiry) in our Parliament, an interview with BOM employees would produce a discussion that would be completely over the heads of the Senators asking the questions and the media.reporters. The latter settling upon the trends and changes theme as ‘logical’ and easily understood. They may even bring out that old chestnut Occam’s razor to appear credible.
Another observation on that report
Under Data is this….
I know Observatory Hill, near the Harbour Bridge and Sydney Harbour, a recreational reserve subject to sea breezes, especially in the afternoon. In Tim Flannery’s latest outburst, OH seemed very stable in comparison (in his opinion) to the western suburbs.
Let’s have a look at 066062, Observatory Hill. over the recent 30yr period (I use max) http://reg.bom.gov.au/jsp/ncc/cdio/cvg/av take a look at ANY 30yr period, very stable indeed.
So, why omit it, are they trying to fool people into thinking it is ‘urban’, or, doesn’t it suit their agenda ?
http://reg.bom.gov.au/jsp/ncc/cdio/cvg/av is the link but itdoes not seem to work. Go to http://reg.bom.gov.au/climate/averages/tables/cw_066062.shtml and to the right ‘plot’.
Well, Mosher should read this passage from Jo Nova’s site about the ” adjustments ”
QUOTE
The official CAWCR Technical Report No. 04:
“There is an approximate balance between positive and negative adjustments for maximum temperature but a weak tendency towards a predominance of negative adjustments (54% compared with 46% positive) for minimum temperature.”
But the independent auditors point the positive adjustments are larger than the negative:
” While there may be a numeric balance of positive and negative adjustments, analysis of a representative sample indicates that adjustments predominantly increase warming.”
UNQUOTE
Richard111 says:
Had an email the other day about FROST in Zimbabwe! Crop damage etc. My wife is from a farming family in what was Southern Rhodesia and never experienced frost until I brought her to the UK. Looks like the southern hemisphere is going to have a record winter this year.
So far as “the team” are concerned that would have to be “local”. That Zimbabwe is north of the Tropic of Capricorn they’d probably dismiss out of hand.
Steven Mosher says:
June 17, 2012 at 10:47 pm
http://cawcr.gov.au/publications/technicalreports/CTR_050.pdf
worth reading
The instant takeaway for me was the following in the abstract:
Temperature changes from 1911 to 1960 show some degree of sensitivity to the choice of
network and analysis method, which reflects structural uncertainty due to sparser network
coverage during this time. Temperature changes from 1961 to 2010 are much less sensitive to
these issues, and the network coverage is fairly stable over this later period.
I have older series HQ and raw data + some R script at my finger tips. These show over 100 year period a variability between HQ and raw of 40% higher anomaly figure. Now rerunning for line fit periods 1910-1960 and 1960-2010. I get following:
1910-1960: HQ anomaly 0.04. Raw anomaly: -0.25.
1960-2010 : HQ anomaly 1.73. Raw anomaly: 1.49
I am uncertain as to how to compare these in order to meaningfully measure sensitivity. Though the absolute variation between HQ and Raw is the same for both periods. That warming is greater after 1960 is certainly borne out by the numbers I get.