Via Jo Nova, an Australian Surface Stations Project has just reported its results.
The BOM say their temperature records are high quality. An independent audit team has just produced a report showing that as many as 85 -95% of all Australian sites in the pre-Celsius era (before 1972) did not comply with the BOM’s own stipulations. The audit shows 20-30% of all the measurements back then were rounded or possibly truncated. Even modern electronic equipment was at times, so faulty and unmonitored that one station rounded all the readings for nearly 10 years! These sloppy errors may have created an artificial warming trend. The BOM are issuing pronouncements of trends to two decimal places like this one in the BOM’s Annual Climate Summary 2011 of “0.52 °C above average” yet relying on patchy data that did not meet its own compliance standards around half the time. It’s doubtful they can justify one decimal place, let alone two?
We need a professional audit.
A team of independent engineers, scientists, statisticians and data analysts (brought together by the joannenova blog) has been going through the Australia Bureau of Meteorology records (BOM). They’ve audited some 8.5 million daily observations across 237 High Quality and other close sites in Australia. Shockingly, while the BOM calls their database “High Quality” and instructed observers before 1972 to record in tenths of a degree Fahrenheit, the auditors started finding sites with long stretches of records where the weather suspiciously rose and fell only in Fahrenheit quanta, like 72.0, 73.0, 72.0, 71.0, 73.0, 72.0. After 1972, the BOM went metric, and oddly, so did parts of the Australian climate. Numerous sites started warming and cooling in pure Celsius integers.
The bottom line:
- The BOM records need a thorough independent audit,
- It’s possible that a significant part of the 20th Century Australian warming trend may have come from something as banal as sloppy observers truncating records in Fahrenheit prior to 1972.
- Many High Quality sites are not high quality and ought to be deleted from the trends.
- Even current electronic equipment is faulty, and the BOM is not checking its own records.
- Even climate scientists admit that truncation of Fahrenheit temperatures would cause an artificial warming effect.
Ken Stewart has the whole in-depth report at his site: “Near Enough For a Sheep Station”
They have done a huge amount of data crunching. Ken has all the graphs of maxima and minima (people were extra lazy on the minima).
Then there is the wierd effect of rounding Fahrenheit to Celcuis and back and getting results of 0.1 and 0.9 when the regenerated Fahreheit records are used instead of the original.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Australian temperature records shoddy, inaccurate, unreliable
Will BEST and HadCRUT4 use these data? And what should we think if they show that 2010 was warmer than 1998?
Slightly off topic, but a recent release in Australia suggests that oceanic temperatures around Australia are at “a record high”. Given the paucity of quality daya onshore I wondered how they meausured the oceanic temperature in 1973/4. Two blokes and a tinnie? Equally I note that NOAA has oceanic heat content records running back to 1955. I know you Americans are a clever bunch, but how was that done?
Prevaricator.
Warwick Hughes brings a bit of perspective to the BOM’s recent alarmist spin concerning record heatwaves in Perth through the summer, March period.
No doubt about it, here in Australia the BOM and the CSIRO provide the best climate science a government can buy!
I would also like to point out that vast swathes of rural Australia have been cleared for farming between 1910 (when the measurements started) and the present. Even those in uncleared areas have been subject to significant grazing and are well recognised as being under vegetated compared with the prisitne condition. The cooling effect of vegetation is well understood and this alone could have produced the half a degree of warming that the BoM and CSIRO has identified. And I’m not even joking – the report identifies significant errors in the results from Katanning Western Australia. For the sake of History Katanning was gazetted as a town in 1898, just 12 years before the BoM records are supposed to have started. Today, Kataning is in the middle of the Wheatbelt and is subject to broadscale wheat cropping. For those of you that don’t know – in WA wheat is grown in the winter and harvested just before Christmas leaving the land cleared during the hottest time of the year.
Rounding up from 5 is correct because 0 (zero) is also a number, and therefore there are 5 numbers below 5 and 5 numbers from 5 to 9.
garymount says:
Rounding up from 5 is correct because 0 (zero) is also a number, and therefore there are 5 numbers below 5 and 5 numbers from 5 to 9.
**************************
But 0 (zero) is never rounded and therein lies the problem.
Here’s why:
1.0 + 1.1 + 1.2 + 1.3 + 1.4 + 1.5 + 1.6 + 1.7 + 1.8 + 1.9 = 14.5 or avg = 1.45
Rounded
1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 2.0 + 2.0 + 2.0 + 2.0 + 2.0 = 15 or avg 1.50
Beware of stories such as this preceding the dumping of older data.
Very recenly, many Australian temperature records available on the internet have been cut off to now only go back to 1950, wheras previously they went right back to the early 1900s or late 1800s.
The very interesting thing about that is that all the current Australian record high temperature readings were made in the 1940s during the WWII drought. (and yes, Mabel, it was hotter than today!)
Reg Nelson says:
March 15, 2012 at 6:43 pm
1.0 + 1.1 + 1.2 + 1.3 + 1.4 + 1.5 + 1.6 + 1.7 + 1.8 + 1.9 = 14.5 or avg = 1.45
Rounded
1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 2.0 + 2.0 + 2.0 + 2.0 + 2.0 = 15 or avg 1.50
——
You then have to round 1.45 to 1.5 because your 1.0 etc. has implied errors, showing only accuracy to the one tenth, you can not have the hundredths column in your avg. result. Therefore both rounded and unrounded calculations of their respective average are identically 1.5, ergo I am correct.
Easy there, unless you can somehow disabuse me of the notion that IR spectroscopy rests, no, depends on the various absorption ‘bands’ of various gases like CO2, H2O, CH4 etc.
.
The odd-even rounding is a standard method in psychometric and academic achievement standardized testing. For the very reason stated above. Simple rounding rules result in artificially inflated data. Anyone with a graduate level stats class under their belt would understand this. But I have heard that our Ivy league climate scientists are not versed in graduate level statistics.
Non-conforming material (like ‘bad thermometers’ not meeting a particular spec they were purchased against) is one thing; proper specification of material (or test apparatus, including thermometers) is another …
The simple answer might have been to create a ‘cal’ chart for each of a required number of so-called ‘inaccurate’ thermometers, if their repeatability was good (could be assured) against a more expensive ‘reference’ thermometer.
I have seen ‘Lab’ personnel on occasion with surprisingly unrealistic expectations of the equipment they use …
.
garymount says:
You then have to round 1.45 to 1.5
****
No, you don’t. 1.45 is the actual average measurement, 1.5 is the rounded average number which is always higher. Therefore given a random distribution, the rounded number will be greater.
If you round the total average of 1.45 to 1.50, you introduce 0.5 degrees of warming where there was none.
Case uing round down instead of round up at the 5:
No rounding, until after the calculations
1.0 + 1.1 + 1.2 + 1.3 + 1.4 + 1.5 + 1.6 + 1.7 + 1.8 + 1.9 = 14.5
Round to proper significant digits, using round down at 5
= 14
Avg = 1.45, after round = 1.4
1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 2.0 + 2.0 + 2.0 + 2.0 = 14
Avg = 1.4
My conclusion, consistent use of rounding, not mixing the varieties of rounding one is using, seems to be proper technique.
From Kasuha on March 15, 2012 at 4:06 pm:
I decided to check if rounding can cause a bias. I started with °F rounded to an integer, as it was mentioned above this was apparently happening with strings of XX.0 numbers being reported, then converted to °C with rounding to one place, then considering the effect mentioned by Anthony Watts at the end of the piece I converted back to °F with rounding to one place. Then the differences in °F readings were calculated and summed. This was done in a spreadsheet with 2000 random numbers between -40 and 120 as the starting values, which I figured was a good large range for ground level Fahrenheit values. If rounding has no effect, all the differences should sum to zero.
OpenOffice 2.4.1, Calc (the spreadsheet). Generating new random values was done by highlighting all of them, then filling downward. As this wouldn’t change the first value, an extra row at the top held a starting random number using the same equation, this cell was highlighted with the rest.
Layout: 1st row: column descriptions; 2nd row: starting random value in 2nd column; 3rd row: start of values. 1st column starting at R3, numbers 1 to 2000.
Equations:
2nd column, Initial °F value, rounded to integer, range -40 to 120:
=ROUND(RANDBETWEEN(-400;1200)/10;0)
3rd column, Conversion to °C, round to one place decimal:
=ROUND((B3-32)*5/9;1)
4th column, Convert back to °F, round to one place decimal:
=ROUND(C3*9/5+32;1)
5th column, Compute difference between last and first °F:
=D3-B3
Sum differences:
=SUM(E2:E2001)
Starting random value 21.
Run, difference sum
1, +2.8; 2, -0.6; 3, +1.9; 4, -2.7; 5, +1.7
6, +1.9; 7, +2.3; 8, -1.8; 9, -2.1; 10, -4.7
11, -2.4; 12, +5.0; 13, +1.5; 14, +0.7; 15, +3.1
16, +1.5; 17, +1.4; 18, +6.4; 19, +2.9; 20, +2.6
Average difference of 20 runs:
1.07, round to 1.1°F
On individual runs, a positive bias was introduced 13 of 20 times. An individual run is comparable to a fixed amount of temperature records. Overall there was an average bias over twenty runs.
I expanded to 10,000 rows of values and ran 10 runs, using 3 different starting random values. Reseeding the starting value done by copying its equation and pasting it back into the same cell.
Starting random #: run results
# + runs, # – runs, 10 run average
89: +6.1, +7.8, -2.0, +9.3, +2.6, +13.4, +12.8, +7.8, -1.0, +17.2
+8, -2, +7.4
116: +2.8, +5.5, +3.6, +9.4, -0.9, +20.1, -5.6, +21.7, +14.4, +17.8
+8, -2, +8.9
15: +10.1, +11.8, -5.0, +8.8, -0.9, -6.6, +6.9, +6.0, -6.6, +19.7
+6, -4, +3.4
With consistent use of the same rounding method over a large range of expected values, a significant persistent positive bias is noted. As seen on individual runs, which are comparable to a fixed series of temperature records, the amount of total bias can be quite large.
Thus I conclude that rounding of temperature records can introduce bias thus rounding should be avoided.
Sorry, typo. s/b 0.05 degrees of warming.
LazyTeenager says:
March 15, 2012 at 3:34 pm
“Personally I wouldn’t trust JoNova’s fan base to be able to add up a shopping list without stuffing up.”
============
Very funny, which is not to say I agree.
Who would you trust ?
ThePowerofX says:
March 15, 2012 at 11:55 am
“A team of independent engineers, scientists, statisticians and data analysts (brought together by the joannenova blog)”
I stopped reading after this.
______________
Let’s see… weren’t you one of those people who always told everyone else to ‘think outside the box’? (when that phrase was hip and cool)
Reg Nelson says:
March 15, 2012 at 7:43 pm
If you round the total average of 1.45 to 1.50, you introduce 0.5 degrees of warming where there was none.
—————
The problem in all this is that the error margins must be shown. You say that you have exactly 1.45. But you don’t have exactly that number. Only if you are counting individual objects such as I have exactly 5 brothers and sisters. The more numbers you calculate with that have error margins, the errors keep accumulating, the spread gets larger, the accuracy decreases, and you must show this when rounding. It is not proper to round then reassemble pretending you still have the same accuracy as you had before rounding. And I believe that is what we are complaining about what is taking place here in climatology.
I have spent aver 2,000 hours in recent years studying calculus, and I have to admit, I haven’t spent much time on whether or not bias is introduced to a warming effect when rounding numbers, but I still remain skeptical, and do so because the addition of showing the newly added error margins should take care of any bias that may be introduced.
I think I need new glasses, aver should be over, in case anyone has noticed in my last comment. Also, apparently aver is a proper word according to my autocorrect spelling software I am using when composing comments for this blog.
Phillip O’Neill says: March 15, 2012 at 3:15 pm
[ … ] So Mr Watt, you owe Australian Observers a huge apology for your statement and await yuour writtena apology…..
=============================
Not so Phil. I’m an Aussie and I find this episode embarrassing given how we are quick to remind anybody about how clever we are … if I ever hear “world’s best practice ” again I will clock that person.
KeithH at JoNova, March 16, 2012 at 8:57 am, sleuthed a bit following a comment from me and found the emails that I referred to”
“Streetcred @ur momisugly 5. This may be what you remember but if not, it sure highlights the point!
“I recall reading recently, one of the ClimateGate 2.0 emails regarding records of Australian temperatures from BoM. If my memory serves me correctly, the email lamented the poor quality of the Australian records.”
Quote:
■FOIA\documents\HARRY_READ_ME.txtgetting seriously fed up with the state of the Australian data. so many new stations have been introduced, so many false references.. so many changes that aren’t documented. Every time a cloud forms I’m presented with a bewildering selection of similar-sounding sites, some with references, some with WMO codes, and some with both. And if I look up the station metadata with one of the local references, chances are the WMO code will be wrong (another station will have it) and the lat/lon will be wrong too.
■FOIA\documents\HARRY_READ_ME.txtI am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that’s the case? Aarrggghhh! There truly is no end in sight!
■FOIA\documents\HARRY_READ_ME.txt OH F..K THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done I’m hitting yet another problem that’s based on the hopeless state of our databases. There is no uniform data integrity, it’s just a catalogue of issues that continues to grow as they’re found.
http://wattsupwiththat.com/2009/11/25/climategate-hide-the-decline-codified/ “
garymount says:
The problem in all this is that the error margins must be shown. You say that you have exactly 1.45. But you don’t have exactly that number.
*****
Error margins are an entirely different issue. If you have unreliable data, you have unreliable data. Rounding unreliable data still leaves you with unreliable data. It doesn’t correct the problem.
If you have unreliable data you can’t make reliable projections (or predictions) based on the data.
_Jim says:
March 15, 2012 at 7:18 pm
Ian W says March 15, 2012 at 1:32 pm
…
These climatologists come from the large group who believe that so called ‘green house gases trap temperature;
Easy there, unless you can somehow disabuse me of the notion that IR spectroscopy rests, no, depends on the various absorption ‘bands’ of various gases like CO2, H2O, CH4 etc.
Jim, I see you are one of the large group believing that green house gases trap temperature.
I have no need to disabuse you of the notion that IR spectroscopy depends on the various ‘absorption’ bands of IR. What you do need to be disabused of is being able to measure the quantity of IR (that is heat ) absorbed by the atmosphere using a thermometer. The heat capacity of the atmosphere varies considerably dependent on its humidity. See http://www.engineeringtoolbox.com/enthalpy-moist-air-d_683.html
If you don’t know the humidity of the atmosphere at the time you measure the atmospheric temperature you have little idea of the heat that has been absorbed by the gases in it. If you look at the examples given in the URL above, and do a little calculation, you will see that a Louisiana Bayou at 100% humidity after an afternoon storm with the temperature at 78F will hold twice as much heat energy as the air in the Arizona desert at close to zero humidity but at 100F. Whereas you would be happily saying the opposite just measuring temperature – and you would be wrong.
As you say the entire claim of ‘global warming’ is that heat energy is absorbed by the atmosphere… but then climatologists measure atmospheric temperature and not heat content. It is the incorrect metric for measuring heat. If climatologists were really serious they would be measuring the ocean temperatures because unlike the atmosphere there is little variance in enthalpy in the oceans.
Therefore arguing about the level of precision and accuracy of the incorrect metric is a waste of effort but that is the kind of debate that climatologists get involved in. The money spent on these misguided efforts would be far better spent elsewhere on real problems.
OK, I have figured it out with the rounding issue. So long as the number of recordings below x.5 remain relatively the same as the number of recordings of x.5 to x.9 through time, i.e. the number of recordings show a consistent randomness or spread throughout the time series, then there will be no warming bias trend. The temperatures throughout the series might be biased into warmer temperatures, but as long as the same rounding algorithm is used throughout the time series, there will be no magical increase in temperatures just because you are increasing the number of readings, and rounding’s through time. Just as long as you use the same rounding method in the early part of the series as you do in the later part, and actually throughout the series. Once again, the temperatures might be higher than they should be, but there will be no warming trend bias introduced.
The article that this thread is about clearly shows that they found that the type of rounding done had indeed changed from earlier in the time series from the rounding done in the later part of the time series which did introduce a warming bias. If a consistent rounding was used throughout, there would not be an introduced warming bias.
Kasuha at 3.02 pm hits the nail on the head. The Bureau of Meteorology (BOM) data probably does favour an increase in Australian temperatures and it certainly needs careful scrutiny. There is (and I live in Australia) an attitude in government, the CSIRO (premier scientific body in Australia)and the BOM to disregard any evidence that suggests global warming is entirely due to human burning of fossil fuels. Until results such as these are published in a peer reviewed paper (possibly not E&E which the climate scientists denigrate) and hit the MSM, no amount of publicity in internet blogs will have the slightest impact