Via Jo Nova, an Australian Surface Stations Project has just reported its results.
The BOM say their temperature records are high quality. An independent audit team has just produced a report showing that as many as 85 -95% of all Australian sites in the pre-Celsius era (before 1972) did not comply with the BOM’s own stipulations. The audit shows 20-30% of all the measurements back then were rounded or possibly truncated. Even modern electronic equipment was at times, so faulty and unmonitored that one station rounded all the readings for nearly 10 years! These sloppy errors may have created an artificial warming trend. The BOM are issuing pronouncements of trends to two decimal places like this one in the BOM’s Annual Climate Summary 2011 of “0.52 °C above average” yet relying on patchy data that did not meet its own compliance standards around half the time. It’s doubtful they can justify one decimal place, let alone two?
We need a professional audit.
A team of independent engineers, scientists, statisticians and data analysts (brought together by the joannenova blog) has been going through the Australia Bureau of Meteorology records (BOM). They’ve audited some 8.5 million daily observations across 237 High Quality and other close sites in Australia. Shockingly, while the BOM calls their database “High Quality” and instructed observers before 1972 to record in tenths of a degree Fahrenheit, the auditors started finding sites with long stretches of records where the weather suspiciously rose and fell only in Fahrenheit quanta, like 72.0, 73.0, 72.0, 71.0, 73.0, 72.0. After 1972, the BOM went metric, and oddly, so did parts of the Australian climate. Numerous sites started warming and cooling in pure Celsius integers.
The bottom line:
- The BOM records need a thorough independent audit,
- It’s possible that a significant part of the 20th Century Australian warming trend may have come from something as banal as sloppy observers truncating records in Fahrenheit prior to 1972.
- Many High Quality sites are not high quality and ought to be deleted from the trends.
- Even current electronic equipment is faulty, and the BOM is not checking its own records.
- Even climate scientists admit that truncation of Fahrenheit temperatures would cause an artificial warming effect.
Ken Stewart has the whole in-depth report at his site: “Near Enough For a Sheep Station”
They have done a huge amount of data crunching. Ken has all the graphs of maxima and minima (people were extra lazy on the minima).
Then there is the wierd effect of rounding Fahrenheit to Celcuis and back and getting results of 0.1 and 0.9 when the regenerated Fahreheit records are used instead of the original.
It’s not clear to me why rounding would always bias up.
When you round data off, its usually up, otherwise you call it truncating data.
[Multiple screen names violate site Policy. ~dbs, mod.]
More of what we knew already: even advanced countries temp records are patchy, riddled with siting/equipment/operator error, so the Democratic Republic of Congo’s record is probably somewhat less good. Put them all together and you can, limitations notwithstanding, get a rough idea of global temperature. But a rough idea is no good for convincingly constructing the argument that the world has significantly warmed in recent times (because we don’t actually know what the ‘normal’ temperature might have been) so every other possibility being eliminated, it must be CO2 which means it must be mankind. Now if I, merely a lucky taxpayer, can figure that out, why can’t our ‘political elite’ hahahaha?
Bill, if one truncates the data and simply lops off everything after the decimal, you’re left with an integer value that is, in fact, less than the value that included the decimal. Once you restore use of the decimal value, any trend that includes the old, truncated values and the new decimals would show a warming trend.
ThePowerofX says: @ March 15, 2012 at 11:55 am
“A team of independent engineers, scientists, statisticians and data analysts (brought together by the joannenova blog)”
I stopped reading after this.
__________________________________
Why because it was a bunch of independent people who are not eating from the public trough like the Climate Scientists?
Warmers quit reading after anything the least bit contradictory to their religious beliefs.
Imagine how accurate those wet bulb analog readings were… not. Of course, if you can beleive tenths of a degree accuracy from millenia old tree rings… I guess you can believe anything.
Bill says:
March 15, 2012 at 11:35 am
It’s not clear to me why rounding would always bias up.
1 – 4 round down
5 – 9 round up
That’s 4 rounding down, 5 rounding up. On average, rounding will result in an increase.
I remember an old story from Boeing, back in the slide rule days. They recognized this bias and added an extra rule. If the whole number was even, then 5 rounds down. If it’s odd, it rounds up.
4.5 rounds to 4. 5.5 rounds to 6.
Is this the same Australian BOM that some years ago advised the Government that prolonged drought would be the norm in Queensland? This advice led them to build a $6 Billion desalination plant which was never used and has now been mothballed.
If you cant trust their advice what is their point?
I think that people forget that the older mercury thermometers are only accurate to a degree or so; the standard error of the readings was plus or minus a degree or two. I remember using some maxima-minima thermometers and I needed twenty of them to get a dozen of them to give the same reading! Scientifically one could explain a warming trend of less than a degree a century, which it is, by this explanation, but politically this is just not accepted.
And now GHCN are making their “Iceland” temperature adjustments in Alice Springs, creating a false warming trend.
http://notalotofpeopleknowthat.wordpress.com/2012/03/15/an-adjustment-like-alice/
Sounds like a bunch of Jazz players trying to pass off a classical ensemble, playing by ear
Tez says:
March 15, 2012 at 12:41 pm
Is this the same Australian BOM that some years ago advised the Government that prolonged drought would be the norm in Queensland? This advice led them to build a $6 Billion desalination plant which was never used and has now been mothballed.
If you cant trust their advice what is their point?
Tez. I just recently read that the $6 Billion desalination plant has to run a minimum amount of water though to keep it from a Flannery seizure or some like that. The good news that it only costs a mere $40+ million / year to keep it at minimum output. Isn’t that sweet for thee taxpayers?
Thanks to Tim Flannery the Flim Flam Man of Climate Change!!!
@ PowerofX
““A team of independent engineers, scientists, statisticians and data analysts (brought together by the joannenova blog)”
I stopped reading after this.”
Deny all you like, buddy !!
If you don’t read the truth, you will never know it..
which , of course, is your religious rite.
Robert Orme says: @ March 15, 2012 at 12:57 pm
I think that people forget that the older mercury thermometers are only accurate to a degree or so; the standard error of the readings was plus or minus a degree or two. I remember using some maxima-minima thermometers and I needed twenty of them to get a dozen of them to give the same reading!
_______________________________________________
Then you were lucky. I went nuts ordering thermometer after thermometer and trying to get even ONE that was accurate enough for my lab. These were L shaped specialty thermometers and had to be calibrated against an NBS standard. Heck I could not even use the darn things because the correction factor put the reading off the scale!
Anyone who has mess with thermometers in a lab has a distrust for thermometers if they are smart.
ThePowerofX says:
March 15, 2012 at 11:55 am
“A team of independent engineers, scientists, statisticians and data analysts (brought together by the joannenova blog)”
I stopped reading after this.
Mr. X it is hard to learn anything if you stop reading when the words get difficult.
Without digesting it all, I comment that I was taught that rounding is that anything less than 5 in the last place goes down, anything 5 or higher goes up. Over a large sample size that should average out.
But I’ve seen people truncating (just drop the last digit – which biases the data) but incorrectly call it rounding.
Of course it all depends on the calculation and data. If you ask me before leaving your abode to go to the street for me to pick you up what the temperature is, I won’t worry about precision – it is enough to know if it is well below freezing or near freezing or warm (if cold, knowing that there is high wind is more significant than precision). But if using the difference between two values that are close in size precision is critical. That’s the case with atmospheric temperature.
Thanks for posting, Anthony.
The whole point is: if data were simply rounded, it would cause uncertainty but no change to the trend. But as half of all sites examined had very probably half of their readings rounded, even a small amount of truncating at these sites would cause artificial warming. If truncating was significant, the effect would be larger. There are large regional areas where we estimate the effect could be 0.1 to 0.4C difference in trend.
Ken
As anyone who has done weather obs knows – the last thing that you would be thinking about is that some twerp of a climatologist would be trying to use your manual readings of a thermometer to generate averages to an accuracy of hundredths of a degree.
These climatologists come from the large group who believe that so called ‘green house gases trap temperature; so that is what they measure. Who cares if its illogical and the incorrect metric they can use it and the man in the street and PhD climatologists believe them.
However, those who are truly scientific like forecasters and real weather men know that so called ‘green house gases’ interrupt the radiation of heat and that measuring temperature without knowing the enthalpy of the atmosphere at the time is meaningless. It is perfectly possible for the heat content of the atmosphere to drop even as the temperature rises.
Nevertheless, we have long records of the incorrect metric – so the climatologists happily patch that onto the almost as reliable treemometer records and even then they have to fiddle statistics and hide declines.
While you read this around 12 children have died of starvation, tainted water or malaria…
one every 5 seconds
A $1 a day can save a life
How much are these people paid?
The artifical trend arises because OLDER measurements were truncated i.e. these readings are lower than actual levels. More RECENT records are not truncated in the same way, so these readings are not lower than actual.
Imagine there was a constant temperature of 15.5 deg. Older temperatures would be recorded at 15 deg, recent temperatures would be recorded at 15.5 deg. hence there would be an artifically measured trend.
AS posted earlier, the trend is caused not by “rounding”, but by “truncation” of the pre 1970 values. Truncation simply removes the decimal, recording everything from 72.0 to 72.0 as 72. Once this stopped being done, the new figures would be between 0 and 0.9 degrees higher, thus creating a warming trend.
Without reading the report in full, it is not clear what the evidence is for truncation vs rounding. There may be none, but it certainly introduces a reduction in quality when data of uncertain provenace is used.
The BOM only shows temp results from 1910 (both land and sea). This hides the high Australian temps in the 1880s and 1890s. Their reason is that stevenson screens were not widely used before 1910, but the BOM provides no evidence of this.
1. Did they use Stevenson screens for sea temp?
2. When did they start using Stevenson screens in remote places such as central Africa and the Amazon?
3. Why does GISS use pre 1910 Austrailan Temps?
Oops, seems like a lot of people have picked up on this. Sorry for the repetition.
Maybe the folks running these networks of thermometers should work in a brewery.
After losing a few batches of beer, and maybe their job, they would appreciate there is nothing simple about getting accurate temperature readings.
See http://www.mosi.org.uk/media/33871137/jamesprescottjoule.pdf
I stopped reading after this.
Keith Sketchley says:
March 15, 2012 at 1:23 pm
“The Stevenson screen was first introduced to Australia in the 1880s and was installed everywhere, with a few exceptions, by 1910. ”
http://www.abs.gov.au/ausstats/ABS@.nsf/Previousproducts/1301.0Feature%20Article22005?opendocument&tabname=Summary&prodno=1301.0&issue=2005&num=&view=
Temperatures recorded to +/- .5 degrees :
http://www.srh.noaa.gov/ohx/dad/coop/EQUIPMENT.pdf page 11
I can’t imagine that Australia would be any better than the US surface station report showing that 90% of stations have greater than 1% error. CRU is even worse than BOM, reporting temperatures to a 3 decimal place accuracy. To me it seems that if I held a forefinger in the air and estimated a sufficient number of readings, I too could claim a 3 decimal place accuracy for my forefinger as a temperature sensor.
ThePowerofX says:
For shots and goggles, I checked out the Google cache of your site, named My dog is more sceptical than you.
I found much moaning about the shutting down of pirate sites, this screenshot captioned “A newbie complains” on a post titled Easy when you know how with text “More illegal files, more links, more money.” thus the obvious conclusion is you’re poking fun at someone who’s not making enough off of pirated illegal content.
And the obvious pride of your site, the top post Helping visitors learn, which is composed of well-linked choice quotes from Anthony Watts and many WUWT guest contributors, which highlights the wide-ranging backgrounds and diversity of thought available on WUWT, thus admirably showing the high quality of true skepticism on WUWT despite the harsh criticisms you frequently level against WUWT in your comments here.
Indeed, since the text is just quotes from WUWT, I heartily recommend that Anthony repost it here on WUWT so others may bear witness to this assembled proof of the strength of the skepticism here on WUWT, with appropriate credit to ThePowerofX for putting it together. Every reader here should see it, the compilation is impressive.
Good job, lad or lass. Your work is appreciated here.
Most australians think that the csiro are a bunch of gooses the gillard gov,t controls them it is all about the carbon tax we will be hit with in july , the csiro must report what gillard is preaching or NO grants. a bloody disgrace
I stopped reading after this.
“Warmers quit reading after anything the least bit contradictory to their religious beliefs.”
Sorry, but I have lost count of how many times I have read people of this blog state “I topped reading after they said “we used a model””..
We can’t have it both ways, leave that to the warmists.
You gotta love computers.
They can give you a trend accurate to 1/100 of a degree, using data hand collected from instruments that are only accurate to 1 degree.
Once again it getting the basic stuff wrong in this case accuracy of taken measurements, that ‘defines’ climate science. And once again you have to say this approach would not be accepted coming from an undergraduate but its seen has ‘fine’ by those claiming the robe of ‘expert’
@ ThePowerofX
You probably stopped after reading the word INDEPENDENT
So this is the same BOM that did the due diligence on the recent NIWA temp record in NZ. Wow, we KIWIs are so lucky. Lucky, lucky, lucky.
Tez says:
March 15, 2012 at 12:41 pm
Is this the same Australian BOM that some years ago advised the Government that prolonged drought would be the norm in Queensland? This advice led them to build a $6 Billion desalination plant which was never used and has now been mothballed.
If you cant trust their advice what is their point?
=======================================================================
The Tugun Desal Plant cost $1.2 billion, not $6 billion as you state.
And you’re complaining the BOM are rounding figures up.
MarkW says:
March 15, 2012 at 12:34 pm
“I remember an old story from Boeing, back in the slide rule days. ”
I haven’t heard of that app. Where can I download it?
These guys did very good job. Maybe they ought to try to publish it as a paper? It definitely deserves attention but I’m afriad it won’t get enough as a blog post…
Bill says:
March 15, 2012 at 11:35 am
It’s not clear to me why rounding would always bias up.
1 – 4 round down
5 – 9 round up
That’s 4 rounding down, 5 rounding up. On average, rounding will result in an increase.
I remember an old story from Boeing, back in the slide rule days. They recognized this bias and added an extra rule. If the whole number was even, then 5 rounds down. If it’s odd, it rounds up.
4.5 rounds to 4. 5.5 rounds to 6.
——————————————————————————————————————–
I think you’ll find there is a bit of a misunderstanding in your idea and any alleged Boeing “stories” are likely to be apocrophal at least. 0.000001 rounds to 0, 0.999999 rounds to 1. 0.499999 rounds to 0 and 0.5 rounds to 1. No need for extra rules and no bias introduced. Rounding produces NO bias, truncation does.
ThePowerofX says:
March 15, 2012 at 11:55 am
I stopped reading after this.
For some reason the first thing that popped into my head when I read this was was a small child with his eye’s closed and his fingers in his ears, loudly chanting “I can’t hear you! It’s still warming!” over and over again.
Somewhat insulting to Australian Observers of which I was one for 30 years, and over the last few years in charge of Townsville station quality control. In my last year our station was the best performing in Queensland with audited errors showing NIL ERRORS – the only such one in the State. I also Instructed for a period before and after 1972. And at no time were Observers taught nor did they in the Field – round upwards to whole degree. It was to the nearest tenth before and after metrification. If there was a need to round it was done to the nearest odd tenth, for instance 7.55 deg went to 7.5 deg and 10.05 deg went to 10.1 deg.
Any rounding that may have occurred would have been in porocessing data much further down the track, if at all.
I do agree on siting problems however. But here BoM was subjected to an ever decreasing Federal Budget as it was a Federal Dept to which lip service was paid and bounced from Dept to Dept each election. Funding and inspection of correct sitings took a back seat compared to automation and building up Head Office needs.
So Mr Watt, you owe Australian Observers a huge apology for your statement and await yuour writtena apology…..
[REPLY: No offense was intended to the mainly voluntary observers in Australia. People who have taken on this sort of civic-minded responsibility are to be applauded. The facts, however, are what they are. Take it up with the people who did the research. If anything, apologies are owed to those observers from CSIRO and others, perhaps like yourself, who are using the efforts of their work and dedication in areas they were never intended to address in the first place. If there is a “fault”, it does not exist among the observers. -REP]
Dr Burns says: at March 15, 2012 at 2:08 pm
” . . . . To me it seems that if I held a forefinger in the air and estimated a sufficient number of readings, I too could claim a 3 decimal place accuracy for my forefinger as a temperature sensor.”
Using all ten digits, for the same number of readings, would, perchance, allow four decimal places. How accurate is that!
[Is that ‘sarc’ or just slightly cynical about the measuring of temperature – or anything else]?
I feel that there is a culture that’They’ are manipulating ‘us’ – not just in climate science, but in life, and many of the [once legitimately called] professions, too.
Thanks indeed to Dr. Burns for is incisive image that I am here allowed to enhance a little.
Smile when you say that, junior. My Pickett N4-T sits in the top right-hand drawer of my desk at work, just in case.
power of X
Shoddy temperture recording,inaccurate,unreliable
funny, i started to read when i saw this….
Jason Blair says
When you round data off, its usually up, otherwise you call it truncating data.
——-
No. Rounding is supposed to be done in an unbiased way.
Personally I wouldn’t trust JoNova’s fan base to be able to add up a shopping list without stuffing up.
The truncation issue will only have an effect on the trend under certain conditions. I am betting the effect is minor.
The data is being strected beyond its capabilities because it was never the intention that data collected in these global stations would be used for assessing global temperatures to one or two decimal places. They were never designed to be that accurate, no one was that troubled when the stations were introduced.
It is a fact, but one which the warmists do not like acknowledging, that given the margin of error, we do not know whether today it is warmer than it was in the 1880s or 1930s. That is not to say that we cannot be reasonably certain that there has been some warming this past century but in absolute terms, we do not know for sure that it is warmer today than back in the 1880s and 1930s. This, of course, is of utmost materiality to the assessment of natural variation and the ranges that that may encompass.
The only quality data set is satellite data and even that has issues.
Note to all who contemplate warming trends caused by rounding:
Rounding does _not_ introduce false trends regardless on rounding method (including truncating or always rounding up) as long as it is done consistently for all values and as long as the value range is large enough. The second is true for temperature record. The problem is the first one – the same station may have reported precise or rounded numbers completely irregularly, based on current mood of the observer. The frequency of rounded numbers was changing over time (including over time of day) and rounding generally diminishes towards the current time when most of stations are automatic and report precise numbers. It’s the changes of rounding methods over time which may introduce false trends and the nature of the data does not even allow us estimate how big the error is. Not only we don’t know how exactly were the data rounded, it’s even problem for most sites to tell exactly which measurements are rounded and which are not.
Go read the original article, it’s all in there and it is definitely worth reading!
LazyTeenager says:
March 15, 2012 at 3:34 pm
This is an example of dramatic irony, right?
yamaka says at March 15, 2012 at 3:05 pm
I think you’ll find there is a bit of a misunderstanding in your idea and any alleged Boeing “stories” are likely to be apocrophal . . .
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
I work at a Boeing facility (that makes helicopters), and I can tell you I have seen the rounding issue described – BUT as a famous newsman used to say, there is ” . . the REST of the Story . . ” .
The rounding procedure is mentioned in a spec about measuring hardness of metals. The equipment gives a reading to 1 decimal place; for example, 43.1 or 44.6 , and so on. If a number of these readings were to be rounded to the nearest whole number, then four of the decimal point numbers (1 – 4) get rounded down, but 5 decimal point numbers (5-9) get rounded up. This would bias the resulting data upward slightly. That is why the ‘rule’ of if the whole number is even, rounded the ‘.5 ‘ down, and if the whole number is odd, round the ‘.5′ up.
Now in reality, it may not really make any significant difference; and I can’t answer why the readings with the decimal points weren’t added together and averaged in the first place, and then the final number rounded ( I am not and work and don’t have the spec to review). OR if the procedure is used in other specifications, or with more than one number behind the decimal point. The main point is addressing the comment of the ’rounding rule’ – that it is there and I have seen it.
Note that regular WUWT commenter Geoff Sherrington was part of the unpaid group of volunteers (dang, that Big Oil cheque seems to have gone astray in the mail yet again) who put this study together over many hundreds of hours of their own time and using their own money. Hopefully he will be along at some stage to clarify any questions people raise.
Australian temperature records shoddy, inaccurate, unreliable
Will BEST and HadCRUT4 use these data? And what should we think if they show that 2010 was warmer than 1998?
Slightly off topic, but a recent release in Australia suggests that oceanic temperatures around Australia are at “a record high”. Given the paucity of quality daya onshore I wondered how they meausured the oceanic temperature in 1973/4. Two blokes and a tinnie? Equally I note that NOAA has oceanic heat content records running back to 1955. I know you Americans are a clever bunch, but how was that done?
Prevaricator.
Warwick Hughes brings a bit of perspective to the BOM’s recent alarmist spin concerning record heatwaves in Perth through the summer, March period.
No doubt about it, here in Australia the BOM and the CSIRO provide the best climate science a government can buy!
I would also like to point out that vast swathes of rural Australia have been cleared for farming between 1910 (when the measurements started) and the present. Even those in uncleared areas have been subject to significant grazing and are well recognised as being under vegetated compared with the prisitne condition. The cooling effect of vegetation is well understood and this alone could have produced the half a degree of warming that the BoM and CSIRO has identified. And I’m not even joking – the report identifies significant errors in the results from Katanning Western Australia. For the sake of History Katanning was gazetted as a town in 1898, just 12 years before the BoM records are supposed to have started. Today, Kataning is in the middle of the Wheatbelt and is subject to broadscale wheat cropping. For those of you that don’t know – in WA wheat is grown in the winter and harvested just before Christmas leaving the land cleared during the hottest time of the year.
Rounding up from 5 is correct because 0 (zero) is also a number, and therefore there are 5 numbers below 5 and 5 numbers from 5 to 9.
garymount says:
Rounding up from 5 is correct because 0 (zero) is also a number, and therefore there are 5 numbers below 5 and 5 numbers from 5 to 9.
**************************
But 0 (zero) is never rounded and therein lies the problem.
Here’s why:
1.0 + 1.1 + 1.2 + 1.3 + 1.4 + 1.5 + 1.6 + 1.7 + 1.8 + 1.9 = 14.5 or avg = 1.45
Rounded
1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 2.0 + 2.0 + 2.0 + 2.0 + 2.0 = 15 or avg 1.50
Beware of stories such as this preceding the dumping of older data.
Very recenly, many Australian temperature records available on the internet have been cut off to now only go back to 1950, wheras previously they went right back to the early 1900s or late 1800s.
The very interesting thing about that is that all the current Australian record high temperature readings were made in the 1940s during the WWII drought. (and yes, Mabel, it was hotter than today!)
Reg Nelson says:
March 15, 2012 at 6:43 pm
1.0 + 1.1 + 1.2 + 1.3 + 1.4 + 1.5 + 1.6 + 1.7 + 1.8 + 1.9 = 14.5 or avg = 1.45
Rounded
1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 2.0 + 2.0 + 2.0 + 2.0 + 2.0 = 15 or avg 1.50
——
You then have to round 1.45 to 1.5 because your 1.0 etc. has implied errors, showing only accuracy to the one tenth, you can not have the hundredths column in your avg. result. Therefore both rounded and unrounded calculations of their respective average are identically 1.5, ergo I am correct.
Easy there, unless you can somehow disabuse me of the notion that IR spectroscopy rests, no, depends on the various absorption ‘bands’ of various gases like CO2, H2O, CH4 etc.
.
The odd-even rounding is a standard method in psychometric and academic achievement standardized testing. For the very reason stated above. Simple rounding rules result in artificially inflated data. Anyone with a graduate level stats class under their belt would understand this. But I have heard that our Ivy league climate scientists are not versed in graduate level statistics.
Non-conforming material (like ‘bad thermometers’ not meeting a particular spec they were purchased against) is one thing; proper specification of material (or test apparatus, including thermometers) is another …
The simple answer might have been to create a ‘cal’ chart for each of a required number of so-called ‘inaccurate’ thermometers, if their repeatability was good (could be assured) against a more expensive ‘reference’ thermometer.
I have seen ‘Lab’ personnel on occasion with surprisingly unrealistic expectations of the equipment they use …
.
garymount says:
You then have to round 1.45 to 1.5
****
No, you don’t. 1.45 is the actual average measurement, 1.5 is the rounded average number which is always higher. Therefore given a random distribution, the rounded number will be greater.
If you round the total average of 1.45 to 1.50, you introduce 0.5 degrees of warming where there was none.
Case uing round down instead of round up at the 5:
No rounding, until after the calculations
1.0 + 1.1 + 1.2 + 1.3 + 1.4 + 1.5 + 1.6 + 1.7 + 1.8 + 1.9 = 14.5
Round to proper significant digits, using round down at 5
= 14
Avg = 1.45, after round = 1.4
1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 2.0 + 2.0 + 2.0 + 2.0 = 14
Avg = 1.4
My conclusion, consistent use of rounding, not mixing the varieties of rounding one is using, seems to be proper technique.
From Kasuha on March 15, 2012 at 4:06 pm:
I decided to check if rounding can cause a bias. I started with °F rounded to an integer, as it was mentioned above this was apparently happening with strings of XX.0 numbers being reported, then converted to °C with rounding to one place, then considering the effect mentioned by Anthony Watts at the end of the piece I converted back to °F with rounding to one place. Then the differences in °F readings were calculated and summed. This was done in a spreadsheet with 2000 random numbers between -40 and 120 as the starting values, which I figured was a good large range for ground level Fahrenheit values. If rounding has no effect, all the differences should sum to zero.
OpenOffice 2.4.1, Calc (the spreadsheet). Generating new random values was done by highlighting all of them, then filling downward. As this wouldn’t change the first value, an extra row at the top held a starting random number using the same equation, this cell was highlighted with the rest.
Layout: 1st row: column descriptions; 2nd row: starting random value in 2nd column; 3rd row: start of values. 1st column starting at R3, numbers 1 to 2000.
Equations:
2nd column, Initial °F value, rounded to integer, range -40 to 120:
=ROUND(RANDBETWEEN(-400;1200)/10;0)
3rd column, Conversion to °C, round to one place decimal:
=ROUND((B3-32)*5/9;1)
4th column, Convert back to °F, round to one place decimal:
=ROUND(C3*9/5+32;1)
5th column, Compute difference between last and first °F:
=D3-B3
Sum differences:
=SUM(E2:E2001)
Starting random value 21.
Run, difference sum
1, +2.8; 2, -0.6; 3, +1.9; 4, -2.7; 5, +1.7
6, +1.9; 7, +2.3; 8, -1.8; 9, -2.1; 10, -4.7
11, -2.4; 12, +5.0; 13, +1.5; 14, +0.7; 15, +3.1
16, +1.5; 17, +1.4; 18, +6.4; 19, +2.9; 20, +2.6
Average difference of 20 runs:
1.07, round to 1.1°F
On individual runs, a positive bias was introduced 13 of 20 times. An individual run is comparable to a fixed amount of temperature records. Overall there was an average bias over twenty runs.
I expanded to 10,000 rows of values and ran 10 runs, using 3 different starting random values. Reseeding the starting value done by copying its equation and pasting it back into the same cell.
Starting random #: run results
# + runs, # – runs, 10 run average
89: +6.1, +7.8, -2.0, +9.3, +2.6, +13.4, +12.8, +7.8, -1.0, +17.2
+8, -2, +7.4
116: +2.8, +5.5, +3.6, +9.4, -0.9, +20.1, -5.6, +21.7, +14.4, +17.8
+8, -2, +8.9
15: +10.1, +11.8, -5.0, +8.8, -0.9, -6.6, +6.9, +6.0, -6.6, +19.7
+6, -4, +3.4
With consistent use of the same rounding method over a large range of expected values, a significant persistent positive bias is noted. As seen on individual runs, which are comparable to a fixed series of temperature records, the amount of total bias can be quite large.
Thus I conclude that rounding of temperature records can introduce bias thus rounding should be avoided.
Sorry, typo. s/b 0.05 degrees of warming.
LazyTeenager says:
March 15, 2012 at 3:34 pm
“Personally I wouldn’t trust JoNova’s fan base to be able to add up a shopping list without stuffing up.”
============
Very funny, which is not to say I agree.
Who would you trust ?
ThePowerofX says:
March 15, 2012 at 11:55 am
“A team of independent engineers, scientists, statisticians and data analysts (brought together by the joannenova blog)”
I stopped reading after this.
______________
Let’s see… weren’t you one of those people who always told everyone else to ‘think outside the box’? (when that phrase was hip and cool)
Reg Nelson says:
March 15, 2012 at 7:43 pm
If you round the total average of 1.45 to 1.50, you introduce 0.5 degrees of warming where there was none.
—————
The problem in all this is that the error margins must be shown. You say that you have exactly 1.45. But you don’t have exactly that number. Only if you are counting individual objects such as I have exactly 5 brothers and sisters. The more numbers you calculate with that have error margins, the errors keep accumulating, the spread gets larger, the accuracy decreases, and you must show this when rounding. It is not proper to round then reassemble pretending you still have the same accuracy as you had before rounding. And I believe that is what we are complaining about what is taking place here in climatology.
I have spent aver 2,000 hours in recent years studying calculus, and I have to admit, I haven’t spent much time on whether or not bias is introduced to a warming effect when rounding numbers, but I still remain skeptical, and do so because the addition of showing the newly added error margins should take care of any bias that may be introduced.
I think I need new glasses, aver should be over, in case anyone has noticed in my last comment. Also, apparently aver is a proper word according to my autocorrect spelling software I am using when composing comments for this blog.
Phillip O’Neill says: March 15, 2012 at 3:15 pm
[ … ] So Mr Watt, you owe Australian Observers a huge apology for your statement and await yuour writtena apology…..
=============================
Not so Phil. I’m an Aussie and I find this episode embarrassing given how we are quick to remind anybody about how clever we are … if I ever hear “world’s best practice ” again I will clock that person.
KeithH at JoNova, March 16, 2012 at 8:57 am, sleuthed a bit following a comment from me and found the emails that I referred to”
“Streetcred @ 5. This may be what you remember but if not, it sure highlights the point!
“I recall reading recently, one of the ClimateGate 2.0 emails regarding records of Australian temperatures from BoM. If my memory serves me correctly, the email lamented the poor quality of the Australian records.”
Quote:
■FOIA\documents\HARRY_READ_ME.txtgetting seriously fed up with the state of the Australian data. so many new stations have been introduced, so many false references.. so many changes that aren’t documented. Every time a cloud forms I’m presented with a bewildering selection of similar-sounding sites, some with references, some with WMO codes, and some with both. And if I look up the station metadata with one of the local references, chances are the WMO code will be wrong (another station will have it) and the lat/lon will be wrong too.
■FOIA\documents\HARRY_READ_ME.txtI am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that’s the case? Aarrggghhh! There truly is no end in sight!
■FOIA\documents\HARRY_READ_ME.txt OH F..K THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done I’m hitting yet another problem that’s based on the hopeless state of our databases. There is no uniform data integrity, it’s just a catalogue of issues that continues to grow as they’re found.
http://wattsupwiththat.com/2009/11/25/climategate-hide-the-decline-codified/ “
garymount says:
The problem in all this is that the error margins must be shown. You say that you have exactly 1.45. But you don’t have exactly that number.
*****
Error margins are an entirely different issue. If you have unreliable data, you have unreliable data. Rounding unreliable data still leaves you with unreliable data. It doesn’t correct the problem.
If you have unreliable data you can’t make reliable projections (or predictions) based on the data.
_Jim says:
March 15, 2012 at 7:18 pm
Ian W says March 15, 2012 at 1:32 pm
…
These climatologists come from the large group who believe that so called ‘green house gases trap temperature;
Easy there, unless you can somehow disabuse me of the notion that IR spectroscopy rests, no, depends on the various absorption ‘bands’ of various gases like CO2, H2O, CH4 etc.
Jim, I see you are one of the large group believing that green house gases trap temperature.
I have no need to disabuse you of the notion that IR spectroscopy depends on the various ‘absorption’ bands of IR. What you do need to be disabused of is being able to measure the quantity of IR (that is heat ) absorbed by the atmosphere using a thermometer. The heat capacity of the atmosphere varies considerably dependent on its humidity. See http://www.engineeringtoolbox.com/enthalpy-moist-air-d_683.html
If you don’t know the humidity of the atmosphere at the time you measure the atmospheric temperature you have little idea of the heat that has been absorbed by the gases in it. If you look at the examples given in the URL above, and do a little calculation, you will see that a Louisiana Bayou at 100% humidity after an afternoon storm with the temperature at 78F will hold twice as much heat energy as the air in the Arizona desert at close to zero humidity but at 100F. Whereas you would be happily saying the opposite just measuring temperature – and you would be wrong.
As you say the entire claim of ‘global warming’ is that heat energy is absorbed by the atmosphere… but then climatologists measure atmospheric temperature and not heat content. It is the incorrect metric for measuring heat. If climatologists were really serious they would be measuring the ocean temperatures because unlike the atmosphere there is little variance in enthalpy in the oceans.
Therefore arguing about the level of precision and accuracy of the incorrect metric is a waste of effort but that is the kind of debate that climatologists get involved in. The money spent on these misguided efforts would be far better spent elsewhere on real problems.
OK, I have figured it out with the rounding issue. So long as the number of recordings below x.5 remain relatively the same as the number of recordings of x.5 to x.9 through time, i.e. the number of recordings show a consistent randomness or spread throughout the time series, then there will be no warming bias trend. The temperatures throughout the series might be biased into warmer temperatures, but as long as the same rounding algorithm is used throughout the time series, there will be no magical increase in temperatures just because you are increasing the number of readings, and rounding’s through time. Just as long as you use the same rounding method in the early part of the series as you do in the later part, and actually throughout the series. Once again, the temperatures might be higher than they should be, but there will be no warming trend bias introduced.
The article that this thread is about clearly shows that they found that the type of rounding done had indeed changed from earlier in the time series from the rounding done in the later part of the time series which did introduce a warming bias. If a consistent rounding was used throughout, there would not be an introduced warming bias.
Kasuha at 3.02 pm hits the nail on the head. The Bureau of Meteorology (BOM) data probably does favour an increase in Australian temperatures and it certainly needs careful scrutiny. There is (and I live in Australia) an attitude in government, the CSIRO (premier scientific body in Australia)and the BOM to disregard any evidence that suggests global warming is entirely due to human burning of fossil fuels. Until results such as these are published in a peer reviewed paper (possibly not E&E which the climate scientists denigrate) and hit the MSM, no amount of publicity in internet blogs will have the slightest impact
Hopefully the final word, from me on rounding:
Unrounded temperature series example:
1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 = 14.5
Average = 1.45
Now rounded calculations follow of the above series:
1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 = 15
Average : 1.5
In time more recordings are taken, with the same distribution, i.e. the average temperature has not changed:
No rounding:
1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 + 1.4 + 1.5 = 29.0
Average: 29/20=1.45
Rounding the above series:
1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 + 1.0 + 2.0 = 30
Average: 30/20=1.5
—————-
As you can see, whether you use rounding or no rounding, no warming bias is introduced.
I hope I rest my case.
garymount;
the issue is “trends”, as you point out, since that’s the basis of “warming or not”, by definition. However, the absolute errors can’t be so easily sidestepped, since necessarily huge collections of measurements from many global (and national) locations and environments are being merged, averaged, compared, subtracted, etc. Your “consistency” constraint is un-meetable. So the only hope is best absolute accuracy available across and within sites and regions. And that requirement is massively violated.
_________
Ian, your point can also be discussed in terms of “extensive/intensive” variables. Intensive variables like temperature and density can’t be summed or averaged; they are, however, the outcome of other (extensive) variables which can be summed and averaged. Temperature requires heat, humidity, and density, at least, be known for both locations to be added or averaged, etc. It is meaningless, e.g., to average the temperature of a cu’ of air and a cu’ of water. Or a cu’ of air on a mountain top and a cu’ of air at sea level. Yet the latter is done routinely by climastrology.
Philip O’Neill:
As one of the authors of the report, I do apologize if any current or past observers feel this is a ciriticism of them. Far from it, it is a criticism of BOM management fair and square, and of course various Commonwealth governments for not giving them the resources to do the job properly. It is
laughable to expect untrained part time observers to read Fahrenheit thermometers to the nearest tenth in any weather conditions. Townsville, not being one of the Annual HQ sites, was not part of the survey. If it was, from what you say I’m sure it would be near the top rankings- as are other
Regional Offices, Met Offices, and airports mostly. However, the worst 10% of sites we looked at had rounded Fahrenheit data very probably more than 89% of the time. These were mostly country post offices and a couple of lighthouses. Very probably half of all Fahrenheit data surveyed were
rounded, but there were less than 10% of extra unexpected whole degrees in the Celsius era. (I say ‘very probably’ because the conversion from F to C to F introduces its own uncertainty which we allowed to be +/- 5%). This means that if truncation occurred, it would cause an artificial warming trend.
This report, which I urge you to read for a description of our methods and findings, is not a criticism of observers but is a criticism of current Bureau quality control, homogenisation methods, and climate analyses.
I’m sure other members of our team would reply to you in similar vein.
Ken Stewart
Phillip O’Neill wrote:
Somewhat insulting to Australian Observers of which I was one for 30 years, and over the last few years in charge of Townsville station quality control. In my last year our station was the best performing in Queensland with audited errors showing NIL ERRORS – the only such one in the State. I also Instructed for a period before and after 1972. And at no time were Observers taught nor did they in the Field – round upwards to whole degree. It was to the nearest tenth before and after metrification. If there was a need to round it was done to the nearest odd tenth, for instance 7.55 deg went to 7.5 deg and 10.05 deg went to 10.1 deg.
Any rounding that may have occurred would have been in porocessing data much further down the track, if at all.
As a co-author of the audit, I can assure you that we in no way intended offence against the observers who recorded temperatures on behalf of the BoM.
The Townsville station does indeed have an excellent record in logging accurate temperatures and you should be congratulated for your precise quality control of that station. The available BoM raw minima data for Townsville Aero 32040 from the first recording 19 Oct 1940 to 31 Dec 2011 shows that in the Celsius era since September 1972 there were no blocks at all of 10 consecutive days or more with temps rounded to .0 or .5. Excellent and I wish that was the case in all Australian stations as presented on the BoM’s national raw temp database.
In the Fahrenheit era before September 1972, 14.8% of all Townsville temperatures were rounded to .0F. That is, 1,713 out of 11,610. However, the mathematical bias of rounding revealed by proportions of .1F and .9F shows this is almost certainly an underestimate:
F degrees
0.0 1713
0.1 1383
0.2 1112
0.3 1087
0.4 987
0.5 911
0.6 1002
0.7 1012
0.8 1133
0.9 1270
.9+.0+.1 = 4,366 which is 37.6% of the total. This infers a .0F proportion slightly above 20% – only twice the expected proportion of 10%. Phillip, as you say, Townsville has one of the most accurate historical records in Queensland (possibly in Australia) – but it seems some of your trainees still occasionally rounded inaccurately.
You contend that the F data recorded by observers was in fact almost all accurate to the tenth degree, but the BoM may have manipulated it to round as .0F when converting to C. I hope you’re wrong because that would be a final nail in the coffin of BoM credibility.
Since September 1972 within which there is no scale conversion, the record shows 12% at .0C and 23.3% at .0C+.5.C This is within the margin of error and although hinting at a bit of rounding it is not significant.
The evidence within our audit shows that since 1972 almost all the C rounding in Australian weather stations has been up or down (to the odd whole if .5 in accordance with BoM protocol) rather than truncated down, and thus has little influence on the long-term rounded or precise averages since Celsius metrication.
Our audit does not claim pre-72 Fahrenheit temperatures were truncated rather than rounded but does provide evidence from the data that down rounding was more likely than up, and calculates the effect based on different possible proportions.
The audit does not state a definitive impact on temperature trends since 1910 but again based on the available data displayed, speculates that there may have been artificial warming between .1 and .4C over 100 years. However, the audit does state and prove that a large proportion of the raw temperatures upon which the HQ series is based is not accurate.
An accurate impact may be estimated through a thorough professional and independent audit of all BoM records.
Again, Phillip, no offence was meant to yourself or other observers who may have rounded or truncated temperatures. It would hardly be a sin if some (many, based on the audit results) thought 72.7, for example, is sort of correct as 72 but 73 is obviously wrong – particularly if it’s raining, stinking hot or there’s a mob of sheep that needs to be put on the trucks. That’s honest human nature and we are not criticising the practice itself. However, the data cannot be ignored.
The Excel macros built to conduct this audit are freely available at http://www.waclimate.net/round/rounded-background.html if anyone wants to replicate above or any other Australian station.
Brian H says:
March 16, 2012 at 12:41 am
Your “consistency” constraint is un-meetable.
———
As long as the ratio of the number of recordings below the .5 remain roughly equal to those .5 and above from past measurements, then my “consistency” will be met, within a close enough tolerance which should get even closer with more measurements. For example if the ratio of below .5 to .5 and above remains say 2 to 3 (any ratio will do), then there will be no introduced bias.
There certainly can be a bias of a warmer temperature, but your whole temperature set will start out with this little bit too warm than actual when using rounding than you would have produced from not rounding, but unless you start to have an increasing ratio of numbers in the .5 and above range compared to the below .5 range, you will not get a warming bias. That is the consistency I am talking about. It is just as likely a chance to have more recordings compared to previous measurements in the below .5 range and see a cooling bias set in, even when you use a rounding that records temperatures warmer than they actually are.
One would have to deliberately or I suppose some how unintentionally be recording an ever increasing quantity of temperatures in the range that gets rounded up in order for a warming bias to become present in the measurements as time passes, for the rounding to cause a warming bias. This could of course happen by chance, but seems higly unlikely, and by chance a cooling bias could also take place, but again, highly unlikely (with sufficient quantities of measurements, which was mentioned very early on in this thread).
So, once again, it is the ratio that must remain consistent for no bias to be introduce, to either warming or cooling, no matter which rounding method is used.
As a chemistry lab instructor at the University of Texas many years ago I found one of the most difficult concepts for the students, especially the non-majors, was “significant digits.” When asked to report the results of a calculation they would routinely report every last digit their calculators would display. Thus a solution of 1 g of salt in 70 ml would be reported to have a concentration of 14.28571428 g/l.
While I successfully enlightened many, I am sure many others, including some who now work in fields considered ‘scientific’, have never grasped the concept.
Not all numerical uncertainties are statistical.
Science is uncertain – everywhere.
“Consensus’ is a political concept.
Phillip O’Neill, I can understand your chagrin as one of the professional observers caught up in this audit of all High Quality sites in Australia. I was aware that this would happen and was hoping that some observers would come forward and comment and I thank you for that.
I did some observing myself as a schoolboy before 1972 at my amateur home weather site and put my own F readings through the program and was not surprised to see a bias towards some decimal fractions over others. I was particularly reluctant to record the .5 fraction. I was untrained but I’d call myself a diligent observer. I badly wanted to becoem a BOM observer but alas it did not happen.
What is so surprising is that before this year none of us even suspected there were these question marks about the quality of the observations generally. We knew there were some places of doubtful quality at some stage of their existence, but not that this extended to most stations.
As is often the case this study more or less happened by chance rather than design, with people making casual remarks and then others chipping in. I became involved when Chris said it took him two hours to work out the results for one station in Western Australia and I knew I could write a program to get the job done in less than a minute.
This was an audit waiting to happen.
Ian Hill
yamaka says:
March 15, 2012 at 3:05 pm
I think you’ll find there is a bit of a misunderstanding in your idea and any alleged Boeing “stories” are likely to be apocrophal at least. 0.000001 rounds to 0, 0.999999 rounds to 1. 0.499999 rounds to 0 and 0.5 rounds to 1. No need for extra rules and no bias introduced. Rounding produces NO bias, truncation does.
—-
Where did you get the idea that people using slide rules would calculate out to 6 digits?
Even in your example, there is still a bias, the magnitude of the bias decreases as the number of digits of accuracy increases.
LazyTeenager says:
March 15, 2012 at 3:34 pm
Personally I wouldn’t trust JoNova’s fan base to be able to add up a shopping list without stuffing up.
The truncation issue will only have an effect on the trend under certain conditions. I am betting the effect is minor.
——
Personally, I wouldn’t trust you to go to the bathroom without supervision.
Truncation vs. not truncation always produces a bias.
The effect is minor, then again the claimed warming is als minor, and off the same magnitude.
Brian H says:
March 16, 2012 at 12:41 am
……..Ian, your point can also be discussed in terms of “extensive/intensive” variables. Intensive variables like temperature and density can’t be summed or averaged; they are, however, the outcome of other (extensive) variables which can be summed and averaged. Temperature requires heat, humidity, and density, at least, be known for both locations to be added or averaged, etc. It is meaningless, e.g., to average the temperature of a cu’ of air and a cu’ of water. Or a cu’ of air on a mountain top and a cu’ of air at sea level. Yet the latter is done routinely by climastrology.
But if you look at this thread the majority of posters have been drawn into the ‘climastrology’ argument. Or to use the logic aphorism “they are searching for their lost keys under the lamp post as its lighter there”.
I thought this was a scientific blog 😉
We also see the same in satellite metrics where the microwave sounders are used to look at temperature – but if you look at GOES EAST imagery you can see the huge amounts of IR being radiated from water changing state in fronts and storms – and the over quoted Stefan-Boltzmann formula has no relevance to latent heat release which happens on change of state at any temperature.
What should happen is that _every_ time a climate scientist starts discussing the average global temperatures they should be stopped and told that atmospheric temperature is the incorrect metric for measuring heat absorption; and it is heat absorption and imbalance that they claim to be the problem. They should provide a value in Kilo-Joules and perhaps the break down of energy budgets from ERBE and CERES. Don’t hold your breath waiting for this as if they were to use the correct metric there would be no ‘catastrophic values’ to trumpet.
garymount says:
March 15, 2012 at 6:04 pm
Rounding up from 5 is correct because 0 (zero) is also a number, and therefore there are 5 numbers below 5 and 5 numbers from 5 to 9.
—-
Zero doesn’t round. It stays the same.
garymount says:
March 15, 2012 at 7:15 pm
ergo I am correct.
—
On this sequence of numbers. Add a few more numbers to the series, then you stop being correct.
markx said @ March 15, 2012 at 6:58 pm
Not so. Some years ago, after hearing Phil Jones being interviewed by Robyn Williams on the Science Show I had occasion to check Australian record temperatures. Jones claimed that the recent spate of record highs was a clear indication of Global Warming. The majority of Australian record highs were in the 1920s, 30s and 40s, not the 1940s only.
I have hardly bothered listening to the Science Show since.
Further to my previous remark, markx is correct in noting that the records prior to 1957 appear to have been disappeared from the interweb. Based on the now missing records, I wrote the following on Monday 3 February 2003:
@MarkW
Please continue to read the comments. You will see where I acknowledge my mistake. However, there are no biased trends with any rounding method consistently used, as I have shown in my further comments.
johanna says: March 15, 2012 at 4:50 pm :Hopefully he (Geoff Sherrington) will be along at some stage to clarify any questions people raise.
Thank you, Johanna, Four points,
1. There is continuing work that will show more adverse consequences that could arise from the work so far reported.
2. If there are any questions, please direct them to the primary authors. They did the hard yards. I merely added an idea now and then. I would not like to misprepresent them by answering for them.
3. Collectively, with a few hundred man-years of combined experience, we would not have written the above if the conclusion was that there was no error of concern. Conversely, the more we dug, the more problems we found. They have not all been reported and indeed a number of people have not read them closely enough to understand them, as judged by comments here.
4. People are encouraged to conduct similar exercises in their respective countries.