Australia and ACORN-SAT

Since Australia is in the southern hemisphere, occasionally things can be upside down?

Juliar may have some time off now to go sort this out …. and she should have the funding too ….
…. in the money she lied about up-front.

Are they using regression to infill missing data from other stations (…and not following through with due diagnostics)? I found something like that while doing contract work many years ago. Of course there are many other possibilities…

Completely OT but I’m looking for the Monckton v rgbatduke discussion on atheism if anyone remembers which article that was. I would love to share it with a friend.

I wrote on this BoM report a few weeks ago
— http://donaitkin.com/dodgy-stuff-from-the-bureau-of-meteorology/ —
and am not surprised.

Australian Science and Research has completely lost its way and is now third world rate. I should know I did my PhD there in 1982. It used to be good about until 1985 when Keating and Dawkins started to destroy it. The results are plain to see today with scientist in BOM and the importation of really BAD scientists such as Lewy. Australian research has not produced anything of notable value for some time in the world context compared to the 40’s, to 70’s. Your typical scientist there these days are guys like Nick, Flannery ect you can work it out…..a result of the Keating Dawkins influence on higher education there (BTW I used to vote labor)..Its manufacturing base is about as shot as you can get with nearly all manufacturing going abroad (ie car industry).Australia will unfortunately have to stick to tourism, food production and entertainment, unless there are drastic changes which will not happen anyway. Maybe Abbot if elected (which I now doubt) could change things there LOL.

TimTheToolMan said @ June 29, 2013 at 12:40 am

Well according to your list my hometown of Hobart has 7 days where the maximum is less than the minimum and some days it sure feels that way.

Perhaps because it is that way. I live 40 minutes drive south of Hobart and it’s not unknown for the temperature at sunrise, usually the low point in any 24 hour period, to exceed the mid-afternoon temperature, usually the highest temperature in the day. Dunno about Wilson’s Prom, though. Fond memories of chasing bellbirds (and belle birds) in the bush there in the late 60s 🙂

Willis I question your term ”metadata”. I am a geologist and anything with meta in front means it has been subject to metamorphism ie., heat and pressure. So climate data subjected to heat and pressure?—- sounds about right in today’s political alarmist climate.

The BOM clicks on to a new “day” at 9am (presumably when the sleepyheads roll into the office). It is quite possible, but quite misleading, for minima to exceed maxima for a 24 hour period given this. All it takes is a fast moving weather system, of which we get plenty on this vast continent.
I guess whatever cutoff point they choose will introduce anomalies, but it also matters – in terms of real descriptions of the weather – that sunrise/sunset times vary by over four hours (cumulatively) where I live between seasons. In summer, at 9am the sun has been beating down for nearly four hours. In winter, it is just hitting the frost on the lawn with weak, slanting rays – although it has technically been risen for a couple of hours. I realise that it doesn’t affect the statistics if consistently applied, but it does provide a warped picture of actual temperature patterns throughout the day, across the seasons.
As for the BOM’s reliability, I hope that Geoff Sherrington and a few other hardy souls who have been working on the data for a long time on their own dime drop by to comment. All that rhetoric about state-of-the-art, peer reviewed, bla bla bla is just self-congratulatory nonsense. It is on a par with statements about the IPCC being a source of “gold plated” science. They have adjusted data (without explanation), changed goalposts, ignored evidence of obvious errors and campaigned relentlessly for CAGW for many years.

Yep. One thing is just about certain. If one finds that many totally blatant errors, there will be multiple numbers of other errors lurking which are not so obvious. Almost a law of nature! This is just the tip of the iceberg. The dataset is probably unfixable. Sad.

Oh great mayte, yeah o allright right mate….so says Nick Stokes. A O KAAAAAAAAY.
Because at 9am the high is yesterdays and the low is todays….eeerrrrrrrrr uhhhhhhhhhhh on only 917 records. Whoooowwhhaaaaattttt!?!?!?! So, the weather stations are then wrong for all of the other records?
Okay okay…what are they correct for then nick stokes….they are either correct only 917 times or they are completely incorrect. And, if they are only correct 917 times or they are completely incorrect why haven’t they been fixed yet?
Australian science mate. Gooooooday

Confusedandinfuriated says:
June 29, 2013 at 12:04 am
“They originally wanted to use “Cruel Summer” but Bananarama beat them to it. :)”
Fantastic! Best part is, I listened to the song 5 minutes ago! Crazy coincidence.

I worked at Yeelirrie Station, a million acre Pastoral Station in the middle of WA. Records and BOM data had been collected for the last 80 years. In the winter if 2012 we were regularly the coldest or one of in the whole of WA, nightly news would have to read out Yeelirrie minus whatever. BOM contacted us about halfway through winter and informed us that our 6am temp readings would no longer be required. To me this seemed odd, but I guess they did not want regular minus readings in the middle of WA? What effect does this have on calculating mean temps?

Nick Stokes says:
June 29, 2013 at 4:20 am
… They can’t re-do the readings.
——————————
They “redo” the readings every month. Making the past colder and the current temps warmer.
Everyone knows what a “day” is. If you miss a reading, it is missed. You do not make up the missed day with obviously incorrect data.

John Marshall
Meta is a prefix used in English (and other Greek-owing languages) to indicate a concept which is an abstraction from another concept, used to complete or add
Metadata is hence data about data

gaelan clark – there has been a whole team of Aussies (Kenskingdom above being one of them and Jo Nova another) that have been investigating the BoM and their methods. Prior to ACORN there was the HQ (High Quality) data yet when this team made an FOI request to BoM for the algorithms used to create the HQ data set the BoM initially ignored it but then replied that they no longer used the HQ set and now used the ACORN set so the algorithms weren’t necessary.
Such is the arrogance of the BoM senior management.

jeremyp99 said @ June 29, 2013 at 4:30 am

Well, maximum, minimum, husbands, wives … what’s in a word?

The genesis of this was the perversion of using a grammatical term, “gender”, as a synonym for “sex”. The genders are masculine, feminine and neuter; the sexes are male and female. Gender and sex don’t even map 1:1. No wonder confusion reigns…

JohnWho said @ June 29, 2013 at 5:11 am

I see Nick is here defending the indefensible once again. At least that is consistent. 🙂

So how would you suggest we go back and redo the readings? Enquiring minds…

Nick Stokes says: “They can’t re-do the readings.”
But they can adjust them. They can close stations that don’t agree with their agenda and only use those that do.
Such is the bias built into the current BoM. It’s disgraceful.
Lismore centre street has an unblemished record going back to 1907 – they closed it in 2003.
http://www.bom.gov.au/jsp/ncc/cdio/weatherData/av?p_display_type=dataGraph&p_stn_num=058037&p_nccObsCode=36&p_month=13
It’s pretty clear why they don’t use it in their current records.
similarly with Casino:
http://www.bom.gov.au/jsp/ncc/cdio/weatherData/av?p_display_type=dataGraph&p_stn_num=058063&p_nccObsCode=36&p_month=13

For Britain’s recent run of weather might I suggest ‘sullen summer’.

Kevin Rudd and Julia Gillard two politicians destroyed each and their party over implementing Carbon Tax.

Anthony,
I think it is time to start independent temperature/weather spotter program using the same digital devices exclusively from wattsupwiththat.
Could become largest independent automatic temperature recording reading system network and could start with one state or the USA
Need to find manufacturer that can make a weather station that can automatically send wireless readings at locations to a network you can create. Individuals can purchase the units through wattsupwiththat and hook up at their home or work place in a location that is not near Pavement or air-conditioning/ heating units or can allow some . Pictures of location of each weather station sensor must be recorded also.
Sound good?
Have s good weekend

DonK – Willis posted
“A panel of world-leading experts convened in Melbourne in 2011 to review the methods used in developing ACORN-SAT. It ranked the Bureau’s procedures and data analysis as amongst the best in the world.”
and this
“The Bureau of Meteorology’s climate data experts have carefully analysed the digitised data to create a consistent – or homogeneous – record of daily temperatures over the last 100 years.”

In principle, which situation is worse for good science?
A) have a dataset that contains errors
B) have a dataset where the errors are “fixed” and the origional problems unseen?
Data is data.
If the data has errors, we do not decrease uncertainty by adjusting the data.
I argue that the idea situation is to have
A) the dataset that shows the errors, and
C) a dataset that shows the fixes and how they were fixed.
The key is to realize that the uncertainty in (C) is greater than that of (A).
(A) has an error component.
(C) is not the subtraction of error from (A),
but the ADDITION of corrections which themselves have error.
Granted the corrections are highly correlated with the hypothesized error in (A). If you track down the source of each error to root cause and can reliably correct the error, such as a clear transposition of keypunched data entry, you can indeed reduce error in (C). But without an autopsy on each error, realistically the total error in (C) should be treated as greater than the error in (A).

what do the terms minimum and maximum mean?
are they defined by time of observation?
or are they defined by being the lowest and highest in any 24 hr period?

Tamino has a post “like candy from a baby” about the “angry” summer. Disparaging both Willis and Bob Tisdale. To my amazement he posted my response and responded as follows:
Gonzo: [Drawing any conclusions from the Oz land temp record is like trying to determine ocean heat content pre-ARGO. Sparse data and in many cases bad data ie….many of the older records were recorded in whole degrees or when they converted to celsius in 1972. Save for a few quality stations which amount to a regional effect the Ozzie data should be taken with a serious dose of doubt. Much ado about nothing. BTW how many state heat records were broken during the “angry” summer? Oh none!
[Response: Gonzo proves the point. He doesn’t like what the thermometer says, so his comment amounts to nothing more than calling the thermometer a liar. That’s what those in denial have to resort to. But wait, there’s more!]
BTW how many state heat records were broken during the “angry” summer? Oh none!
Tamino: [Response: Bonus points — Gonzo adds “moving the goal posts” to denying the facts. The relevant fact is that last summer, and especially January, was scorching hot in Australia — not that some state broke a heat record. Gonzo hopes that by pointing to one factoid which isn’t a record-breaker he can distract everyone from such facts as:
During this period, Australia registered the warmest September–March on record, the hottest summer on record, the hottest month on record and the hottest day on record.
A record was also set for the longest national scale heatwave.
Does Gonzo actually believe that just because no state heat record was broken, that will magically transform the hottest summer on record nationwide, the hottest month on record, the hottest day on record, and the longest national-scale heat wave, into “nothing at all unusual about the 2012 summer”? Not too bright.
Perhaps most important for those of us who are interested in the truth, Gonzo has denied the reality of Australia’s scorching hot summer in order to distract us all from the fact that summers like that are now more likely than they used to be, by a lot. He must distract everyone from that fact, because that’s the real point.]
To which I responded back with this:
You don’t have the stones to post this but here goes anyway. cheers
Your response is puzzling to me as I would call you the Mr T of cherry picking…..HEY FOOOO YOU’RE CHERRY PICKING!!!
Who “denied” OZ is hot? Not me. You do know Oz is the hottest continent yes? Have you been to Oz? I have many times surfing both east and west coasts. WA is efffn hot always has been ie .. it gets hot there.
Tamino: [During this period, Australia registered the warmest September–March on record, the hottest summer on record, the hottest month on record and the hottest day on record.] the hottest day on record? really? You sure about that mate?
” Whilst it is probable that remote areas of the Australian desert have seen extreme temperatures that have gone unrecorded, the outback Queensland town of Cloncurry originally held the record for the highest known temperature in the shade, at 53.1 °C (127.5 °F) on 16 January 1889. Cloncurry is a small town in northwest Queensland, Australia, about 770km west of Townsville.
The Cloncurry record was later removed from Australian records because it was measured using unsuitable equipment (that is, not in a Stevenson screen, which only became widespread in Australian usage after about 1910). According to the Australian Bureau of Meteorology, the current heat record is held by Oodnadatta, South Australia, 50.7 degrees Celsius, occurring on 2 January 1960.
The world heat record for consecutive days goes to Marble Bar in Western Australia, which recorded maximum temperatures equaling or over 37.8°C on 161 consecutive days, between
Concerning Cloncurry why would the BoM call a thermometer a liar.
Of course he only posted my response as follows:
[You don’t have the stones to post this but here goes anyway. cheers] Followed up with disparaging remarks………
Par for the course over their when in doubt attack the messenger.
REPLY: there’s no point in paying attention to Grant Foster aka “Tamino” his rants are irrelevant – Anthony

gnomish said on June 29, 2013 at 9:18 am:

what do the terms minimum and maximum mean?
are they defined by time of observation?
or are they defined by being the lowest and highest in any 24 hr period?

With daily reporting a 24 hour period is assumed, all measuring occurring on a single day. Although the daily measurements were rarely taken exactly at midnight, leading to adjustments.
Minimum is the coldest measured temperature during the measured period. This is normally expected in the early morning just before sunrise, after the ground has been cooling off all night. But with weather fronts moving in, and the evaporative cooling of wind and rain, the minimum could happen anytime.
Maximum is the hottest measured temperature. Presumably that happens in the afternoon, with the Sun warming up the ground. Which is a problem. If you take your daily measurements at 9AM, you must then assume the highest reading was actually from yesterday afternoon, so you mark it down as yesterday’s high. Even though the cold front passed through, your “high” is from the morning you took the measurement as yesterday was bitterly cold morning to night.
This gets adjusted later as quality control. The weather station in the deep valley reports a 23°C high on a sunny Tuesday, but stations within 100km on flat ground reported a similar high amount on a rainy Monday, therefore the valley was reported wrong and their high gets moved to Monday so the similar highs happened at the same time.
Since ideally all of the measurements should go from midnight to midnight, this brings about the Time of Observation (TOBS) adjustments. It is excessively argued exactly how important the TOBS adjustments are for an accurate record, with the “usual suspects” insistent how absolutely necessary they are, and will promptly point to some analysis one of them did that proves it as the temperature trends are much too low without it.
(I’m still waiting for a coherent brief explanation of what TOBS really is and how it’s calculated. As expected, normally we’re told to read some paper by Hansen or someone else who presides over a screwed-up temperature dataset.)
Of course for much of the world’s “raw temperatures” we’re lucky to have numbers reported for a day at all, let alone the observation times, so TOBS is basically meaningless except perhaps for countries like the US.
Then comes the real magic. The twenty minutes when there was a break in the clouds and the Sun shone directly on the thermometer shelter/housing, will be averaged together with the five hours of hurricane-strength winds that plastered 10cm of packed snow on the side of the shelter/housing, to determine the average temperature that day was sufficiently high to cause a significant amount of snow melt. Which is in agreement with the projections of the climate models thus is confirmation of (C)AGW theory.

Billy liar, that document is an interesting read. Makes you realize how many adjustments are being made and how many are potentially questionable.
This caught my attention,
Figure 10 shows an example of data flagged by this check, a minimum temperature of 20.6°C at
Giles (25°S 128°E) on 11 January 1988. The lowest three-hourly temperatures at the site were
26.2°C at 09:00 the previous day, and 27.5°C at 06:00, while other sites in the broader region
mostly exceeded 27°C. (As there are no sites to the north or south of Giles within 500 km, the
value affects the analysis over a large area). Such differences, if real, would almost certainly be
associated with a thunderstorm at Giles, but no significant rain was recorded there, suggesting
that the value on that day was suspect.
Their thunderstorm assumption is wrong. I can speak from personal experience of the central desert that non thunderstorm forming areas of cloud occur on a regular basis and cause a significant drop in temperature over a short period of time.

kadaka (KD Knoebel) says:
June 29, 2013 at 9:18 am
From jeremyp99 on June 29, 2013 at 4:49 am:
Meta is a prefix used in English (and other Greek-owing languages) to indicate a concept which is an abstraction from another concept, used to complete or add
Metadata is hence data about data
And metaphysics is physics about physics. With metaphysics completing or adding to physics.
=================================================================
Way back when I was studying philospophy, I learnt the origin of ‘metaphysics’ as being ‘beyond physics’. From Aristotle’s works; his writings on the subject now known as part of philosphy coming after those on physical science. (‘meta’ being beyond or after in Attic Greek)
‘meta’ has changed meaning in the last 40 or so years to its current usage

Nick, any change to raw data is an adjustment to the data set. To call some changes Quality control is mere semantics.
As usual, you don’t address the substantive issue. That they are making a changes to raw data based on a clearly wrong assumption. And note, Giles is probably the station that has the largest geographic effect on Australian temperatures.

The BoM document states,
In general, data that were classified as suspect after review were flagged and excluded from
further analysis.
So in the example, I originally gave, I’d assume the claimed ‘suspect’ data was in fact excluded from the dataset.
The example was from 1988 at a professionally manned station with presumably automatic temperature recording. ;What people ‘wrote down’ or potentially misread is irrelevant. And anyway the data are what was recorded, errors and all.

What is meant by the “minimum” and “maximum” temperature?
We’ve had at least two days recently in Auckland, NZ where the temperature at midnight was higher that the temperature at 3pm (Warm previous day, persistent cloud over night and during the day, with a strong southerly during the day to cool things down). If the “minimum” is actually the “overnight low” and the maximum is the “afternoon high”, then you could end up with the minimum higher than the maximum.

I prefer to adjust raw data myself. After watching Hansen adjust temperature data I am absolutely certain that I would want raw temperature data if I were doing a correlation or a proof for some reason. But I would be very careful what adjustments I would use. Let me demonstrate with an example that you can do. Go to Google and input “temperature Washington DC”, select wunderground.com from the list and scroll down to Washington Weather Stations and review the 30 or 40 temperature readings. These are within 25 miles to 50 miles of DC. The temperatures range from 83F to 94F. I have done this same chore for where I live in all kinds of weather/time of day and the differential range is always similar to DC.
The measurement stations are MADIS, RAPIDFIRE and NORMAL. You can click on the name and get time graphs of the data. You can click on those stations with an > at the end and get a full range of weather information.
My point is that one can fool ones self with small adjustments. Looking at the big picture often yields better decisions. I would never do what BEST did, which is to make a mulligan stew by throwing all readings into a pot. I would carefully select perhaps 100 or fewer temperature records around the globe, examine them very carefully for suitability and work with those data sets.
I agree with Wayne, we got side tracked on this one. But thank you very much, Willis. You are a tremendous worker with great ability to analyze data and spot problems.

Giles has been manned since 1956, although only since 1972 by BoM staff. I’ll take your word for it not being automated until 1992.
The issue here is the data is the data. Transcription errors are something else. Trying to find errors in the data as recorded, decades later, just gives free rein to confirmation bias.
The example I gave is arguably an example of this, where an unusually low temperature is removed from the record, for what to me seems faulty reasoning. The problem with confirmation bias, is the people doing it, don’t know they are doing it, and it is devilishly hard to pin down after the fact.

I’ve been looking at “time of observation” issues for several years now, and I cannot come up with a reason why it would produce minimum temperatures higher than maximum temperatures, even with the TOBS causing the pairs of readings to be recorded for separate days, and even if days were missed (unless some very creative infilling methods were used).
For those unfamiliar with the issue, for a long time temperatures were taken with vertical mercury thermometers that had two effectively “ratcheted” markers. The maximum-temperature marker could be pushed up by the rising column of mercury, but would not fall if the mercury column fell. The minimum-temperature marker could fall with a falling column, but would not rise when the column rose.
Generally once per day, the observer would go out to the Stevenson box, record the settings of the maximum and minimum markers, then manually move them to the present height (temperature) of the column, which would allow them to be moved by the column over the next 24 hours.
If the reading were not taken at midnight, and it virtually never was, then the question could arise as to which calendar day the extreme really occurred on. For the 9am readings, the most reasonable assumption (barring other info) was that the maximum occurred the previous afternoon and the minimum occurred the present early morning. This is apparently what the ACORN records did.
But could this by itself explain the “inverted” readings? I don’t see how. Let’s say that on the 6th of the month, the observer notes a maximum of 20C and a minimum of 10C. The 20C max is recorded for the 5th and the 10C min is recorded for the 6th. The markers are reset to the temperature at the time of the reading, which MUST BE between 10C and 20C.
Let’s say a cold front is moving through that day, and the temperature in the next 24 hour period never again gets higher than that at the time of the reading. When the observer comes out on the morning of the 7th, the maximum temperature marker will always read at least 10C. To continue our example, he reads a maximum of 10C, which is recorded for the 6th, and a minimum of 5C, which is recorded for the 7th.
Let’s say that instead, the observer does not make any reading on the morning of the 7th, and his next reading (and resetting!) is on the morning of the 8th. His maximum reading, which is now for the previous 48-hour period, still must be at least 10C, even if the actual maximum temperature for the previous 24-hour period never got as high as 10C.
So to continue our example, our observer notes a 10C maximum and a 0C minimum. The 10C he records for the 6th, and the 0C he records for the 7th. Now he has a gap to fill in the records. It would be extremely doubtful, IMO, if he did not do some sort of interpolation, filling in a max for the 6th of around 15C to go between 20C for the 5th and 10C for the 7th. The same with minimum temperatures.
Fundamentally, though, I cannot construct a reasonable scenario in which either of these effects would lead to a minimum temperature greater than a maximum temperature.

Willis, The reason the BOM went to the trouble of doing a whole new data set was because a group including Sen. Cory Bernardi, myself, Ken Stewart, and the BOM independent audit team (which I put together in 2010) asked the Australian National Audit Office to do an independent audit of the BOM data. Bernardi raised this in parliament. They had to respond.
The original audit request: http://joannenova.com.au/2011/02/announcing-a-formal-request-for-the-auditor-general-to-audit-the-australian-bom/
My write up of the BOM response:
http://joannenova.com.au/2012/06/threat-of-anao-audit-means-australias-bom-throws-out-temperature-set-starts-again-gets-same-results/
There are 20 + articles analyzing the BOM data: http://joannenova.com.au/tag/australian-temperatures/
The independent audit team are the ones who spotted the problems with F–>C conversions, with different datasets, with weighting, grids, maxes greater than mins, inexplicable adjustments. Several of them engage with the BOM constantly asking for the original data and methods.
Credit to Chris Gillham, Ken Stewart, Geoff Sherrington, Ed, Andrew, Ian, Lance, Janama, David Stockwell, Warwick Hughes and several others behind the scenes. They keep the BOM under pressure.

Sorry if this is a duplicate. Please delete if the last came through.
—————————————————————————–
Willis, The reason the BOM went to the trouble of doing a whole new data set was because a group including Sen. Cory Bernardi, myself, Ken Stewart, and the BOM independent audit team (which I put together in 2010) asked the Australian National Audit Office to do an independent audit of the BOM data. Bernardi raised this in parliament. They had to respond.
The original audit request: http://joannenova.com.au/2011/02/announcing-a-formal-request-for-the-auditor-general-to-audit-the-australian-bom/
My write up of the BOM response:
http://joannenova.com.au/2012/06/threat-of-anao-audit-means-australias-bom-throws-out-temperature-set-starts-again-gets-same-results/
There are 20 + articles analyzing the BOM data: http://joannenova.com.au/tag/australian-temperatures/
The independent audit team are the ones who spotted the problems with F–>C conversions, with different datasets, with weighting, grids, maxes greater than mins, inexplicable adjustments. Several of them engage with the BOM constantly asking for the original data and methods.
Credit to Chris Gillham, Ken Stewart, Geoff Sherrington, Ed, Andrew, Ian, Lance, Janama, David Stockwell, Warwick Hughes and several others behind the scenes. They keep the BOM under pressure.

Philip Bradley said @ June 29, 2013 at 2:49 pm

The error is in trying to retrospectively apply the WMO temperature recording standard.

Given the rather infrequent application of the WMO Standard at temperature recording stations, it does seem rather pointless. Then some of us find the idea of using average temperature as a proxy for entropy rather pointless.
@ Gonzo
You left off the dates for Marble Bar’s record hot spell: Oct. 30, 1923 to Apr. 7, 1924.
D’you think Ms Gaia was angry that summer, too? 😉

Streetcred said @ June 29, 2013 at 5:00 pm

stan stendera says:June 29, 2013 at 12:08 am
>… What is wrong with these people that they just can’t tell the simple truth. No grant is worth your soul.
…
Mastering the art of Lying is an entry requirement for Australian socialist politicians and senior public servants … much like Obama and his mob.

How odd! Back in the early 1970s I worked in the CES and PMG department. I was told that since I was not Roman Catholic and I was a member of the Labor Party, that I would never be promoted. After teaching two people their job when they were promoted above me, I realised the truth of this and quit.

@ Lew Skannen
Wasn’t Crooked Mick the bloke with a face like a buffalo turd? And if you think that’s bad, you should have seen his bird! 🙂
The Git has fond memories of an outback cop called Blue who introduced him to the philosophy of Spinoza over a few beers…

Patrick,
“They always come crawling back to the “The Zapper” for some more…. sweet, sweet candy”
Great character!! LOL

Willis, you really should store the data in an Access database. No programming required to find those errors,just a simple SQL would have done it. Access will also allow you to do other data mining, such as the number of days per year the temp is above a certain line, also the number of record breaking days, etc. All simple SQL, the results of which you copy and paste into Excel for plotting.
Just a suggestion.

What is also interesting is that anomalous TMin>TMax was in 2010. 2010 was actually an abnormally cool year. Not once did it get above 29C. 232 days were below average. On that one anomalous day, Mar 1, and the day before were the coolest TMax of the year, a large plunge down for TMax in that period.

Obviously, The Scientist In Charge of ACORN-SAT didn’t take the trouble to do what I did, look at each individual error. He made an incorrect assumption, and has now hurried off to implement an incorrect solution.

What was the assumption you allege he made? Seems to me like he explained how such anomalies occur.

PS—I did love The Scientist’s explanation of how it will all come right …
“Clearly in these cases either the estimate of the max is too low or the min is too high; however, providing the adjustment process is unbiased, these cases will be offset by cases where the max in too high/min is too low, and there is no overall bias.”
A real scientist would, you know, actually determine if “the adjustment process is unbiased” before making such an unsupported claim, rather than simply assuming that it is unbiased …

He didn’t state an assumption, he gave a conditional caveat. Aren’t you making an assumption yourself? A real scientist would test the proposition. A real scientist would have asked questions and investigated. Looks to me like you threw your hands up when you came across some anomalies and made a grand statement about the quality of ACORN-SAT. That is blog-standard science. At any time you could have contacted ACORN and asked about the anomalies, but it took a commenter to go through the painstaking task of discovering the email address of the ACORN director, and laboriously constructing some sentences to discover more about the issue. That commenter followed a reasonable procedure in investigating the issue.

And that is why, as an accountant, I become concerned when I see such small “trivial” errors. Because it makes me wonder—what other errors are hidden in there?

You discovered a 0.02% ‘error’ based on the 917 inverted min/max in the data out of several million data points. The BOM say they have an error rate of a few tenths of a percent, an order of magnitude greater than you discovered. How is that hiding the errors? They even discuss some of the errors that crop up. If you want to find out what kinds of errors there have been, you can read their reference material, and, if you are a real investigator, contact them for further details. ‘Real scientists’ go the distance.
But if you can’t be bothered doing that, how about calculating what difference the ‘errors’ you discovered would make to the claim that, based on BOM surface data, summer 2012/2013 was the warmest on record. If the record was broken by 0.2C, then how much impact could 917 data errors have of 4 million? For instance – summer data is 10416 data points per annum for a network of 112 stations. If all 917 errors occurred in 2012/2013 summer, and all were biased high by 3.9C, what impact would that have on the record if you removed those anomaly altogether? But you know when these anomalies occurred, so you can simply take them out of the record and see what impact that has. Then you would be applying a statistical test to the issue that kicked off your investigation.
Either work with the information you have and do some statistical analysis with appropriate caveats, or find out more about the information you don’t have and do a throrough job. So far you’ve made sweeping criticisms with little effort to explore them.

Ken, can you describe what the error is, exactly, and how it leads to a bias in the records sufficient to undermine the assertion that Australian summer 2012/13 is the warmest on record? I can’t see how this ‘error’ would make a substantial difference.

barry says:
July 1, 2013 at 9:23 pm
Ken, can you describe what the error is, exactly, and how it leads to a bias in the records sufficient to undermine the assertion that Australian summer 2012/13 is the warmest on record? I can’t see how this ‘error’ would make a substantial difference.
———-
I’ve only looked at Alice Springs, but TMax for the year has gone up since 1910. However, it was a faster rise from 1910 to 1960. 1960 had the highest Tmax, since then the rise over all is very shallow. But what is interesting is that every 9 to 11 years Alice Springs has an abnormally cool summer. Those years are quite prominent.

Barry said:
But if you can’t be bothered doing that, how about calculating what difference the ‘errors’ you discovered would make to the claim that, based on BOM surface data, summer 2012/2013 was the warmest on record. If the record was broken by 0.2C, then how much impact could 917 data errors have of 4 million? For instance – summer data is 10416 data points per annum for a network of 112 stations. If all 917 errors occurred in 2012/2013 summer, and all were biased high by 3.9C, what impact would that have on the record if you removed those anomaly altogether?
——————————–
Barry, I am not a scientist nor a mathematician. But, I did spend the best years of my life feeding numbers to very senior politicians to sprout in Parliament. In the early years, I had people up the line checking everything I did, but later, as I got better at it, so not so much. To the best of my knowledge, nobody ever gave wrong information to Parliament based on my briefs.
What I learned is (i) always do a back-of-the-envelope check about where the decimal point should be; and (ii) small errors often conceal, or flag, large ones.
The point that Willis has very patiently been trying to make to you is that the thing the BOM consistently avoids is transparency about errors and weird results. Nobody is jumping up and down yelling “gotcha” because errors occur. Of course they do. Nobody is instantly claiming conspiracy theories because of errors or weird results – they happen, sometimes for valid reasons.
The problem is that they refuse to be transparent about what they are doing. They make adjustments, expunge records from their working datasets, use new algorithms, invent new metrics – without leaving a visible trail or providing more than a bunch of platitudes to explain what they are doing.
When preparing briefs for Premiers and Prime Ministers, when confronted with this kind of bullshit, I was always careful to insert the words “I am advised by the BOM (or whoever) that …”. No way was I going to drop some uninformed politician into saying that he/she actually believed it. Some of them chose to drop the qualifier – that was their call.
But “I am advised that …” doesn’t work so well on WUWT. What possible justification is there for not laying on the table exactly what is happening with publicly funded weather statistics, when, and why?