An evaluation of the use of min/max temps to determine average temps, and a comparison of three observation times.
Guest essay by: Robin McMeeking
I was an Air Force weather observer in the early 1960’s and have retained an interest in weather things. Later in in my civilian career I spent many years developing software. Several months ago while reading about adjustments that had been made to the historical temperature record it piqued my interest. I was initially surprised to learn about the reliance on min/max thermometers in these early records.
But in the days before flight there wasn’t any need for regular systematic tracking of temperature. At any rate much of the historical temperature record is based on averaging min/max values. To be useful these values need to be recorded daily at reasonably close to the same time of day. This record keeping was done by volunteers reading thermometers at selected locations. The assumption is that the volunteers were fastidious in performing this task.
It’s not intuitive, but has been recognized for many years that the time of day used for the readings can exert some bias toward higher or lower average temperature determinations. It’s called Time of Observation Bias (TOB). I was not convinced by the explanations of the reasons for TOB, but I was even more dubious about the validity of relying min/max values to calculate an accurate average temp.
So, when the issues about TOB and average temps came up I was interested in performing some analysis, but didn’t have any data to work with. Some time later I discovered the network of Personal Weather Stations (PWS) on Weather Underground. Eventually I found that daily historical data from the PWSs can be downloaded (one day at a time!). Fortunately I was able to develop software to perform the downloading. And when I was almost done with my preliminary analysis I reread some of the posts on TOB and discovered the NOAA and other sources of official temperature data. Oh well!
I wasn’t trying to look at climate trends, I basically wanted to evaluate the suitability of using Min/Max temps to derive daily average temps. And I wanted to simulate a variety of observation times. I selected 16 PWS’s from around the US and downloaded all of 2015 plus 1/1/16 to allow for shifting observation times (well over 1.5 million records).
Most of these PWS’s record at 5 minute intervals, a few at 15. I calculated a mean temp from the min/max values in each 24 hour period, averaged all of the reported temps, and averaged the last reported temp in each hour to simulate what would be available from historical WBAN records (like we did in the ’60s). I evaluated TOBs of midnight, 0800 and 1600.
I selected PWS’s that are in rural locations, but I realize that I have no information on the suitability of the siting of any of these stations. That bothered me so when I finally became aware of NOAA data (after I was done with the PWS stuff) I downloaded data for one station from NOAA and performed the same analysis. Results from that data were consistent with the PWS analysis.
Below are tables of the summarized data. If you know of anyone that would like to see results from the intermediate steps (daily totals with monthly summaries) I’d be happy to share it. What I have concluded is that Min/Max method tends to yield a higher value than the more inclusive averages at all three TOBs, with 1600 being worst and 0800 being “best” of the times I tested. Averages based on hourly readings (which frequently don’t include the max &/or min) are virtually indistinguishable from averaging all readings.
The tendency of Min/Max to be high led me to wonder how much time per day is spent with temp above or below the mean. The results on that surprised me. Shown in the tables below.
Based on 12 months of data for 2015. All temps in Farenheit.
![tob00[1]](https://wattsupwiththat.files.wordpress.com/2016/05/tob001.gif)
![tob08[1]](https://wattsupwiththat.files.wordpress.com/2016/05/tob081.gif)
![tob16[1]](https://wattsupwiththat.files.wordpress.com/2016/05/tob161.gif)
Note from Anthony: Some readers might suspect the PWS network has exposure and placement problems, like the NOAA network, and they would be right. However, since he isn’t looking at trends, just the diurnal variation of temperature and the issue of recording times, in my opinion, that issue doesn’t apply here.
The bottom line is the data for the last 130 years is junk data. The SH time series is almost all fabrication
The truth is half of global warming comes from adjusting the past, half of what is left is natural and half of what is left of that is our influence since 1970 (alleged) meaning, of the “measured” 1 degree of warming in that time, we “may” be responsible for 10% of the warming since 1850. “may”.
For me, when NOAA told us a record temperature in 2015 only using land data, because they tried to detach it from El Nino was the most pathetic sly tactic one could imagine.
When you see what they have done to the records of countries like Iceland, and openly talked about “getting rid of some of the 1940s blip” and you see the NASA adjustments, wow, how does one deluded themselves into thinking this is not suspicious, very suspicious.
What amazes me is the lack of logical thinking in “scientists”
They either dont believe or cant admit that if half of warming is from adjustments, half of warming is HIGHLY uncertain.
max min night time temps and all recorded temps inbetween give a nightly average. Daily max min and all measurements inbetween give a daily average.
This shows the duration of temperatures in the average for example, if there are 4 consecutive hours at Max temp 54f and 6 hours at 49 min temp with 2 at 51f, an average of max min will create a warmer trend than measurements actually show.
How long any one temperature is sustained is very relevant which is why you need to average all measurements, 8 hour periods are not great and only serve to paper over the fact that 3 is almost as useless as 2 values to average.
Monthly max and min in calculations are useless if the underlying monthly averages, derived from incorrectly calculated daily averages, is meaningless.
The Washington Post weather page includes a section that compares the difference between the “30 year average (Reagan)” with the current month and year to date temperatures. Today it indicated that this month is -2.7 and YTD is +1.5. Does anyone know how this 30 year average is computed? Is it the historical daily average of highs and lows and has anyone looked at the UHI effect at Reagan airport. My guess is the Washington Post may use this as definitive proof of AGW.
Sadly for the Wapo a temperature trend has no bearing on AGW theory. Warming does not testify to the fact CO2 can actually have the effect claimed by AGW proponents.
All long term stations at airports that have grown will see increased UHI influence, the heat drifting from runways in a breeze as well as siting issues, and all that heat comes right back out at night time.
London city is now home to species of plant that would not grow there previously, because of UHI. BBC had a documentary on it years ago, when they were skeptical about man made global warming of course, they’d never do a doc on UHI’s effects now!
What about bias due to “timezone time” vs solar time?
US time zones are more political than having anything more than a passing relationship to solar time (position of the sun in the sky). Throw in the other political factor – daylight saving, and trying to compare readings from different locations, in different seasons becomes almost meaningless.
Why get into a tiswas over TOBs and whether readings should be hourly or whatever? In
T min mathematical language even hourly readings are “discrete” whereas time is “continuous”.
(Tmax + Tmin)/2 can only approximate to Tmean , no matter how frequent the readings. Why not calculate Tmean directly from the area under the curve?
sorry, gremlins got in, corrected post below.
Why get into a tiswas over TOBs and whether readings should be hourly or whatever? In
mathematical language even hourly readings are “discrete” whereas time is “continuous”.
Any (Tmax + Tmin)/2 can only approximate to Tmean , no matter how frequent the readings. Why not calculate Tmean directly from the area under the curve?
To produce what?
Time is quantised into seconds minutes hours ect. If you took a measurement every second the data would be continuous too, each measurement would be discrete as each second the measurement was taken.
You are replacing valid data capture and averaging with statistical artifact.
If I have a jug of water and I measure it at sunrise, and every 15 minutes thereafter until sunset, all of those values contain time and temperature! Cant you see that? There is no need to put into the statistical mincer
Time is captured because if you measure at 15 minute intervals, or 30 or an hour, and if 4 consecutive measurements are the same temperature, that’s 1, 2 or 4 hours respectively of temperature data and it adds the correct value to the total average for the day. There is your time right there if you want higher resolution take more readings, say 30 minute intervals as temperature can reasonably be expected to change within an hour, less than 30 minutes is probably too much. Smaller fluctuations are noise.
The data is the data, you average it all to get average daily temp, simple as, doing anything else is creating an artifact.
The obsession with anomalies is disturbing, all we need is the measurements and averages, but we only use the measurements for future work, not the averages. The averages when used in 2 more consecutive statistical operations mean the final work is 3 steps removed from the actual data.
This is why temperature record now looks nothing like the temperature record we had from measurements.
All through the numbers mincer to create artifacts that really have no bearing on the real world.
Essentially all you have left after all that chicanery is a fictional residue of something that has nothing to do with the real world, if you are stuck in a concept, you wont see it that way
“The obsession with anomalies is disturbing, all we need is the measurements and averages, but we only use the measurements for future work, not the averages.”
if you dont like anomalies dont use them.
People tend to like to make things more complicated than they are to feed intellectualism, academics especially so.
Simplest answer is more often than not the best answer.
I do love Heller’s gifs though, they amuse me 😀
http://realclimatescience.com/wp-content/uploads/2016/05/Teigarhorn-2011-2016-1.gif
The 1940s “blip” had to go, models cant recreate it without CO2, the same reason the MWP had to go, the same reason Karl’s data is more and more resembling the CO2 growth curve
That station in the picture looks like it is in a reasonable position but then they put some steps there and painted them a dark colour which when the sun is out would generate heat.
Recently I saw a photo of a station with an incinerator within ten metres of it.
Robin: In your post, what is it that is biased? If temperatures are consistently biased in one direction or another, the temperature TREND will remain constant. If a bias in temperature readings CHANGES, the TREND will be biased.
If we had continuous monitoring of temperature, we might think we have a better idea of what the temperature at a location really is. However, there is no “right” way to measure temperature at 2 m above the surface and we are stuck with historical temperature data from min/max thermometers. Our job is to measure WARMING or TREND as accurately as possible, to remove changing biases, not constant ones.
Robin: In your post, what is it that is biased? If temperatures are consistently biased in one direction or another, the temperature TREND will remain constant. If a bias in temperature readings CHANGES, the TREND will be biased.
If we had continuous monitoring of temperature, we might think we have a better idea of what the temperature at a location really is. However, there is no “right” way to measure temperature at 2 m above the surface and we are stuck with historical temperature data from min/max thermometers. Our job is to measure WARMING or TREND as accurately as possible, to remove changing biases, not constant ones.
_____________________________________________________
Temperature adjustment bias have an axis centered on around 1960. Adjusted lower pre 1960 and adjusted more linear post 1960 (not adjusted to increase but to match CO2 growth)
Tom Karl tried to then revise the slow down in warming which would have further revised the record with a bias towards CO2 growth)
“historical temperature data from min/max thermometers.”
A large loss of stations in cold parts of the world too in Russia
Massive gaps and the closer you get to 1900 the more data % is US only.
Revisionism of records for countries.
The open discussion of “getting rid” of the 1940s SST and the remaining land blip
Making adjustments based on assumptions like “what they think people were doing with thermometers in the past”
Some think statistical analysis is science, it’s a tool used in science, it is not science.
From the author: Not quite sure how to answer. I have run across the term Time of Observation Bias in several articles. The term bothered me and it took some research to understand. But even once I understood the claim I wasn’t completely convinced of its validity.
The claim is only a concern with a particular type of record keeping. But that type of record keeping is what most of the first 50 years of temperature records are based on. As I understand it, some government department before there was a weather bureau (after the Civil War) decided to try to develop a climate history for the US which included temperature data from a variety of selected locations. In these locations they installed instrument shelters that housed recently developed thermometers that recorded maximum and minimum temperatures between settings. The sticky part is “between settings”. To get daily Min/Max temps the thermometer needs to be read and reset daily. This task was performed by volunteers (which raises other issues). For the most part the readings were initially supposed to be done around 4:00PM local time (which raises other issues). Somewhere around 1900 the recommended time was changed to 8:00 AM. In both cases some local discretion was allowed. At some point the question was raised about the effect of doing the readings at various times of day. Was there some bias introduced by choosing different times of observation.
Eventually it was concluded that “Time of Observation Bias” was a real artifact and should be corrected for. I had a hard time getting my head around the issue so I decided to perform the analysis described above. Hope that helps.
Mark,
“Some think statistical analysis is science, it’s a tool used in science, it is not science.”
Very true. But no scientist should fail to use it. Because proper understanding and use of statistical evidence is essential for verifying scientific discoveries, particularly when experimental replication is complicated or, as in the case of much climate “science”, impossible.
“…we are stuck with historical temperature data from min/max thermometers. Our job is to measure WARMING or TREND as accurately as possible, to remove changing biases, not constant ones.”
True. But the fact that modern methods of recording temperatures continuously were not available in the past is no reason not to use them today.
Consider the purely hypothetical case of the thermometer placed close to the runway at a wartime air strip on an underpopulated island, currently in use as the island’s sole airport. In the fifties and early sixties the only flights were props but, later, one service each day was replaced by a jet. As the island became more popular with tourists one of the jets was upgraded and by the late 1990s this aircraft was very large. If Tmax is recorded once every 24 hours what is the betting that Tmax in the 21st century is significantly greater than Tmax in the fifties and early sixties? And what is the betting that the “trend” in Tmax is far greater than the trend in Tmin? And would the trend in (Tmax+Tmin)/2 really represent the increase over the period?
Of course it could be said that this is a unrealistic example, since all thermometers are better located. (are they?) Or that no natural phenomena replicate the temporary increase in temperature that would be produced by a jet engine. But, for many years, I lived in a country where, in the summer months, the maximum temperature during siesta time in the rural areas, inspired by the midday sun, invariably recorded a spike of several degrees Fahrenheit higher than that shown on the thermometer for the remaining eight or nine daylight hours.
Incidentally I agree with most of your posts, particularly “People tend to like to make things more complicated than they are to feed intellectualism, academics especially so. Simplest answer is more often than not the best answer.”