I decided to do myself something that so far NOAA has refused to do: give a CONUS average temperature for the United States from the new ‘state of the art’ United States Climate Reference Network (USCRN). After spending millions of dollars to put in this new network from 2002 to 2008, they are still giving us data from the old one when they report a U.S. national average temperature. As readers may recall, I have demonstrated that old COOP/USHCN network used to monitor U.S. climate is a mishmash of urban, semi-urban, rural, airport and non-airport stations, some of which are sited precariously in observers backyards, parking lots, near air conditioner vents, airport tarmac, and in urban heat islands. This is backed up by the 2011 GAO report spurred by my work.
Here is today’s press release from NOAA, “State of the Climate” for July 2012 where they say:
The average temperature for the contiguous U.S. during July was 77.6°F, 3.3°F above the 20th century average, marking the hottest July and the hottest month on record for the nation. The previous warmest July for the nation was July 1936 when the average U.S. temperature was 77.4°F. The warm July temperatures contributed to a record-warm first seven months of the year and the warmest 12-month period the nation has experienced since recordkeeping began in 1895.
OK, that average temperature for the contiguous U.S. during July is easy to replicate and calculate using NOAA’s USCRN network of stations, shown below:
The U.S. Climate Reference Network (USCRN) consists of 114 stations developed, deployed, managed, and maintained by the National Oceanic and Atmospheric Administration (NOAA) in the continental United States for the express purpose of detecting the national signal of climate change. The vision of the USCRN program is to maintain a sustainable high-quality climate observation network that 50 years from now can with the highest degree of confidence answer the question: How has the climate of the nation changed over the past 50 years? These stations were designed with climate science in mind. Three independent measurements of temperature and precipitation are made at each station, insuring continuity of record and maintenance of well-calibrated and highly accurate observations. The stations are placed in pristine environments expected to be free of development for many decades. Stations are monitored and maintained to high standards, and are calibrated on an annual basis. In addition to temperature and precipitation, these stations also measure solar radiation, surface skin temperature, and surface winds, and are being expanded to include triplicate measurements of soil moisture and soil temperature at five depths, as well as atmospheric relative humidity. Experimental stations have been located in Alaska since 2002 and Hawaii since 2005, providing network experience in polar and tropical regions. Deployment of a complete 29 station USCRN network into Alaska began in 2009. This project is managed by NOAA’s National Climatic Data Center and operated in partnership with NOAA’s Atmospheric Turbulence and Diffusion Division.
So clearly, USCRN is an official effort, sanctioned, endorsed, and accepted by NOAA, and is of the highest quality possible. Here is what a typical USCRN station looks like:
A few other points about the USCRN:
- Temperature is measured with triple redundant air aspirated sensors (Platinum Resistance Thermometers) and averaged between all three sensors. The air aspirated shield exposure system is the best available.
- Temperature is measured continuously and logged every 5 minutes, ensuring a true capture of Tmax/Tmin
- All stations were sited per Leroy 1999 siting specs, and are Class 1 or Class 2 stations by that siting standard. (see section 2.2.1 here of the USCRN handbook PDF)
- The data goes through quality control, to ensure an errant sensor hasn’t biased the values, but is otherwise unchanged.
- No stations are near any cities, nor have local biases of any kind that I have observed in any of my visits to them.
- Unlike the COOP/USHCN network where they fought me tooth and nail, NOAA provided station photographs up front to prove the “pristine” nature of the siting environment.
- All data is transmitted digitally via satellite uplink direct from the station.
So this means that:
- There are no observer or transcription errors to correct.
- There is no time of observation bias, nor need for correction of it.
- There is no broad scale missing data, requiring filling in data from potentially bad surrounding stations. (FILNET)
- There are no needs for bias adjustments for equipment types since all equipment is identical.
- There are no need for urbanization adjustments, since all stations are rural and well sited.
- There are no regular sensor errors due to air aspiration and triple redundant lab grade sensors. Any errors detected in one sensor are identified and managed by two others, ensuring quality data.
- Due to the near perfect geospatial distribution of stations in the USA, there isn’t a need for gridding to get a national average temperature.
Knowing this, I wondered why NOAA has never offered a CONUS monthly temperature from this new network. So, I decided that I’d calculate one myself.
The procedure for a CONUS monthly average temperature from USCRN:
- Download each station data set from here: USCRN Quality Controlled Datasets.
- Exclude stations that are part of the USHCN-M (modernized USHCN) or USRCRN-Lite stations which are not part of the 114 station USCRN master set.
- Exclude stations that are not part of the CONUS (HI and AK)
- Load all July USCRN 114 station data into an Excel Spreadsheet, available here: CRN_CONUS_stations_July2012_V1.2
- Note stations that have missing monthly totals data. Three in July 2012, Elgin, AZ, (4 missing days) Avondale, PA,(5 missing days) McClellanville, SC, (7 missing days) and set their data aside to be dealt with separately.
- Do sums and calculate CONUS area averages from the Tmax, Tmin, Tavg and Tmean data provided for each station.
- Do a separate calculation to see how much difference the stations with missing/partial data make for the entire CONUS.
Here are the results:
USA Monthly Mean for July 2012: 75.72°F
(111 stations)
USA Monthly Average for July 2012: 75.51°F
(111 stations)
USA Monthly Mean for July 2012: 75.74°F
(114 stations, 3 w/ partial missing data, difference 0.02)
USA Monthly Average for July 2012: 75.55°F
(114 stations, 3 w/ partial missing data, difference 0.04)
============================
Comparison to NOAA’s announcement today:
Using the old network, NOAA says the USA Average Temperature for July 2012 is: 77.6°F
Using the NOAA USCRN data, the USA Average Temperature for July 2012 is: 75.5°F
The difference between the old problematic network and new USCRN is 2.1°F cooler.
This puts July 2012, according to the best official climate monitoring network in the USA at 1.9°F below the 77.4°F July 1936 USA average temperature in the NOAA press release today, not a record by any measure. Dr. Roy Spencer suggested earlier today that he didn’t think so either, saying:
So, all things considered (including unresolved issues about urban heat island effects and other large corrections made to the USHCN data), I would say July was unusually warm. But the long-term integrity of the USHCN dataset depends upon so many uncertain factors, I would say it’s a stretch to to call July 2012 a “record”.
This result also strongly suggests, that a well sited network of stations, as the USCRN is designed from inception to be, is totally free of the errors, biases, adjustments, siting issues, equipment issues, and UHI effects that plague the older COOP USHCN network that is a mishmash of problems that the new USCRN was designed to solve.
It suggests Watts et al 2012 is on the right track when it comes to pointing out the temperature measurement differences between stations with and without such problems. I don’t suggest that my method is a perfect comparison to the older COOP/USHCN network, but the fact that my numbers come close, within the bounds of the positive temperature bias errors noted in Leroy 1999, and that the more “pristine” USCRN network is cooler for absolute monthly temperatures (as would be expected) suggests my numbers aren’t an unreasonable comparison.
NOAA never mentions this new pristine USCRN network in any press releases on climate records or trends, nor do they calculate and display a CONUS value for it. Now we know why. The new “pristine” data it produces is just way too cool for them.
Look for a regular monthly feature using the USCRN data at WUWT. Perhaps NOAA will then be motivated to produce their own monthly CONUS Tavg values from this new network. They’ve had four years to do so since it was completed.
UPDATE: Some people questioned what is the difference between the mean and average temperature values. In the monthly data files from USCRN, there are these two values:
T_MONTHLY_MEAN
T_MONTHLY_AVG
http://www.ncdc.noaa.gov/crn/qcdatasets.html
The mean is the monthly (max+min)/2, and the average is the average of all the daily averages.
UPDATE2: I’ve just sent this letter to NCDC – to ncdc.info@ncdc.noaa.gov
Hello,
I apologize for not providing a proper name in the salutation, but none was given on the contact section of the referring web page.
I am attempting to replicate the CONUS temperature average of 77.6 degrees Fahrenheit for July 2012, listed in the August 8th 2012, State of the Climate Report here: http://www.ncdc.noaa.gov/sotc/
Pursuant to that, would you please provide the following:
1. The data source of the surface temperature record used.
2. The list of stations used from that surface temperature record, including any exclusions and reasons for exclusions.
3. The method used to determine the CONUS average temperature, such as simple area average, gridded average, altitude corrections, bias corrections, etc. Essentially what I’m requesting is the method that can be used to replicate the resultant 77.6F CONUS average value.
4. A flowchart of the procedures in step 3 if available.
5. Any other information you deem relevant to the replication process.
Thank you sincerely for your consideration.
Best Regards,
Anthony Watts
===================================================
Below is the response I got to the email address provided in the SOTC release, some email addresses redacted to prevent spamming.
===================================================
—–Original Message—–
From: mailer-daemon@xxxx.xxxx.xxx
Date: Thursday, August 09, 2012 3:22 PM
To: awatts@xxxxxxx.xxx
Subject: Undeliverable: request for methods used in SOTC press release
Your message did not reach some or all of the intended recipients.
Sent: Thu, 9 Aug 2012 15:22:43 -0700
Subject: request for methods used in SOTC press release
The following recipient(s) could not be reached:
ncdc.info@ncdc.noaa.gov
Error Type: SMTP
Error Description: No mail servers appear to exists for the recipients address.
Additional information: Please check that you have not misspelled the recipients email address.
hMailServer
===============================
UPDATE3: 8/10/2012. This may put the issue to rest about straight averaging -vs- some corrected method. From http://www.ncdc.noaa.gov/temp-and-precip/us-climate-divisions.php
It seems they are using TCDD (simple average) still. I’ve sent an email to verify…hopefully they get it.
Traditional Climate Divisional Database
Traditionally, climate division values have been computed using the monthly values for all of the Cooperative Observer Network (COOP) stations in each division are averaged to compute divisional monthly temperature and precipitation averages/totals. This is valid for values computed from 1931 to the present. For the 1895-1930 period, statewide values were computed directly from stations within each state. Divisional values for this early period were computed using a regression technique against the statewide values (Guttman and Quayle, 1996). These values make up the traditional climate division database (TCDD).
Gridded Divisional Database
The GHCN-D 5km gridded divisional dataset (GrDD) is based on a similar station inventory as the TCDD however, new methodologies are used to compute temperature, precipitation, and drought for United States climate divisions. These new methodologies include the transition to a grid-based calculation, the inclusion of many more stations from the pre-1930s, and the use of NCDC’s modern array of quality control algorithms. These are expected to improve the data coverage and the quality of the dataset, while maintaining the current product stream.
The GrDD is designed to address the following general issues inherent in the TCDD:
- For the TCDD, each divisional value from 1931-present is simply the arithmetic average of the station data within it, a computational practice that results in a bias when a division is spatially undersampled in a month (e.g., because some stations did not report) or is climatologically inhomogeneous in general (e.g., due to large variations in topography).
- For the TCDD, all divisional values before 1931 stem from state averages published by the U.S. Department of Agriculture (USDA) rather than from actual station observations, producing an artificial discontinuity in both the mean and variance for 1895-1930 (Guttman and Quayle, 1996).
- In the TCDD, many divisions experienced a systematic change in average station location and elevation during the 20th Century, resulting in spurious historical trends in some regions (Keim et al., 2003; Keim et al., 2005; Allard et al., 2009).
- Finally, none of the TCDD’s station-based temperature records contain adjustments for historical changes in observation time, station location, or temperature instrumentation, inhomogeneities which further bias temporal trends (Peterson et al., 1998).
The GrDD’s initial (and more straightforward) improvement is to the underlying network, which now includes additional station records and contemporary bias adjustments (i.e., those used in the U.S. Historical Climatology Network version 2; Menne et al., 2009).
The second (and far more extensive) improvement is to the computational methodology, which now addresses topographic and network variability via climatologically aided interpolation (Willmott and Robeson, 1995). The outcome of these improvements is a new divisional dataset that maintains the strengths of its predecessor while providing more robust estimates of areal averages and long-term trends.
The NCDC’s Climate Monitoring Branch plans to transition from the TCDD to the more modern GrDD by 2013. While this transition will not disrupt the current product stream, some variances in temperature and precipitation values may be observed throughout the data record. For example, in general, climate divisions with extensive topography above the average station elevation will be reflected as cooler climatology. A preliminary assessment of the major imapacts of this transition can be found in Fenimore, et. al, 2011.
JJB MKI says:
August 9, 2012 at 4:11 am
@Nick Stokes
Last time I looked, the GISS data set for England was constructed from a selected homogenised set of over 70 stations in the early 20th century, spanning both rural and urban locations, narrowing down to about a dozen stations, all located at busy airports in the present day, with the information presented as anomalies. No obvious reason given for the data cull btw, as the culled stations did not stop reporting. By your own logic, it would be fallacious to use GISS to claim warming over this period.
_______________________________
TO add to that the Station Dropout vs Temperature Graph The number of stations used to determine Global Warming has not been constant over the entire 170 year period.
Therefore there is no reason not to use the new pristine high quality station 1&2 data instead of the munged-up data set with a large number of 3&4&5 stations. Unless of course the whole objective of the exercise is something different than reporting the weather.
“The whole aim of practical politics is to keep the populace alarmed (and hence clamorous to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary.” ~ H. L. Mencken
And do not forget GISS was caught data tampering before GISS caught red-handed manipulating data to produce Arctic Climate History Revision
Another set of interesting graphs at Digging in the Clay show the number of North & Central America stations and how that changes over time and how much of the “World” is actually just North America temperature. Graph 1 and Graph 2
For North America reading off the graph, the data set goes from ~ 200 data stations in 1840 to ~ 100 stations now with a maximum of over 2000. IF GISS is using ~ 100 stations now, how is that different than the U.S. Climate Reference Network (USCRN) consisting of 114 stations?
If they ditched over two thousand data stations why would they not now be using this highest quality data set?
Also as Anthony has suggested if all the data adjustments were done correctly and the station siting is correct then there should be very little difference between these two data sets. Afterall these data sets are BOTH supposed to give us the temperature of the USA.
Then there is the fact that the monthly surface temperature updates for the contiguous U.S., based upon 280 International Surface Hourly (ISH) stations which have reasonably complete temperature records since 1973 and the “TEAM’s” calculated data sets are diverging. graph
differences graph
SO for those who are suggesting an apples and oranges problem. A separate set done by someone besides Anthony picks up the same problems.
I ran this past a Warmist and his reply was:
” that’s an oranges / apples comparison. You’re comparing 76-year-old records from all US weather stations with records from a new, specialised climate monitoring network. It’d be like measuring the length of a football field with an old fibreglass tape, and comparing it to the measurement made with a high-precision laser rangefinder. You’re going to see a difference in the reported number, despite them being measurements of the exact same thing.
Have you adjusted (I believe “homogenised” is the term climate scientists use) the USCRN results to match those from the old temperature series? Or vice versa?
Yes, the USCRN network is designed to give the best possible results, free of any local influences (other than the weather!). But, given that we’re interested in the trend here, homogenised results from the old network (which correct, as much as possible, for those “errors, biases, adjustments, siting issues, equipment issues, and UHI effects” you talk about) still give us useful information about the long-term trend, from prior to the USCRN coming into existence (which was only 12 years ago, IIRC).
Or is it your position that, because we didn’t have a super-high-quality climate monitoring network in place, that we should completely ignore any climate data gathered prior to the USCRN being commissioned?
Never mind that the USHCN data correlates very well with the satellite data, which is completely unaffected by those problems with the surface data you’re talking about…”
I was looking at the NOAA site for adjustments,
ttp://www.ncdc.noaa.gov/oa/climate/research/ushcn/ushcn.html#KDK88
For urbanizaion they referrence Karl et al. 1988 but when searching google scholar the link gives an “Error – Page Not Found”. Does anyone have a valide link for the paper?
The UAH July satellite temperatures for the Lower US 48 are out and do not show anything remarkable at +0.91C.
For those that do not know this, UAH lower troposphere does provide a very good match to the NOAA/NCDC’s temperature record for the Contiguous US.
The NOAA/NCDC has, however, a 27.2% higher trend than UAH. This is “another” check on the NCDC’s temperature adjustments which are obviously too high and are not justifiable.
This chart shows there was nothing noteworthy about July, 2012. The anomaly was down from a few months earlier and UAH has a much lower anomaly in July.
http://s10.postimage.org/rtykuwrgp/US_UAH_vs_USHCN_V2_July2012.png
And then the daily Global UAH satellite temps back to 2010 – showing nothing remarkable happening at all last month – flat.
http://s11.postimage.org/8j5gu2nf7/Daily_UAH_Temp_July2012.png
Steve (at August 9, 2012 at 6:00 pm),
I am pretty sure that is not how they compute the national average. The national average *used* to be calculated in that manner by aggregating the individual climate division data, but NCDC stopped doing it that way a number of years ago. Consequently, the national average calculated from climate divisions does not match the national average that NCDC commonly reports (the underlying temperature data is not the same…the climate division datasets only use a TOBS adjustment).
So the national data (code 110) in the file whose README you point to does *not* derive from the other data in the file. I thought that NCDC made an annotation about this fact after I discussed it with them several years ago, but I can’t seem to find it.
As you might guess, the national average temperature since 1895 warms up quite a bit slower if aggregated from the climate division data than in the NCDC “official” national temperature record–the construction about which Anthony is inquiring about (but we do know that it is constructed using the fully adjusted USGCN v2 data).
-Chip
But….. But…. When Obama became president all this was supposed to change! /sarc
@Tom Harris at August 9, 2012 at 9:59 pm
Thanks Tom for taking the moment to understand what I was saying, it seems few grasped it. That aspect began worrying me when I read a story, I think here at wuwt last year, when an anomalous high record in the climate record was recorded in Kansas at 1 a.m. in the morning at 122°F. It was called a microburst and only lasted a few minutes but our new instruments are so sensitive in time, the temporal axis, that this was actually recorded as the high for the day. Now the hourly average was much lower, can’t remember exactly but it was something like 101°F. Now recording that freak temperature for historic reasons is fine but I care little in the climate sense what a five minute slice of temperature was.
So, I started looking at this hourly in June here in okc and sure enough, the maximums were usually, not always, but usually 1 to 2°F higher than the highest hourly temperature. Once again, that is fine for history books of curious peaks but the shame is that this is the temperature that is carried on into the climate records, not the highest hourly temperature. That to me is wrong and is another factor that is skewing our ‘maximum highs’ and ‘minimum lows’. They are now basically instantaneous where in the past they were hourly or even from bi-daily readings.
On day here in July NOAA read 113°F as maximum according but I watched that continuously for the two hour period from my patio and it never got over 111°F even in the shade above my 200 sq.ft. concrete patio that is in full sunlight. Talk about contaminated reading, my patio could not be worse! That is when I knew something was amiss. I don’t question that at the airport for a few minutes that it actually reached 113°F, I’m a sailplane pilot and I do realize this phenomena of hot downdrafts (read potential temperature) do frequently occur but they only last minutes and when averaged into an hour they are rather meaningless… but that’s what the nightly news will report and that is what you will see in the climate records.
In the 1930’s they probably were not even aware of this effect but our current high-speed high-accuracy temperature devices detect these bursts.
Well, thanks for realizing the magnitude this can have on our climate records. Are we really 1°C warmer now that then, are we having more new highs? I doubt it and have evidence to back that up, it’s in the NOAA hourly records. It’s all a matter of ‘time’ and the way the records are being recording.
See the update to the post about TCDD and GrDD methods of obtaining averages. Waiting for NCDC to verify that 77.6F was computed via TCDD.
Here is how it works, from an email sent in reply to my question:
station data => divisional data => area weighted to the larger climate regions => national number
And there you have the method.
REPLY: Show the whole email. I simply don’t trust you given your history here. – Anthony
Yes, it was using the TCDD, they haven’t made the transition yet.
REPLY: I’ll take NOAA’s reply as final. NSIDC once slipped in a new algorithm with nobody noticing at NSDIC brass, we bloggers had to point it out. – Anthony
Gail Combs,
Station dropout! Now that makes me nostalgic. Try this data instead: http://curryja.files.wordpress.com/2012/07/fig2.jpg
DCA,
The Karl et al reference is out of date. USHCN v2 doesn’t have any explicit UHI adjustment, and relies (albeit imperfectly) on the Pairwise Homogenization Algorithm to remove any urban bias.
Anthony,
Here:
“Hi John,
No problem. TCDD is still being used this year…we expect to transition to GrDD next year.
Right now, the station data ===> divisional data ===> area weighted to the larger climate regions ===> national number.
Have a look at
http://www.ncdc.noaa.gov/temp-and-precip/us-climate-divisions.php”
REPLY: That’s not a complete email. What was the request, and who sent the reply? Show the whole email, please, it is public record since it was NOAA correspondence. – Anthony
For those of you who are confused about “mean” and “average” and “median”. Mean is the value of (Tmax+Tmin) / 2. Average is the the sum of all Tmax readings plus the sum of all Tmin readings divided by the total number of readings. Median is the value of the mid point of a set of values.
For those of you confused by Mean, Average and Median I offer the following. Mean is simply: (Tmax+Tmin)/2. Average is the sum of all Tmax plus the sum of all Tmin divided by the total number of data pairs. Median is the middle number of a set of data points. If you have an even number of points (e.g. 6), then the median is the average of value of points three and point four. If you have 7 data points then the median is the value of point 4)
Actually it should be: mean is the second definition and average is the first definition
It was Scott Stephens, and that is the entire contents of the reply.
[Then show the context with earlier emails. ~dbs, mod.]
The NCDC national average is *not* computed from the same data used to produce climate division (CD) averages. So Rattus’ chain is incorrect… unless NCDC is computing an alternate set of CD averages using fully-adjusted USHCN v2 that they are not reporting.
That the national average is not derived from the traditional NCDC CD data can be verified from the dataset itself.
The data is here and the areal weightings are here. When I calculate a national average for the US for July based on the weighted regional averages, I get 78.0 deg F, which is different from the 77.6 that is being advertised.
Using NCDC’s CD data produces a different national timeseries than the one that NCDC makes available.
-Chip
And that’s the rub Chip, reproducibility. Hopefully my request will provide a reproducible method that gives us 77.6F
Agreed, Anthony!
-Chip
@ur momisugly Nick Stokes says: Aug 10 at 2:38 am
The fact is that these files have over a century of data. For the majority there are not 8640 data points, but just 3 per day (Max, Min and temp at time of reading). That’s all we have. For modern instrumentation, they emulate that measure. Otherwise comparison with the past would not work.
No, Nick, you have it wrong. The subject is the new USCRN, the Climate REFERENCE Network created with automation to record temperatures every 5 minutes. The USCRN was created to correct the errors of past practice! With the USCRN, we should be doing things right. We should be using all 8640 readings each month, not just the min and max of yesterday.
What you described is a red carpet to adjusting USCRN data to match the much longer historical records of COOP. We must not do that. With USCRN we do it right from day one.
@greymouser70, think you meant the mid-range or average of the extents when referring to (Tmax+Tmin)/2. A ‘mean’ and a common ‘average’ are define as the same thing, the sum of all values divided by the number of values.
@ur momisugly Nick Stokes says: Aug 10 at 2:38 am
The fact is that these files have over a century of data. For the majority there are not 8640 data points, but just 3 per day (Max, Min and temp at time of reading). That’s all we have. For modern instrumentation, they emulate that measure. Otherwise comparison with the past would not work.
No Nick, that is not the same. [my bold] You are assuming that ‘maximum temperatures’ were measured and recorded the in the same manner in both year periods. I really doubt that each station had someone standing out in a hot field in the afternoon waiting for the ‘absolute’ maximum one minute maximum to occur. But that is exactly what these resisting platinum wire temperature measurements do. They have basically no thermal mass as old glass thermometers and can record instantaneous maximum (or minimum) temperatures no matter how short the temporal duration of that “maximum” occurs. Much of the new “record temperatures” are merely better more sensitive, and much faster to equalize, temperature measurements. It’s just simple sense that this is actually occurring.
Looking at the map Anthony posted, it appears those stations are evenly distributed as well as being quality assured. However; no doubt that if a selection were taken where there is a higher percentage per area of stations midwest and southeast, compared to fewer distribution percentage of area in the west and northwest, then the data would be weighted toward warmer areas of the country.
However; the people misrepresenting the truth are usually slow to produce how they got their numbers as we frequently see in this field of science.
I find it interesting and telling that Rattus Norvegius can’t be bothered to post a link (requested twice upstream) nor the full email from him and NCDC, yet demands openness and detailed accuracy from me.
Me thinks he’s heading for the troll bin as I don’t have a lot of tolerance for his double standards, especially when he hides behind a fake persona.