I decided to do myself something that so far NOAA has refused to do: give a CONUS average temperature for the United States from the new ‘state of the art’ United States Climate Reference Network (USCRN). After spending millions of dollars to put in this new network from 2002 to 2008, they are still giving us data from the old one when they report a U.S. national average temperature. As readers may recall, I have demonstrated that old COOP/USHCN network used to monitor U.S. climate is a mishmash of urban, semi-urban, rural, airport and non-airport stations, some of which are sited precariously in observers backyards, parking lots, near air conditioner vents, airport tarmac, and in urban heat islands. This is backed up by the 2011 GAO report spurred by my work.
Here is today’s press release from NOAA, “State of the Climate” for July 2012 where they say:
The average temperature for the contiguous U.S. during July was 77.6°F, 3.3°F above the 20th century average, marking the hottest July and the hottest month on record for the nation. The previous warmest July for the nation was July 1936 when the average U.S. temperature was 77.4°F. The warm July temperatures contributed to a record-warm first seven months of the year and the warmest 12-month period the nation has experienced since recordkeeping began in 1895.
OK, that average temperature for the contiguous U.S. during July is easy to replicate and calculate using NOAA’s USCRN network of stations, shown below:
The U.S. Climate Reference Network (USCRN) consists of 114 stations developed, deployed, managed, and maintained by the National Oceanic and Atmospheric Administration (NOAA) in the continental United States for the express purpose of detecting the national signal of climate change. The vision of the USCRN program is to maintain a sustainable high-quality climate observation network that 50 years from now can with the highest degree of confidence answer the question: How has the climate of the nation changed over the past 50 years? These stations were designed with climate science in mind. Three independent measurements of temperature and precipitation are made at each station, insuring continuity of record and maintenance of well-calibrated and highly accurate observations. The stations are placed in pristine environments expected to be free of development for many decades. Stations are monitored and maintained to high standards, and are calibrated on an annual basis. In addition to temperature and precipitation, these stations also measure solar radiation, surface skin temperature, and surface winds, and are being expanded to include triplicate measurements of soil moisture and soil temperature at five depths, as well as atmospheric relative humidity. Experimental stations have been located in Alaska since 2002 and Hawaii since 2005, providing network experience in polar and tropical regions. Deployment of a complete 29 station USCRN network into Alaska began in 2009. This project is managed by NOAA’s National Climatic Data Center and operated in partnership with NOAA’s Atmospheric Turbulence and Diffusion Division.
So clearly, USCRN is an official effort, sanctioned, endorsed, and accepted by NOAA, and is of the highest quality possible. Here is what a typical USCRN station looks like:
A few other points about the USCRN:
- Temperature is measured with triple redundant air aspirated sensors (Platinum Resistance Thermometers) and averaged between all three sensors. The air aspirated shield exposure system is the best available.
- Temperature is measured continuously and logged every 5 minutes, ensuring a true capture of Tmax/Tmin
- All stations were sited per Leroy 1999 siting specs, and are Class 1 or Class 2 stations by that siting standard. (see section 2.2.1 here of the USCRN handbook PDF)
- The data goes through quality control, to ensure an errant sensor hasn’t biased the values, but is otherwise unchanged.
- No stations are near any cities, nor have local biases of any kind that I have observed in any of my visits to them.
- Unlike the COOP/USHCN network where they fought me tooth and nail, NOAA provided station photographs up front to prove the “pristine” nature of the siting environment.
- All data is transmitted digitally via satellite uplink direct from the station.
So this means that:
- There are no observer or transcription errors to correct.
- There is no time of observation bias, nor need for correction of it.
- There is no broad scale missing data, requiring filling in data from potentially bad surrounding stations. (FILNET)
- There are no needs for bias adjustments for equipment types since all equipment is identical.
- There are no need for urbanization adjustments, since all stations are rural and well sited.
- There are no regular sensor errors due to air aspiration and triple redundant lab grade sensors. Any errors detected in one sensor are identified and managed by two others, ensuring quality data.
- Due to the near perfect geospatial distribution of stations in the USA, there isn’t a need for gridding to get a national average temperature.
Knowing this, I wondered why NOAA has never offered a CONUS monthly temperature from this new network. So, I decided that I’d calculate one myself.
The procedure for a CONUS monthly average temperature from USCRN:
- Download each station data set from here: USCRN Quality Controlled Datasets.
- Exclude stations that are part of the USHCN-M (modernized USHCN) or USRCRN-Lite stations which are not part of the 114 station USCRN master set.
- Exclude stations that are not part of the CONUS (HI and AK)
- Load all July USCRN 114 station data into an Excel Spreadsheet, available here: CRN_CONUS_stations_July2012_V1.2
- Note stations that have missing monthly totals data. Three in July 2012, Elgin, AZ, (4 missing days) Avondale, PA,(5 missing days) McClellanville, SC, (7 missing days) and set their data aside to be dealt with separately.
- Do sums and calculate CONUS area averages from the Tmax, Tmin, Tavg and Tmean data provided for each station.
- Do a separate calculation to see how much difference the stations with missing/partial data make for the entire CONUS.
Here are the results:
USA Monthly Mean for July 2012: 75.72°F
(111 stations)
USA Monthly Average for July 2012: 75.51°F
(111 stations)
USA Monthly Mean for July 2012: 75.74°F
(114 stations, 3 w/ partial missing data, difference 0.02)
USA Monthly Average for July 2012: 75.55°F
(114 stations, 3 w/ partial missing data, difference 0.04)
============================
Comparison to NOAA’s announcement today:
Using the old network, NOAA says the USA Average Temperature for July 2012 is: 77.6°F
Using the NOAA USCRN data, the USA Average Temperature for July 2012 is: 75.5°F
The difference between the old problematic network and new USCRN is 2.1°F cooler.
This puts July 2012, according to the best official climate monitoring network in the USA at 1.9°F below the 77.4°F July 1936 USA average temperature in the NOAA press release today, not a record by any measure. Dr. Roy Spencer suggested earlier today that he didn’t think so either, saying:
So, all things considered (including unresolved issues about urban heat island effects and other large corrections made to the USHCN data), I would say July was unusually warm. But the long-term integrity of the USHCN dataset depends upon so many uncertain factors, I would say it’s a stretch to to call July 2012 a “record”.
This result also strongly suggests, that a well sited network of stations, as the USCRN is designed from inception to be, is totally free of the errors, biases, adjustments, siting issues, equipment issues, and UHI effects that plague the older COOP USHCN network that is a mishmash of problems that the new USCRN was designed to solve.
It suggests Watts et al 2012 is on the right track when it comes to pointing out the temperature measurement differences between stations with and without such problems. I don’t suggest that my method is a perfect comparison to the older COOP/USHCN network, but the fact that my numbers come close, within the bounds of the positive temperature bias errors noted in Leroy 1999, and that the more “pristine” USCRN network is cooler for absolute monthly temperatures (as would be expected) suggests my numbers aren’t an unreasonable comparison.
NOAA never mentions this new pristine USCRN network in any press releases on climate records or trends, nor do they calculate and display a CONUS value for it. Now we know why. The new “pristine” data it produces is just way too cool for them.
Look for a regular monthly feature using the USCRN data at WUWT. Perhaps NOAA will then be motivated to produce their own monthly CONUS Tavg values from this new network. They’ve had four years to do so since it was completed.
UPDATE: Some people questioned what is the difference between the mean and average temperature values. In the monthly data files from USCRN, there are these two values:
T_MONTHLY_MEAN
T_MONTHLY_AVG
http://www.ncdc.noaa.gov/crn/qcdatasets.html
The mean is the monthly (max+min)/2, and the average is the average of all the daily averages.
UPDATE2: I’ve just sent this letter to NCDC – to ncdc.info@ncdc.noaa.gov
Hello,
I apologize for not providing a proper name in the salutation, but none was given on the contact section of the referring web page.
I am attempting to replicate the CONUS temperature average of 77.6 degrees Fahrenheit for July 2012, listed in the August 8th 2012, State of the Climate Report here: http://www.ncdc.noaa.gov/sotc/
Pursuant to that, would you please provide the following:
1. The data source of the surface temperature record used.
2. The list of stations used from that surface temperature record, including any exclusions and reasons for exclusions.
3. The method used to determine the CONUS average temperature, such as simple area average, gridded average, altitude corrections, bias corrections, etc. Essentially what I’m requesting is the method that can be used to replicate the resultant 77.6F CONUS average value.
4. A flowchart of the procedures in step 3 if available.
5. Any other information you deem relevant to the replication process.
Thank you sincerely for your consideration.
Best Regards,
Anthony Watts
===================================================
Below is the response I got to the email address provided in the SOTC release, some email addresses redacted to prevent spamming.
===================================================
—–Original Message—–
From: mailer-daemon@xxxx.xxxx.xxx
Date: Thursday, August 09, 2012 3:22 PM
To: awatts@xxxxxxx.xxx
Subject: Undeliverable: request for methods used in SOTC press release
Your message did not reach some or all of the intended recipients.
Sent: Thu, 9 Aug 2012 15:22:43 -0700
Subject: request for methods used in SOTC press release
The following recipient(s) could not be reached:
ncdc.info@ncdc.noaa.gov
Error Type: SMTP
Error Description: No mail servers appear to exists for the recipients address.
Additional information: Please check that you have not misspelled the recipients email address.
hMailServer
===============================
UPDATE3: 8/10/2012. This may put the issue to rest about straight averaging -vs- some corrected method. From http://www.ncdc.noaa.gov/temp-and-precip/us-climate-divisions.php
It seems they are using TCDD (simple average) still. I’ve sent an email to verify…hopefully they get it.
Traditional Climate Divisional Database
Traditionally, climate division values have been computed using the monthly values for all of the Cooperative Observer Network (COOP) stations in each division are averaged to compute divisional monthly temperature and precipitation averages/totals. This is valid for values computed from 1931 to the present. For the 1895-1930 period, statewide values were computed directly from stations within each state. Divisional values for this early period were computed using a regression technique against the statewide values (Guttman and Quayle, 1996). These values make up the traditional climate division database (TCDD).
Gridded Divisional Database
The GHCN-D 5km gridded divisional dataset (GrDD) is based on a similar station inventory as the TCDD however, new methodologies are used to compute temperature, precipitation, and drought for United States climate divisions. These new methodologies include the transition to a grid-based calculation, the inclusion of many more stations from the pre-1930s, and the use of NCDC’s modern array of quality control algorithms. These are expected to improve the data coverage and the quality of the dataset, while maintaining the current product stream.
The GrDD is designed to address the following general issues inherent in the TCDD:
- For the TCDD, each divisional value from 1931-present is simply the arithmetic average of the station data within it, a computational practice that results in a bias when a division is spatially undersampled in a month (e.g., because some stations did not report) or is climatologically inhomogeneous in general (e.g., due to large variations in topography).
- For the TCDD, all divisional values before 1931 stem from state averages published by the U.S. Department of Agriculture (USDA) rather than from actual station observations, producing an artificial discontinuity in both the mean and variance for 1895-1930 (Guttman and Quayle, 1996).
- In the TCDD, many divisions experienced a systematic change in average station location and elevation during the 20th Century, resulting in spurious historical trends in some regions (Keim et al., 2003; Keim et al., 2005; Allard et al., 2009).
- Finally, none of the TCDD’s station-based temperature records contain adjustments for historical changes in observation time, station location, or temperature instrumentation, inhomogeneities which further bias temporal trends (Peterson et al., 1998).
The GrDD’s initial (and more straightforward) improvement is to the underlying network, which now includes additional station records and contemporary bias adjustments (i.e., those used in the U.S. Historical Climatology Network version 2; Menne et al., 2009).
The second (and far more extensive) improvement is to the computational methodology, which now addresses topographic and network variability via climatologically aided interpolation (Willmott and Robeson, 1995). The outcome of these improvements is a new divisional dataset that maintains the strengths of its predecessor while providing more robust estimates of areal averages and long-term trends.
The NCDC’s Climate Monitoring Branch plans to transition from the TCDD to the more modern GrDD by 2013. While this transition will not disrupt the current product stream, some variances in temperature and precipitation values may be observed throughout the data record. For example, in general, climate divisions with extensive topography above the average station elevation will be reflected as cooler climatology. A preliminary assessment of the major imapacts of this transition can be found in Fenimore, et. al, 2011.
While I agree that care is required comparing apples and oranges (e.g. these two networks), the disparity doesn’t surprise me at all. Whilst the likes of Nick Stokes et al can hand wave about anomalies and adjustments dealing with this, the reality is this delta – 2.1 deg F – is not a fixed delta but varies with time, and the reality is that the various adjustments and statistical tricks used to generate the temperature series can only remove a percentage of this error.
Which is why I’ve argued that the REAL confidence intervals – even for the anomalies – is some large fraction of that 2.1 F – probably of the order of 1F or so (and I mean 1-sigma here). You can come up with clever algorithms that hide this error, but the corrections are imperfect and arguments will always continue about how effective they truly are. The only realistic way to deal with that today is to widen the CIs.
Hmm, well I would have said that using the old network to compare this year’s temperatures with temperatures gathered in the past /using the old network/ would be a more accurate comparison than comparing old network temps from the past with new network temps from the present.
I applaud the work and thought that went into the new network as described. However, comparing data from that network with data collected from the old network is apples and oranges. And I agree that the new network data is/should be more accurate than the old. It just isn’t a good comparison. I have other thoughts on Global Warming but that isn’t the subject of this discussion.
[SNIP: Yes it is OT. Please submit to Tips and Notes or some other more appropriate thread. -REP]
Anthony, you made one glaring error – you didn’t apply any adjustments to that dataset!! Once you do that, the average temps will come out to around 78.4F, warmer than it has been for 3500 years!!
Could you work out the peak temperature in 1936 using the class 1/2 stations of the time (if known)? Then there would be a better basis for comparison.
I don’t understand why Nick, who is described above as a “team” member, has not already done this analysis and posted it. /sarc off
Funnest and most satisfying thing I’ve read in quite awhile. Thanks Anthony!
Is there any way we can compare the sites in USCRN with sites in nearby areas in earlier years?
At least then we could look at the change in temps, and compare to the change given by USHCN.
Oh geez … who’s in charge over there at NOAA?
Are they operating ‘headless’ (like “Mike the Headless Chicken”)?
Anthony,
Looking at your spreadsheet I noticed some stations have their temperature data incorrectly assigned to another station with the same base name. It only occurs when one station has a single digit appended while the other one has two digits appended. Your spreadsheet is doing an ascii sort while the CRN folder structure lists common names with appended digits by numerical value. All the data is present in the spreadsheet, so it doesn’t affect the column figures, but in these few cases it is assigned to the opposite station with the same name. It’s trivial in most cases. Just thought I’d give you a heads-up.
Here are the ones I found in the spreadsheet which are reversed from the folder structure.
GA Newton 8 W
GA Newton 11 SW
NC Ashville 8 SSW
NC Ashville 13 S
NE Lincoln 8 ENE
NE Lincoln 11 SW
REPLY: I thought I had dealt with this sorting issue early on, but I’ll check again. You are right, it doesn’t affect the outcome – Anthony
Thanks Anthony!
Just first rate work and it is appreciated. Also speaking personally, it is just great that you provided, essentially, instructions for how you derived your work so each of us can replicate these results going forward and don’t have to rely on the official and all too often, inaccurate results trumpeted in the media and by alarmists.
[snip fake email address, proxy server, policy violation]
Anthony,
What is the difference between the average and the mean in the way you do your calculations? Is average obtained by dividing by n-1 and mean by n?
If you include error bars, I don’t see that the difference between 1936 and 2012 for July would be significant. Adjusting temperatures for Tobs or any other type should increase the error bars and those should always be reported, even in a newspaper although if the newspaper decides to leave those off that is their call. But a real scientist would be giving the media the error bars.
Meanwhile July in the UK was 1.0C colder than average, This does not seem to have made headline news at the BBC. Wonder why?
http://notalotofpeopleknowthat.wordpress.com/2012/08/09/uk-weather-reportjuly-2012/
Actually I meant the opposite: did you get mean by dividing by n-1 giving a slightly larger number and average by dividing by n, giving a slightly smaller number?
Great work, Anthony! Perhaps your “monthly feature using the USCRN data” could be compiled and listed in your reference pages…
> Temperature is measured continuously and logged every 5 minutes, ensuring a true capture of Tmax/Tmin
Is this 5 minute data available? I could only find references to hourly data for USCRN:
“Each USCRN station has three thermometers which report independent temperature measurements each hour. ”
http://www.ncdc.noaa.gov/crn/elements.html#temp
REPLY: It is not available AFAIK, but this document might shed some light on the 5 minute data and how it is used: http://www1.ncdc.noaa.gov/pub/data/uscrn/products/hourly02/README.txt
-Anthony
Nick Stokes has a good point: if you are looking for trends, you need to use the data base you already have. The new system cannot be compared to old * for purposes of trend analysis*. But for a global temperature as of now, the new system is perfectly fine, thank you.
REPLY: I’m not looking at trends, just a single monthly number to compare to the monthly number in the NOAA press release. – Anthony
@ur momisugly Nick Stokes
Nick,
Is the lapse rate really identical at every spot on the planet and is it really linear such that a site that is 100 m higher in Colorado between the two networks will have the same deltaT as a site in New Orleans that is 100 m higher?
At any rate, you are correct that this has to be taken into account and that the difference in temp. is probably much smaller than the 2.1 F Anthony was rightly upset over. However, even the 0.2 degrees difference is exactly the same as the difference between 1936 and 2012. Plus you need to add in ALL of the proper errors to get error bars.
My problem is that “scientists” go along with people like Borenstein and deliberately allow scary misrepresentations to be made. This further exacerbates the distrust that skeptics rightly have of the politicization and hyperbole that have become so common in the climate debate.
Paul Homewood says:
August 9, 2012 at 6:58 am
Meanwhile July in the UK was 1.0C colder than average, This does not seem to have made headline news at the BBC. Wonder why
Paul,
Because it is not news when 15 old ladies are helped across a busy street, but it is when one young girl is leered at in the subway.
There’s observation bias, recording bias and reporting bias. Bad things – or “potentially” bad things, get the benefit of all three.
Anthony, with all the work you’re doing on this would a new temperature record be appropriate? Perhaps the WASSP (WAtts Surface Stations Project) Temperature?
Anthony,
If your download process for the new network is correct,
the erroneous July press release may reveal the reason why there has
not been a switch to the new network. We would have “the coldest
[pick a month] on record”. I predict a February switch.
http://www.youtube.com/watch?v=2SlwV7mtsmw
Anthony’s contribution here is to flag the NOAA announcement as coming not from the USCRN, their Climate Reference Standard.
Now it is true that USCRN doesn’t have a long enough history to establish a trend. However, USCRN is long enough to SUPPORT a claim that this is the HOTTEST July. For if it is the hottest July since 1936, then it must be the hottest July since USCRN was set up.
So is 2012 July’s USCRN data hotter than all other years of July USCRN? I’m not asking anyone to do the work. Someone needs to answer it and NOAA is paid to do it.
“If NOAA has issues with the presentation, let them put out a CONUS monthly value”
Why should they use your method? Because you say so?
REPLY: Nooo, they can use any method they want, But they should produce a monthly value. The fact that they aren’t is the most telling part. – Anthony
Comparing two different measurement networks in the way that this post does is so obviously wrong that it damages your credibility.
REPLY: And yet USCRN exists for that very purpose, to compare:
http://www.ncdc.noaa.gov/crn/programoverview.html
Performance Measures
The USCRN was established to help detect climate change in the United States. In order to assess the performance of the network in addressing this goal a performance measure (PM) was developed. This PM is an assessment of how closely the current and past configuration of the network captures the “true” national temperature and precipitation signal as defined by an area-averaged time series of annual temperature and precipitation derived from 4000 U.S. Cooperative Observer Program (COOP) Network stations scattered across the continental U.S. The configuration of the USCRN for a given point in time is used to select stations from the 4000 COOP station network, one station for each operating USCRN site (the one physically closest in location), and the time series derived from these stations is compared, statistically, to the time series derived from all 4000 stations. The result is a “variance explained” that measures how closely the “USCRN” time series follows the “true” time series.