Zeke is upset that I made this statement in a story at Fox news:
Is history malleable? Can temperature data of the past be molded to fit a purpose? It certainly seems to be the case here, where the temperature for July 1936 reported … changes with the moment. In the business and trading world, people go to jail for such manipulations of data.
he says:
In the spirit of civility, I would ask Anthony to retract his remarks. He may well disagree with NCDC’s approach and results, but accusing them of fraud is one step too far.
I’d point out that Zeke has his interpretation but nowhere did I say “fraud”. He’s mad, and people don’t often think clearly when they are mad. That’s OK.
Without getting into semantics, I’d like to ask Zeke these simple questions:
- What is the CONUS average temperature for July 1936 today?
- What was it a year ago?
- What was it ten years ago? Twenty years ago?
- What was it in late 1936, when all the data had been first compiled?
We already know the answers to questions 1 and 2 from my posting here, and they are 76.43°F and 77.4°F respectively, so Zeke really only needs to answer questions 3 and 4.
The answers to these questions will be telling, and I welcome them. We don’t need broad analyses or justifications for processes, just the simple numbers in Fahrenheit will do.
Zeke Hausfather says:
January 23, 2013 at 3:45 pm
I notice that you failed to answer any question Anthony actually asked, but let me throw in a few of my own:
Where did the extra significant digit come from?
Why is the adjustment in the same order of magnitude as the claimed signal?
“Mad”, as in “angry” or “insane”?
REPLY: “angry” – Anthony
Whelp, I made an error. This does illustrate the danger of trying to do things too quickly. My code was accidentally still set to use the raw data rather than the homogenized data.
After homogenization, we get:
July 1936: 74.59
July 2012: 75.35
Anthony,
What exactly do you want to know about July 1963? The simple average of the temperature readings of all stations? The best estimate of the full CONUS temperature field? Its not a simple question with a simple answer.
Stark Dickflüssig,
I gave my best estimate of the CONUS average absolute temperature for that month and year. Given that we don’t have a measurement of ever inch of the U.S., but rather a specific set of locations, any attempt at creating an average temperatures will require at least a few assumptions.
Anthony,
I think CRN is somewhat germane. We need a good unbiased estimate of the absolute temperature field, and I would argue that it provides a rather good one, or at least a better estimate than a weighted average of absolute temperatures in 1936.
REPLY: I think you’ve been overthinking – Anthony
I still read it as fraud, only not trying to hide behind more questions which we all know are irrelevant to the intention.
I still find it amazing that glorified weathermen can clearly see the past, and correct it where it’s wrong…
….and they still can’t pick lotto numbers
who fiddles the data? who diddles the chart?
who plays tickle-monster with the temps?
who has soapy showers with statistics?
would the consensus be ‘climate molesters’?
Zeke Hausfather says:
January 23, 2013 at 3:47 pm
As a practicing mining engineer for 15 years in charge of block model generation and economic analysis to substantiate $Millions to $Billions of investment capital, had I performed the data manipulation on assay values you “climate scientists” do on temperature data, I’d certainly be out of a job and, depending on the severity of the situation, perhaps end up in jail. (Some people call it “salting”; others call it “creative accounting”.)
So continue with your “data manipulation” if you wish, but you’re going to have a hard time getting this engineer to believe your conclusions. (And yes, I did read the article by lucia you linked to in your prior post.)
Climate changes over time.
Records of climate change over time.
All under the heading of ‘climate variability’.
Corporate officers have legal responsibilities beyond not commiting fraud.
A corporate officer can certainly go to jail for ‘a mistake’.
Here is an example.
http://online.wsj.com/article/SB10001424052970204443404577052173679627572.html
I watched the data in GISS figd.txt change in Nov. 2009. (remember that GISS & NCDC are essentially the same.)
2006 came from about 6th to joint warmest with 1998 whilst 1934 went from joint warmest to about 6th.
I’m supposed to believe any of this?
Zeke. Does anyone even know what the RAW data is anymore?
DaveE.
Claiming an error margin of just 0.2 degrees – when calculating the CONUS temperature for a month in 1936 – THAT IS EITHER INCOMPETENCE OR FRAUD, take your pick.
Anthony,
There is a good reason why all the major groups (NCDC, GISS, UAH, RSS, Hadley) primarily report anomalies. Absolute temperatures are tricky things, at least when you aren’t able to sample the full field.
GISS has a rather good explanation: http://data.giss.nasa.gov/gistemp/abs_temp.html
Corporate officers can go to jail for other than fraud. They can go to jail for incompetence or just not exercising due diligence. Here is an example.
http://online.wsj.com/article/SB10001424052970204443404577052173679627572.html
Therefore Anthony’s statement implied one of a range illegal activities for corporate officers. Not just fraud.
“Accusing someone of manipulating data, suggesting that they should go to jail, but not uttering the word “fraud” is a trick.”
Commentators on business networks frequently make a similar observation with respect to government accounting verses private sector accounting practices. There is no implication that the folks in the government entities should be jailed – the observation is that there are clearly different standards for government practices and the rules the rest of us live by.
David A. Evans,
To access raw data, browse to one of the NCDC’s FTP folders (e.g. ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2.5/) and click on the files labeled .raw
I also show the raw vs. adjusted results for the U.S. in my post over at the Blackboard: http://rankexploits.com/musings/wp-content/uploads/2013/01/USHCN-adjusted-raw.png
Global raw vs adjusted temperatures can be seen here: http://www.skepticalscience.com/pics/GHCN_RawvAdj.jpg
” In the business and trading world, people go to jail for such manipulations of data.”
This statement may also be construed to highlight the seriousness of climate data manipulation via comparison with an activity which most would recognize as having consequences, whether legal ones or reputation damaging.
Anthony,
They way that NCDC calculates absolute temperatures is by calculating anomalies and adding in a constant absolute temperature for each month, based (I believe) on modeled outputs of a U.S. temperature field for the climate normals period. I’m well versed in the details, however, as I rarely if ever deal with absolute temperatures.
My approach was to try and replicate what NCDC does, but to use the CRN to determine the “true” absolute temperatures for the period in which CRN and HCN overlap. I realize this is somewhat complicated (and perhaps overthinking), but the point is that there is no “simple” way to get an absolute temperature value.
As far as anomalies go, you can see (homogenized) July anomalies here: http://i81.photobucket.com/albums/j237/hausfath/ScreenShot2013-01-23at43154PM_zps3d02490c.png
REPLY: and this exactly illustrates my concern, nobody seems to be able to reproduce the CONUS average temperature for July 1936. Therefore, making claims about “hottest ever” etc are indeed, nothing but hype. NCDC knows full well what they are doing here.
The historical accounts should be in the Monthly Weather Review for 1936. Since I illustrated here how a simple average of the CRN stations in the CONUS came close to the COOP values, I see the point your are trying to make. That said, today’s data is not yesterday’s data. My interest is in the true data from 1936, not a derivative comparison of what it “might” be.
– Anthony
Zeke, The difference between NOAA and USCRN can be as much as plus or minus 4.7F for individual states for a month.
Yet you argue the “trend” is close enough. It isn’t.
For example, in Sept 2012, NOAA was 4.78F warmer than USCRN for Tennessee.
http://sunshinehours.wordpress.com/2012/10/09/uscrn-vs-noaa-september-2012/
Don’t try and argue that NOAA and USCRN know what the temperature in Tennessee was for September 2012.
Zeke, Anthony and Richard, Why is there no longer any discussion of temp. data prior to 1850? Why would the end of “the little Ice age” be the starting point for data in this discussion, when there are plenty of temp. records form around the globe available from the 1770’s on… when thermometers were simple, uniform and accurate. It seems to me that using a data set starting at the low point would be like reading one heart beat of of an EKG starting with a flat line and ignoring the previous beat. Every heart beat is a hockey stick when analyzed this way!
Fudge the data? Where on earth would scientists learn to do that and get rewarded? Answ. First year students in University science lab classes.
Zeke – I agree that there is a fair amount of measurement error in these calculations due to siting and other issues. This is probably even more serious for global temperature averages than the US alone. The measurement errors are sufficiently large that trying to determine causes and effects of various variables would be very trying. Indeed, it makes it quite difficult for the science to be “settled”. It makes it quite difficult to justify raising the prices of food and energy to the poor and middle class, just in case.
I know I did nothing “just in case” the Mayans were right. Dooming poor people to a lower standard of living, based on theories with such high levels of measurement error is bad policy.
Zeke Hausfather:
Following my post addressed to you at January 23, 2013 at 3:56 pm you have answered each of the posts addressed to you from others. It seems that you may have overlooked my question.
I would be grateful if you were to answer my question. To save you scrolling to find it, and for hope of clarification, I rephrase it and expand it here.
Temperatures were measured decades ago so their combined values should be a constant in the absence of altered homogenisation and consolidation methods. Why does the method used to homogenise and consolidate historic temperatures (regional, hemispheric and global) change with time? Which of the used methods can be considered to be correct and why?
Please note that I am not asking about differences between the methods of different teams (e.g. HadCRU and GISS). I am asking about the sequence of different methods used by your organisation.
Richard
Zeke Hausfather,
When you were an undergraduate taking a lab course, what would have happened to your academic career if you had gone back to your lab book and “adjusted” the recorded data so your conclusion matched expectation? I suggest that you would not be working in science now if you had done such a thing. Experimental data is sacrosanct! It can never be altered. If you measure an item as a meter, it is a meter forever (even if your professor set the lab up with altered meter sticks to make sure everyone was staying honest – and yes I would do that!) There can be no adjustment of the past, only traceable, explained in metadata, correction of the present.
Smoothing algorithms should only change current analytical data and should always be traceable to the original data. Too many of the changes have no documentation of the individual adjustments. Most of us understand adjustments are needed to make apples to apples comparisons for sites that have had significant land use changes around them, but that doesn’t seem to be what is happening. The homogenization process seems to just smear the worst errors or site problems across whole regions and time periods rather than to correct for poor or missing data.
Quite frankly, the whole UHI adjustment process as it now happens seems to be counter-intuitive. If the problem is urban encroachment of the site, the correct adjustments would do one of two things to make an apples to apples comparison of the same site at different points in time and development: adjust the current readings down by an amount equivalent to an experimentally determined value of UHI, or adjust the past up by the same amount, though I’d rather the first than the second, and would rather no adjustment at all until a valid experimental valuation of UHI for different types of sites can be done. The way the current adjustment process is done implies that UHI has a cooling effect on the sensors which is just plain crazy. (See the experiment being done at Oak Ridge on siting issues for ideas on how to get at this value!)
Most of us here understand that science is hard work. I suspect that most people working these data sets are trying to do the right thing for the science with the best information they have available. Thus you will rarely hear the fraud word from this site, but you will hear things like “obviously mistaken”, “out of their gourds”, “too full of themselves to see their error”, or even “suffering from mushroom syndrome” and very rarely for some of the most arrogant practitioners of data manipulation and scientific information suppression, “dishonest”. For the most part, I am looking for data that passes the smell test and so far that has been in short supply. Some of the problem is the ever-popular “science by press release” that the MSM picks up and runs with out of all proportion to the underlying data/paper, but then again, some of the abstracts are written with the obvious intent to engender that response. (I’ve been told my abstracts are too dry, so I am probably not one to ask how to correct that problem.)
The problem with this site is not that people don’t understand science, the problem is that they do, and that makes it hard to put “science in the dark” by us. (not implying that NCDC does this, but there is still the little matter of all the little adjustments of the past – especially those that happen between major dataset methodology revisions!)