UPDATE: As I originally mentioned at the end of this post, I thought we should “give the benefit of the doubt” to GISS as there may be a perfectly rational explanation. Steve McIntyre indicates that he has done an analysis also and doubts the other analyses:
I disagree with both Luboš and David and don’t see anything remarkable in the distribution of digits.
I tend to trust Steve’s intuition and analysis skills,as his track record has been excellent. So at this point we don’t know what is the root cause or even if there is any human touch to the data. But as Lubos said on CA “there’s still an unexplained effect in the game”.
I’m sure it will get much attention as the results shake out.
UPDATE2: David Stockwell writes in comments here:
Hi,
I am gratified with the interest in this, very preliminary analysis. There’s a few points from the comments above.
1. False positives are possible, for a number of reasons.
2. Even though data are subjected to arithmetric operations, distortions in digit frequency at an earlier stage can still be observed.
3. The web site is still in development.
4. One of the deviant periods in GISS seems to be around 1940, the same as the ‘warmest year in the century’ and the ‘SST bucket collection’ issues.
5. Even if in the worst case there was manipulation, it wouldn’t affect AGW science much. The effect would be small. Its about something else. Take the Madoff fund. Even though investors knew the results were managed, they still invested because the payouts were real (for a while).
6. To my knowledge, noone has succeeded in exactly replicating the GISS data.
7. I picked that file as it is the most used – global land and ocean. I haven’t done an extensive search of files as I am still testing the site.
8. Lubos relicated this study more carefully, using only the monthly series and got the same result.
9. Benfords law (on the first digit) has a logarithmic distribution, and really only applies to data across many orders of magnitude. Measurement data that often has a constant first digit doesn’t work, although the second digit seems to. I don’t see why last digit wouldn’t work, and should approach a uniform distribution according to the Benford’s postulate.
That’s all for the moment. Thanks again.
This morning I received an email outlining some work that David Stockwell has done in some checking of the GISS global Land-Ocean temperature dataset:
Detecting ‘massaging’ of data by human hands is an area of statistical analysis I have been working on for some time, and devoted one chapter of my book, Niche Modeling
, to its application to environmental data sets.
The WikiChecks
The WikiChecks Site Says:
‘Managing’ or ‘massaging’ financial or other results can be a very serious deception. It ranges from rounding numbers up or down, to total fabrication. This system will detect the non-random frequency of digits associated with human intervention in natural number frequency.
Stockwell runs a test on GISS and writes:
One of the main sources of global warming information, the GISS data set
You can actually run this test yourself, visit the WikiChecks web site, and paste the URL for the GISS dataset
http://data.giss.nasa.gov/gistemp/tabledata/GLB.Ts+dSST.txt
into it and press submit. Here is what you get as output from WikiChecks:
GISS Frequency of each final digit: observed vs. expected
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Totals | |
| Observed | 298 | 292 | 276 | 266 | 239 | 265 | 257 | 228 | 249 | 239 | 2609 |
| Expected | 260 | 260 | 260 | 260 | 260 | 260 | 260 | 260 | 260 | 260 | 2609 |
| Variance | 5.13 | 3.59 | 0.82 | 0.08 | 1.76 | 0.05 | 0.04 | 4.02 | 0.50 | 1.76 | 17.75 |
| Significant | * | . | * | ||||||||
 |
 |
 |
 |
 |
|
 |
 |
 |
|
| Statistic | DF | Obtained | Prob | Critical |
|---|---|---|---|---|
| Chi Square | 9 | 17.75 | <0.05 | 16.92 |
Stockwell writes of the results:
The chi-square test is prone to produce false positives for small samples. Also, there are a number of innocent reasons that digit frequency may diverge from expected. However, the tests are very sensitive. Even if arithmetic operations are performed on data after the manipulations, the ‘fingerprint’ of human intervention can remain.
I also ran it on the UAH data and RSS data and it flagged similar issues, though with different deviation scores. Stockwell did the same and writes:
The results, listed from lowest deviation to highest are listed below.
RSS – Pr<1
GISS – Pr<0.05
CRU – Pr<0.01
UAH – Pr<0.001
Numbers such as missing values in the UAH data (-99.990) may have caused its high deviation. I don’t know about the others.
Not being familiar with this mathematical technique, there was little I could do to confirm or refute the findings, so I let it pass until I could get word of replication from some other source.
It didn’t take long. About two hours later, Lubos Motl, of the Reference Frame posted his results obtained independently via another method when he ran some checks of his own:
David Stockwell has analyzed the frequency of the final digits in the temperature data by NASA’s GISS led by James Hansen, and he claims that the unequal distribution of the individual digits strongly suggests that the data have been modified by a human hand.
With Mathematica 7, such hypotheses take a few minutes to be tested. And remarkably enough, I must confirm Stockwell’s bold assertion.
But that’s not all, Lubos goes on to say:
Using the IPCC terminology for probabilities, it is virtually certain (more than 99.5%) that Hansen’s data have been tempered with.
To be fair, Lubos runs his test on UAH data as well:
It might be a good idea to audit our friends at UAH MSU where Stockwell seems to see an even stronger signal.
…
In plain English, I don’t see any evidence of man-made interventions into the climate in the UAH MSU data. Unlike Hansen, Christy and Spencer don’t seem to cheat, at least not in a visible way, while the GISS data, at least their final digits, seem to be of anthropogenic origin.
Steve McIntyre offered an explanation in the way rounding occurs when converting from Fahrenheit to Centigrade, but Lubos can’t seem to replicate the same results he gets from the GISS data:
Steve McIntyre has immediately offered an alternative explanation of the non-uniformity of the GISS final digits: rounding of figures calculated from other units of temperature. Indeed, I confirmed that this is an issue that can also generate a non-uniformity, up to 2:1 in the frequency of various digits, and you may have already downloaded an updated GISS notebook that discusses this issue.
…
I can’t get 4,7 underrepresented but there may exist a combination of two roundings that generates this effect. If this explanation is correct, it is a result of much less unethical approach of GISS than the explanation above. Nevertheless, it is still evidence of improper rounding.
Pretty strong stuff, but given the divergence of the GISS signal with other datasets, unsurprising. I wonder if it isn’t some artifact of the GISS Homogenization process for surface temperature data, which I view as flawed in its application.
But let’s give the benefit of the doubt here. I want to see what GISS has to say about it, there may be a perfectly rational explanation that can be applied that will demonstrate that these statistical accusations are without merit. I’m sure they will post something on RC soon.
Stay tuned.
I’ve been wondering when some of the forensic accounting anti-fraud techniques would get applied, but only knew enough about them myself to know that they exist.
First, Steve and Ross applied stats ideas from econometrics and found problems.
Then Anthony applied good old amateur science and found problems.
Now Stockwell has applied stats ideas from auditing and found problems.
I predict that Stockwell will be treated as well by the establishment as Anthony, Steve and Ross have been… but then that is an easy prediction to make.
The checking and double checking of scientific data is how basic science is done. Even if the source of the statistical anomaly turns out to be methodological rather than human malice, it would help to correct and eradicate the error. Nice work, everyone.
<blockquote cite="Using the IPCC terminology for probabilities, it is virtually certain (more than 99.5%) that Hansen’s data have been tempered with."
Tempered or tampered? I suppose either could work in this case.
Great stuff!
BTW, it’s suspicious that Hansen doesn’t sue people who assert that he changes numbers to support his crumbling theory. Me included. The only reason NOT to sue? Discovery.
Hasn’t someone done something similar with the direction of adjustments made to the temperature record and concluded that the adjustments demonstrate a clear pattern that isn’t random?
Stockwell’s work is impressive. Understanding the essence of the process is beyond my pay grade, but the results grab my attention. Assuming the underlying analysis is tight, perhaps a formal peer-reviewed paper is warranted on this.
wow….I can see certain sectors of the blogosphere going psychotic over this….
Management?
I’d suggest using the word manipulation.
(Rounding up? who’d have guessed?)
Not to worry. This datum will be ‘corrected’in the future so as to produce or increase a warming trend, as has been the NASA SOP.
“I want to see what GISS has to say about it”
Judging from Hansen’s past responses to things, it might be a long wait. He doesn’t seems to want to respond to “court jesters” (his terminology, not mine) who question his work.
This is a very interesting way of detecting forgery of data. I’ve tried it on my data in the past. I then tried to edit some data introducing “humanized” random data. I was shocked with the results!
Ecotretas
What happens if you start with the “raw” data? That is, the completely unadjusted USHCN etc.?
A few decades ago there was a flurry of activity when people looked at books of logarithm tables and noticed there was more wear on first pages (e.g. logs of 1-2) than on the final pages (e.g. logs of 9-10). That quickly expanded into the observations that many more numbers start with 1 than 9. While the studies were fascinating, they results were quite believable. Any number that had some multiplication behind it tended to fall into that sort of pattern. The logarithms of various number sets were more likely to be evenly distributed.
I bring that up because from what I recall, the number distributions were quite nice and even without untoward spikes. (Of course if the numbers were the final digit of prices in stores, 8, 9, and 0 are overrepresented, and of course, that has an anthropogenic basis!)
The final digit distribution above is quite worrisome. If it were not anomaly data, I’d suggest that perhaps there were transcription errors where ‘7’ was copied as ‘1’, but that’s not the case here.
The two outlier years are also worrisome, but I’d expect that there should be some sequences that manage to trip the alarm – there should be a similar number of bad years in the other data sets for other years. If there are not, that would be a serious sign that something is wrong with GISS data. Of course, the satellite data sets are too short, so HadCrut may be the best comparison.
Okay, okay, it was four decades ago….
I guess as said, rounding errors, the relatively small size of the sampled set and other things in the pipeline may emerge as a reasonable explanation.
I think the question needs to be put to bed, because it does seem to some (including me) that a clear confirmation bias is used in some of the less rigorous reporting that goes on and gets promoted, I mean just spin with the media.
However a suspicion that this could extend to the data itself could be very harmful, it needs to be cleared up. We need a clear delineation of the process from sampling, to weighting, to reporting. It will help cut through a lot of wasteful doubt.
Congrats to Mr Watts for being part of the process for doing just that ( and winning best Blog? 🙂 )
Hard for James Hansen to avoid this one. Any rational person would demand an explanation and be right to do so.
This is huge. Truly the best science blog.
As Steve McIntyre pointed out to Lubos when he wrote about this, an unequal distribution of final digits is expected when converting between Fahrenheit and Celsius.
What are you going to do when you lose the argument, lie and cheat.
What’s wrong with this statement — Pay more in taxes to government, so government scientists can pretend to control the weather. Isn’t this statement an obvious to everyone conflict of interest?
I never understood what is so complicated about running a summation program on tables of numbers that the process had to be guarded as if it were a top secret nuclear weapon or some such. Wouldn’t it produce much better science if the process were completely open?
So give me one good reason why this entire analysis could not all be done under the publics’ gaze. Does the secrecy not just reek with phony and faked.
Here’s _one_ rational explanation: examine enough data sets and by pure chance some will appear to have been tampered with. Lubos says his significance is 99.5%; I’m not sure where he gets that. According to the WikiChecks data presented above, the Chi square probability is around 0.05, meaning one in 20 data sets will throw up something like this. So how many data sets produced by Hansen have been examined in this way, and how many of these appear ‘suspect’?
What the………..?
I hope there is a sound explanation, if not Hansen and team must go.
Lubos says the 40 out of 10,000 had these properties
Who is going to run this on the Mauna Loa CO2 record?
Tarpon said:
“So give me one good reason why this entire analysis could not all be done under the publics’ gaze. Does the secrecy not just reek with phony and faked.”
I completely agree. The whole point of doing scientific experimentation is to record data that can be corroborated by anyone using the same experimental techniques. Any process of science that has to be hidden is by definition fraud, and this analysis further proves it.
REPLY: It is not proven to be fraud, let’s not use the word. Give GISS a chance to respond and see what other issues may arise. – Anthony
I recreated your results on wikichecks. Then I stripped all the example text and years from the file and resubmitted. After all, we are pretty sure nobody has manipulated the digits in the years. When I resubmitted the stripped down version of the data, the probability of manhandling was reduced.
I’m no expert on this, so I could be doing something wrong without realizing.
Watts, McIntyre, McKitrick, Motl and Stockwell, watch your backs. [snip – none of that talk here]
:(… actually, i do not understand the “skeptics”. You have the raw data, you have GISS code, you can check it. Do it.
I do not know, there is one temperature series that had (has?) really big problems. The trends were completely wrong. You still know that, don’t you? It is the UAH temperature series. The UAH data was totally flawed. And you are not skeptical about their data today? You do not have their source code.
REPLY: I’d point out that FORTRAN code provided by GISS is so antiquated and so environment specific, that to my knowledge, nobody that has tried, has been able to get it to run. – Anthony
Here is something that might be worth placing on a bumper sticker:
“The Debate is Over: Audit Hansen”
http://icecap.us/images/uploads/Stationdropout.jpg
ICECAP blog has an interesting story documenting how 2/3rds of the ground stations dropped out or were kicked out of the system around 1990, most of which were rurally located, at the time that global measured temperatures started averaging higher….
SM on Climate Audit posts that he sees no problem with the datta distribution.
Doesn’t GISS rely heavily on manually entered data from volunteer reporting stations? Could inaccurate daa entry there create an anomoly that would persist through the aggregation process?
I think someone needs to go back to the raw weather station data and see if there is evidence of data manipulation there.
@Anthony
oh yeah, evil Fortran. What do you think the researches do @GISS? Rewriting the code in a new fancy programming language every year? Of course, there are new, better languages now. Furthermore, the code of scientists is often not the best. It has to work. Nothing more.
Is the clearclimatecode.org project still running? They started to re-implement the code in Python. IMHO, that is a good project that cooperated with GISS and tried to help everybody. I hope it is still running.
I suspect that the problem is down to rounding errors. With the urban heat island growth, the rural station removal and a solar Grand Maximum on their side, they probably would have had no reason to fudge the record. I think it’s great that you’re holding them to a high standard, however.
(Above, I meant the rounding ‘process,’ not ‘errors.’)
Anthony,
An interesting post but I think a reference to Steve Mc conversion hypothesis near the top of the artical would be appropriate because these kinds of stories will get blown out of proportion on the blogosphere if the caveats are not make extremely clear.
Mike Lorrey:
Yes that station dropout corresponding to warmer temperatures is widely known. Steven McIntyre has brought it up before and it comes up from time to time in most climate related blogs. The station data are still there, you can get the monthly data over the Internet. NOAA never has responded, as far as I know, as to why they were dropped from the record.
I have wondered why nobody has attempted to “recover” the station data to see if it changes those temperatures. It seems like it could be put to rest easily enough, but nobody ever had.
For what I remember, an equal distribution of digits in a numerical signal is the sign of a homogenously distributed random process, isn’t it ? For example: if a signal is, say, periodic with amplitude oscillating between 3.5 and 4.2, what will be the chance to get a figure ending with .3?
As was mentioned, there is in addition the roundup errors associated with switching from F to C.
I know this is going to come across as negative but that test needs very special care. I just plugged in the beaten lengths from some 1999 horse races for the first listed 300 horses and got that the data were managed. The values were limited to ‘9’ meaning that 9 is really 9+. The rounding part was correct — the values were rounded to integers. Outside of that, no manipulation was done. So, apparently, any data obtained from an instrument that steps and limits the output, like say a digital voltmeter, or even something like trunc(tanh(x)*10+0.5)) is going to come across as ‘managed’.
By ‘special care’ it means it should only be applied to data that are simply tabulated like they are in financial records. ANY processing of the data beyond this is going to be flagged as ‘managed’, which, in a sense, is correct but hardly ‘faked’.
data
When I plug in the values (still rounded but not limited to 9) I get:
When I plug in the finish positions (which don’t have many zeroes) but not managed at all I get:
I have no great love of Hansen or GISS, but rather than accusing them of deliberate falsification, I’ll wait for proof. Showing the data has likely been manipulated is not the same thing as showing that it was for deceptive purposes.
.
By the way . . . congrats on the victory, Anthony!
Hi,
I am gratified with the interest in this, very preliminary analysis. There’s a few points from the comments above.
1. False positives are possible, for a number of reasons.
2. Even though data are subjected to arithmetric operations, distortions in digit frequency at an earlier stage can still be observed.
3. The web site is still in development.
4. One of the deviant periods in GISS seems to be around 1940, the same as the ‘warmest year in the century’ and the ‘SST bucket collection’ issues.
5. Even if in the worst case there was manipulation, it wouldn’t affect AGW science much. The effect would be small. Its about something else. Take the Madoff fund. Even though investors knew the results were managed, they still invested because the payouts were real (for a while).
6. To my knowledge, noone has succeeded in exactly replicating the GISS data.
7. I picked that file as it is the most used – global land and ocean. I haven’t done an extensive search of files as I am still testing the site.
8. Lubos relicated this study more carefully, using only the monthly series and got the same result.
9. Benfords law (on the first digit) has a logarithmic distribution, and really only applies to data across many orders of magnitude. Measurement data that often has a constant first digit doesn’t work, although the second digit seems to. I don’t see why last digit wouldn’t work, and should approach a uniform distribution according to the Benford’s postulate.
That’s all for the moment. Thanks again.
There may be bad at the root of this, but
perhaps not. I have a few patents in process
on related data conversion items.
First I agree with the general concept of digit analysis
to see if there is a Gaussian distribution. If there is
not a Gaussian, well-developed techniques can be
used to point out improbability of number sets.
But, this ONLY works if the underlying data has a
true Gaussian distribution! Is that the case here? We
do not know.
Here are two common (in this realm of science) counter
examples.
1) Imagine the actual temperature measurement was done
in a synthetic unit. In this example, the unit is Franks, and
a Frank is equal to 4 normal Fahrenheit degrees.
No matter how you convert from Franks to Fahrenheits,
you will not get a Gaussian distribution of digits on the
Fahrenheits scale.
The distribution you do get will be convolved with a product
of the prime factor decomposition of the relationship of the
scales. In this example, the only relative prime is 2.
Pure mathematics in this area are “Lattice Analysis” and
“Spanning Spaces”. I could (and have) gone on for hundreds
of pages, but I will stop here.
2) Instead of a formulaic conversion from Franks to Fahrenheit,
a look up table is used. The look up table is constructed so
as to have a minimized maximum error across the whole table.
But the whole table is not used! Only part of the table is used,
and that part does not have minimized error.
In the limit, where only one entry in the LUT is used, large errors
are introduced, and the distribution is a discrete, like a Dirac
delta function.
This problem has a whiff of fraud, but also smells strongly of
incorrect application of differential mathematics where the problem
is discrete in nature, and thus discrete calculus must be used.
Application of differential techniques to a discrete discontinuous
data set _always_ gives the wrong result.
Another thing that might be a factor. In some cases a data value is missing in the raw data stream and a value is calculated to “fill” the missing value. There might be something in that calculation of fill values that tends to favor certain outcomes.
Congratulations on the ‘Best Science Blog’ Anthony. An award well earned.
This article re the adjusting of temperature data if verified means that the suspicions of a lot of people have voiced without more than anecdotal evidence may well be true. We have been lied to, consistently, by the pro AGW camp.
Now if someone with the right mathematical and accounting know how could confirm the audit trail of ‘carbon trading’ funds back to the guilty parties lining their pockets on the back of ‘Anthropogenic Climate Change’………
@frank Perdicaro (15:18:07)
Well put. I’d venture to say again almost ANY manipulation (say multiplication by 2 which yields only even values) will likely be flagged. The fact that the others are also flagged should be a clue. I’m willing to bet that the output of every D/A converter will also get flagged particularly if the output is in the non-linear range.
Even if it were absolute proof of faking at GISS it wouldn’t count much in public opinion because of its technical quality. Think about it. If people ignore increasing ice fields and stalled temperature rises and all that, why would the distribution of digits in reported values suddenly convince them? It even comes across as a desperate cry.
Doesn’t GISS “fill in the blanks”. i.e. Calculate values for missing months based on other years/seasons in the dataset?
Since these filled in values would have a relationship with other months it might skew the results of the test.
“” Ric Werme (13:36:21) :
A few decades ago there was a flurry of activity when people looked at books of logarithm tables and noticed there was more wear on first pages (e.g. logs of 1-2) than on the final pages (e.g. logs of 9-10). That quickly expanded into the observations that many more numbers start with 1 than 9. While the studies were fascinating, they results were quite believable. Any number that had some multiplication behind it tended to fall into that sort of pattern. The logarithms of various number sets were more likely to be evenly distributed. “”
Well I wouldn’t be so quick to jump to conclusions of malfeasance.
Log(2) =0.301 while log (9) is 0.954 or 1-0.046
So the gap between 1 and 2 is about 6.6 times the gap between 9 and 10 in the log tables so I would expect about 6.6 times as much usage of the pages between 1 and 2, compared to between 9 and 10.
No conspiracy at all.
If the GISS temperature adjustments were simply done 1) in public and/or 2) enough information was supplied so that they could be replicated, none of this ancillary statistical analysis (that may or may not be significant/relevant) would need to be done.
It isn’t incumbent on any one to prove there is a problem with the data. It is up to the “scientist” that generates to the data to do it in such a way that the question of a problem never comes up.
Of course this same analysis could be done at the individual station level on the raw data.
We already know that data from Dew Line stations were fabricated:
http://wattsupwiththat.com/2008/07/17/fabricating-temperatures-on-the-dew-line/
Watt about other stations?
How about Mohonk Lake:
http://wattsupwiththat.com/2008/09/17/calling-all-climate-sleuths/
The following is not intended to be, nor should it be taken, as legal advice. No attorney-client relationship is established by these comments. Any person desiring legal advice should consult an attorney.
Fraud: It is well to be very careful in using that word to describe another. A false accusation of fraud could be grounds for a defamation lawsuit. I say *could* because there are a lot of factors involved.
Fraud generally also requires an intent to deceive, which is very difficult to prove. Different jurisdictions have slightly different wordings and requirements.
Please, as Anthony stated, let us wait for more analysis and confirmed results.
Roger E. Sowell, Esq.
Marina del Rey, California
I confidently predict that this will turn out to be a non-story. As our host is a meteorologist and interested in matters climatic, and on the day that WUWT was awarded Best Science Blog, can we expect some comment on the news that the American Meteorological Society has honoured the custodian of the GISS dataset with its highest commendation, the href=”http://www.nasa.gov/centers/goddard/news/topstory/2009/hansen_ams.html”> Carl-Gustaf Rossby Research Medal.
Newsworthy, surely?
I confidently predict that this will turn out to be a non-story. As our host is a meteorologist and interested in matters climatic, and on the day that WUWT was awarded Best Science Blog, can we expect some comment on the news that the American Meteorological Society has honoured the custodian of the GISS dataset with its highest commendation, the Carl-Gustaf Rossby Research Medal.
Or not?
So who has done an analysis on the digits of (Pi) or (e) to see if they have been hand manipulated.
Just asking !
The text in the article says a problem in the data is “a deficiency of zeros and ones” (oh God no!) and then shows a chart showing an excess of zeroes and ones compared to the arbitrary ‘expected’ amounts.
Since temperature measurements are not actually random numbers could someone explain why the data should contain an even distribution of all digits 0-9? Seriously, try to explain this because I’m in the mood for a good belly laugh.
First, you won — Congratulations on the ‘Best Science Blog’ — Anthony. Hats off for the great job you do running this blog.
It’s way past time that the demands be made by the community that the process be opened up for all to see. It’s tables of numbers, not nuclear secrets here. If there is to be any credibility left in the process of establishing facts, the process, methods, software and the raw data needs to be opened up to scrutiny. Only then can we all agree on what the real facts are.
You cannot hope to make policy on this sort of crappy foundation, nor do good science. The point of having a clearing house like this is so the data that is ultimately produced can be relied upon. Whether they are actually fiddling the numbers or not, is not the question, it’s why would they behave the way they do if they weren’t?
A large number of 4’s are rounded up to 7’s.
Doesn’t sound like a big deal except we are only dealing with tenths of a degree here in the first place.
The rate that temps have increased is only 0.064C each year. Over 100 years, the increase is only 0.64C (which implies that global warming may not be such a big problem.)
Oops, I’ve rounded the 0.64C over 100 years to 0.67C and now the increase is 0.7C over 100 years (global warming is now within the range of errors of the models again).
The global warming models are projecting temps to increase at 0.2C per decade (the only number that gets temps close to +3.0C by 2100).
The actual observations since 1980 show an increase of only 0.14C per decade which indicates temps will increase less than 2.0C by 2100 – oops, I’ve now rounded that up to 0.17C per decade and viola, the global warming models are correct again and, yes, there will be dangerous global warming of +3.0C by 2100.
Rounding of such small numbers, especially the 4’s up to 7’s makes a huge impact in the long-term trends of 100 to 200 years that we are talking about here.
You need to bury this post, this argument, this “analysis” and pretend like it never happened. This silly effort could unravel years of effort in confronting the warmers.
Candy for number crunchers. Excuse the random excerpts.
————————————————————-
HEAT STORED BY GREENHOUSE GASES
When investigating the propagation of energy, we must take into account the science of thermodynamics which allows us to predict the trajectory of the process; and the science of heat transfer to know the modes by which energy is propagated from one system to other systems.
To understand heat transfer we have to keep in mind that heat is not a substance, but energy that flows from one system toward other systems with lower density of energy.
http://biocab.org/Heat_Storage.html
Note: If we take the last report from Mauna Loa for this algorithm, the mass of CO2 would be 0.00069 Kg. The change of temperature would be 0.0062 °C. The difference between the ΔT produced by 0.000614 Kg and the ΔT produced by 0.00069 Kg of CO2 is negligible (0.0062 – 0.00553 = 0.00067).
To cause a variation in the tropospheric temperature of 0.52 °C (average global temperature anomaly in 1998; UAH) required 1627.6 ppmv of CO2, a density of atmospheric CO2 which has never been recorded or documented anywhere in the last 420,000 years. (Petit et al. 1999)
The total change in the tropospheric temperature of 0.75 °C was given for the duration of one minute of one year (1998) (UAH); however, CO2 increased the tropospheric temperature by only 0.01 °C. We know now that 1934 was the warmest year of the last century. Where did the other 0.74 °C come from? Answer: it came from the Sun and from the remnants of supernovas.
CHANGE OF THE TROPOSPHERIC TEMPERATURE BY SOLAR IRRADIANCE
Planet Earth would not be warming if the Sun’s energy output (Solar Irradiance) was not increasing. Favorably, our Sun is emitting more radiation now than it was 200 years ago, and so we should have no fear of a natural cycle that has occurred many times over in the lifetime of our Solar System.
When the concentration of atmospheric carbon dioxide increases, the strong absorption lines become saturated. Thereafter its absorptivity increases logarithmically not linearly or exponentially; consequently, carbon dioxide convective heat transfer capacity decreases considerably.
ALGORITHM FOR METHANE (CH4)
ΔT = 0.00013 cal-th /0.0012 Kg (533.3 cal/Kg*°C) = 0.00013 cal / 0.64 cal*°C = 0.0002 °C
Consequently, Methane is not an important heat forcing gas at its current concentration in the atmosphere.
THE CASE ON 14 APRIL 1998 (RADIATIVE “FORCING”)
When we introduce real standards and apply the proper algorithms, the temperature increase caused by CO2 is no more than 0.1 K.
CO2 SCIENCE: THE CASE ON JUNE 22, 2007 (RADIATIVE “FORCING”):
A common error among some authors is to calculate the anomaly taking into account the whole mass of atmospheric CO2, when for any calculation we must take into account only the increase of the mass of atmospheric CO2. The error consists of taking the bulk mass of CO2 as if it were entirely the product of human activity, when in reality the increase in human CO2 contribution is only 34.29 ppmv out of a total of 381 ppmv (IPCC). This practice is misleading because the anomaly is caused not by the total mass of CO2, but by an excess of CO2 from an arbitrarily fixed “standard” density. There is however no such thing as a “standard” density of atmospheric CO2.
Does this mean that air temperature would increase by 0.02 °C per second until it reached scorching temperatures? No, it does not, as almost all of the absorbed heat is emitted in the very next second. Thus the temperature anomaly caused by CO2 cannot go up if the heat source does not increase the amount of energy transferred to CO2.
0.27 K/s is only 1.24% of the temperature difference between the ground and the air, which was 21.8 K. We can see that carbon dioxide is not able to cause the temperature anomalies that have been observed on Earth.
We would be mistaken if we were to think that the change of temperature was caused by CO2 when, in reality, it was the Sun that heated up the soil. Carbon dioxide only interfered with the energy emitted by the soil and absorbed a small amount of that radiation (0.0786 Joules), but carbon dioxide did not cause any warming. Please never forget two important points: the first is that carbon dioxide is not a source of heat, and the second is that the main source of warming for the Earth is the Sun.
WATER VAPOR:
It is evident that water vapor is a much better absorber-emitter of heat than carbon dioxide. Under the same conditions, water vapor transfers 160 times more heat than carbon dioxide.
I wrote earlier:
tarpon:
Exactly.
Why is the taxpaying public being stonewalled over access to the raw data that is collected by GISS? It’s not like GISS is guarding national defense secrets. This is the weather we’re talking about.
What’s good for the goose is good for the gander. If you can look at numbers and do an analysis, and it’s believable, why should anyone complain? This is the game we’re playing in climate science, isn’t it?
Andrew
Read between the lines of this latest release from GISS and you see that there are cracks appearing in the system! About time
http://data.giss.nasa.gov/gistemp/2008/
Congratulations Anthony on your great win and blog
Meanwhile ….
Williston, N.D. — Minus 40F
GISS Surface Temperature Analysis
Global Temperature Trends: 2008 Annual Summation
Updated Jan. 13, 2009
“…it still seems likely that a new global temperature record will be set within the next 1-2 years…”
I disagree with the analysis also. Nothing remarkable here that I can see.
Note the table and an alpha probability of .05. Each one tagged as significant was just above the critical value, (which must be above 3.59 (column 1) and less than 4.02 (column 7). The other way to state columns 1 and 7 would be to display the probability that the data is not manipulated as something less than .05 for each, but not much less since it’s close (for the sake of expediency I’m not going to calculate the P values…)
Just rounding to .05 since it’s close, the probability of seeing a false positive using the binomial distribution (events=10,p=.05) is 31.5% for finding 1 event above .05, and 7.5% for seeing 2 events. The probability of finding all zeros in the last row is 69.5%, so anyone expecting to see all blank spaces in the “significant” row is going to be disappointed about 30% of the time.
So doing this analysis on purely random data and finding this result (2 events with P of around .05) should happen at least 7.5% of the time. Not significant.
Since any adjustments Hansen makes are run through a virtually unknowable meat grinder of calculations before the corrected historical temperature arrives (naturally much colder than it really was then), I wouldn’t expect to be able to discern any fingerprint, even if something untoward was going on with the original data. That something remarkable is happening with the result, on the other hand, is obvious even to the most casual observer.
But may I suggest: I think you would have a higher probability of finding something refutable by simply analyzing the “adjusted” historical temperatures for all stations remaining in the network, compared to the original readings. As Anthony has shown many times, some stations show a colder adjusted temperature many years ago, when we know quite well that people knew how to read thermometers back then just as well as we can today. I have a gut feeling it’s much more than siting, UHI, etc.
The null hypothesis is that the adjusted historical temperature of the nation (or region) is the same in a given year as the original record indicates, with about the same number of stations adjusted high as adjusted low. The alternative hypothesis is that the adjusted average is lower than the originally recorded average. Repeat test on many years.
That test would be easy to analyze, and if the hypothesis is rejected at P=.01 or so, it would be a strong indictment against Hansen’s methods, and could trigger a more in-depth audit.
Anthony, if you have any insights on how to collect such data, I can do the analysis and write up the results…
Meanwhile. Toronto tonight negative 20 C. Normally negative 2 or 5 C. We’re told by the morning news stations that this has not happened since 2005. Not so long ago but Al Gore told me that it would get hotter and hotter each year (somewhat like Siberia has been). Longtime reader, great site that arms me for the odd conversation with “Nichola” types.
Molon Labe (16:48:36) :
You need to bury this post, this argument, this “analysis” and pretend like it never happened. This silly effort could unravel years of effort in confronting the warmers.
AGREED.
There is no good that can come of this. At best it sounds picayunish; at worst it sounds like a group of halfwits gunning for Dr. Hansen with claims that he looked askance at his female intern. Or thereabouts.
Speaking of rounding, Hansen has made a significant “rounding down” of global warming expectations over the past few days.
The warming trend has been reduced to 0.15C per decade (from the over +0.2C per decade GISS and the IPCC previously used.)
At this new rate of 0.15C per decade, there is no way to reach +3.0C by 2100 and only a little over +2.0C can be reached at this rate.
Here is the quote from Hansen of note:
“Greenhouse gases: Annual growth rate of climate forcing by long-lived greenhouse gases (GHGs) slowed from a peak close to 0.05 W/m2 per year around 1980-85 to about 0.035 W/m2 in recent years due to slowdown of CH4 and CFC growth rates [ref. 6]. Resumed methane growth, if it continued in 2008 as in 2007, adds about 0.005 W/m2. From climate models and empirical analyses, this GHG forcing trend translates into a mean warming rate of ~0.15°C per decade. ”
These numbers were repeated in Hansen’s personal blog and on the official GISSTemp page today.
http://data.giss.nasa.gov/gistemp/2008/
http://www.columbia.edu/~jeh1/mailings/2009/20090113_Temperature.pdf
There were 32 more “1s” than expected, and 32 fewer “7s”. Maybe somebody misread a sloppily written “7” as a “1”?
This could easily happen in the US, because we lack the European convention of crossing the 7 to avoid this sort of mis-read.
http://data.giss.nasa.gov/gistemp/2008/
What I find interesting is the above link furnished by Greg Smith makes no mention of the PDO. Strange… it’s influence on the climate for the northern hemisphere is just as important as the ENSO/La Nina/El Nino ocean cycles.
G Alston and Molon Labe,
You may well be right. But why are skeptics’ concerns never addressed? Can you suggest a method of obtaining the raw data that Hansen hides from everyone outside of his clique? [For “clique” see the Wegman Report to Congress.]
It is their refusal to archive the raw data that makes plenty of people justifiably suspicious of GISS, which has a record of diddling with the official temperature chronology: click [look closely]
And note how GISS is out of step with UAH, HadleyCRUT3 and RSS data: click
For an example of a local area “homogenized” by GISS: click
Michael Mann also refuses to disclose his raw data and methodology, resulting in: click
…Which is refuted by: click
These major inconsistencies could be quickly resolved by publicly archiving the raw [taxpayer-funded] data. But Hansen, Mann, GISS and the UN/IPCC refuse to do so. The question is: what are they hiding??
“But can you suggest a method of obtaining the raw data that Hansen hides from anyone outside of his clique? ”
He isn’t hiding the raw data.
Hansen’s raw input data are provided by NOAA. The raw data is available for download. His algorithm for adjustment is archived and also available for download.
crosspatch:
My response was apparently parroting the comments of others which I accepted at face value, and for that I apologize. Can you provide a link to the raw data, and the methodology, that GISS uses? Thanks.
OT
Heard on Alaska public radio this morning:
Sea Ice in the Bering Sea causes problems for crab fishermen
The APRN.org web site wasn’t responding. 🙁
sprats
Good data! Sounds mighty cold. But in the interest of not cherry-picking the temperature data, it is mighty warm in Southern California this week. We are having a heat wave with highs of 83 or so, and lows of 60 more or less. Had an all-time high earlier this week of 88 degrees. All those temps in F, of course!
Roger E. Sowell
Marina del Rey, California
A little help, guys?
I saw an article by somebody whom I have forgotten detailing how as CO2 concentrations rise, their ability to absorp radiation begins declining at a certain point. And the jist of the conclusion was that CO2 would not be able to supply the feedback effect promised by the IPCC.
I saw this in the last month, but can’t find it in my 4-5 dozen GW articles I have collected.
anybody know who and what this was? I realize this is way OT. But any help here?
You guys sometimes make me giggle. You are quibbling (and I do sometimes as well because when I haven’t had a glass of red wine and a bit of chocolate, accuracy is important) over whether or not winters were actually .02 degrees colder than the official data claims.
Well, it has been a glass and a half later and I am more concerned about whether or not to buy the shoes with a racy strap that angles over my ankle and rides high in the back, or the mary janes that have two cute little straps that cross in front.
I took enough classes in Statistics and Probability in college and graduate school to know that I would never be competent at Statistics and that if I tried, I’d get my tail feathers shot off.
The bottom line is that if Dr. Hansen is going to publish statistical data used to influence policy making, his processes must be transparent. Since that is apparently lacking, due diligence now appears to be required.
That due diligence should not only include an analysis of his statistical methods but also a thermodynamic review of the data collection process.
If he truly believes that there is an APGW signal in the data, then as a scientist, he should initiate such a review. I realize that he may be out on a limb having been running a set of “off the cuff” arguments for many years, but if the signal is there, then a rigorous analysis and – if necessary – correction to his methodologies will do more for his cause then denying his methodologies have a problem.
I was always a little skeptical of APGW but by early last year became alarmed and started researching the subject. I was not impressed with the “pro” internet sites that argued that the science was settled or attacked skeptics using emotional “grabbers” as opposed to scientific counter arguments. I was, however, impressed by sites such as this that disseminated large numbers of counter arguments even though I questioned many of those counter arguments as well.
Congratulations Anthony on your win, and by the way, I have a son at Chico State who managed to get an ‘A’ from an APGW believing professor despite stating contrary beliefs in the first week of class!
Pamela:
Get both.
(Beat it out of the squeeze.)
Smokey: What you are looking for is GHCN-D data which is archived at ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/
And here are the stations he uses: http://data.giss.nasa.gov/gistemp/station_data/station_list.txt
The source code is here: http://data.giss.nasa.gov/gistemp/sources/GISTEMP_sources.tar.gz
(cover your eyes and don’t look at the code immediately after eating)
Lots of clues on this thread from Climate Audit from about a year ago:
http://www.climateaudit.org/?p=2746
Good luck!
Pamela,
Get the mary janes, if that is anything like choosing between Ginger and Mary Ann. Mary Ann every time.
Roger
Oh, smokey, look in the “all” subdirectory of the station_data once you have the station list.
Interesting thing is that station data tends to get spotty sometimes and must be calculated for fills. This is even though complete data is available elsewhere. This happened at one Siberian station when they were at record cold this year. Chances are when a value must be calculated from previous averages or surrounding stations, it is probably going to be a much warmer value than a record cold value would be.
One thing I am interested in is the filling of these values that are missing in the NOAA data and see if they impact the GISS grid temps enough to make a difference. I would also be interested to know if stations experience more data “drop outs” in winter than in summer.
Isn’t Hansen’s work publicly funded? How could he avoid FOIA requests for ALL of his methodology?
Has anyone confronted Gavin with this info yet? If so, what is his latest song and dance number?
Gio-
http://giovanniworld.wordpress.com/2009/01/14/the-jews-are-our-dogs/
Well, leave it to a skeptic to go against the consensus. I want with the racy one that rides high in the back. Must have been the wine. Or the chocolate. Or both.
Pamela
I agree with evanjones, get both. I bet you have picked out an ensemble for both pairs.
I should stop trying ti type afyter a nicr glass of gnarlay head zingfandel windee. I wanted them so I went with them.
And now back to science.
If you want to talk about data manipulation in GISSTEMP, how about this?
The updated GISS data including December temperatures, apart from the usual small changes of some past but recent (i.e. less than 3 years old) temperature changes, shows what I want to believe is an unusual data handling of really, really old temperatures (>20 years)
The following monthly temperatures have been COOLED by 0.01ºC:
10/1901, 01/1902, 04/1902, 02/1911, 09/1913, 10/1916, 09/1917, 09/1921, 07/1923, 09/1924, 10/1926, 12/1928, 12/1935, 09/1944, 04/1948, 01/1951, 07/1953, 11/1983 and 11/1985. Data before 1900 has also changed but I didn’t list the changes.
The following monthly temperatures have been WARMED by 0.01ºC:
09/1977 and 12/1985.
Do you see anything especial about the distribution of these changes? You may think that there is an overwhelming interest in cooling the past. I do too.
Additionally, some seasonal averages have changes without any of their monthly averages apparently changing: summer/1901, fall/1910, fall/1912, fall/1990, spring/1991, which suggests that GISS works with many more digits than those published, and that many more monthly temperatures have temperature changes that cannot be seen because of not affecting the first 2 significant digits. The same happens with a year, 1946, which changes its yearly December-to-November average without any of the monthly temperatures apparently changing. And yes, you correctly gessed the direction of all the changes: they cool at the beginning of the 20th century and warm towards its end.
The effect that these only 0.01ºC step changes in only a handful of months has on the global temperature trend WARMS the trend. The increase is really, really little, affecting only the third significant digit, HOWEVER, it is of a similar ammount than the COOLING effect that a cold 12/08 would normally have on the trend. This suggests that whenever we have a cold month, GISS can manage to mantain the warming trend just by adjusting 20 very old and separate monthly data by just 0.01C expecting that few or no people would notice.
After seeing this, I am personally going to keep track of any changes that GISS introduces in its very old data. Whenever they publish for a month, I will not only download the recent months, but all the list, and will keep track of all the versions. I am also going to do a few calculations with what I learn. Does anybody have previous full versions of GISS monthly data stored anywhere?
CU.
If there’s one thing I’ve learned well, it’s that perception is everything, and in AGW my perception is a conflict of interest when one person edits missing data, manipulates the raw data, and professes that he’s just the messenger, my spider senses start to tingle. How is it that one person can control the whole process and not not provide/share that data manipulation formula with anyone else? Who’s data is it anyway, it belongs to the taxpayers that’s who — this isn’t a national secret he’s keeping. I also thought that real scientists enjoyed having other scientists critique their work and look for errors. Take my word, I’m not a crook!
“I am personally going to keep track of any changes that GISS introduces in its very old data. ”
The old data always changes. Here’s why:
If a value is missing in a month, a fill value is calculated. That fill value is calculated from averages over time. If a value for this month is warmer than average, when the new GISSTEMP is calculated, a new warmer “average” value will be calculated and plugged into that data value in the past. But there is a “hinge” point. Data earlier than that date are adjusted downwards and data after that date are adjusted warmer. So you make warming stronger by making the past cooler in addition to “adjusting” the more modern values upward. It has long been a complaint of people looking at the data and never has gotten any kind of answer other than a brush off.
So if you have a “cold” month, it can adjust months going back in time and if you have a warm month the same thing happens. The problem is that the data set Hansen uses seems to have a large number of “missing values” where the data from the same station from other sources do not have the missing values. But the most annoying part is that the past keeps moving around with Hansen’s method. Temperatures today affect temperatures long ago. It’s rather dumb, really. Just pick a temperature that you think the past was and leave it alone.
GISS most probably has its thumbs on the temperature scales. But I think this leads to a difference of only few hundreths of a degree. The big issue remains SITING. The data is measured at the wrong places. That’s the big source of errors.
Thanks a lot crosspath for the explanation.
It is interesting to see that, as long as we are above the average (not very difficult), the warming trend is going to look bigger and bigger every month because we will think that the past was cooler and cooler!
I have been making numbers with an excel sheet. The effect which these temperature changes in the past has over the 1900-2008 warming trend is equivalent to not having changed the past values and having had a December 2008 temperature anomaly of +0.59 instead of +0.45. This gives an idea of the order of magnitude of the FRAUD we are just watching.
Best regards.
Out of the world of conjecture, into the world of evidence.
Here are some consecutive figures from a weather station, described as “raw” data. ALL ARE A DECADE OR MORE AFTER THE CHANGE FROM DEG F TO DEG C.
Tmax on 1st Tmax on 1st
Year Month Day Tmax day of year day of month
1976 1 1 29.6 29.6
1976 2 1 23.1
1976 3 1 24.7
1976 4 1 19.6
1976 5 1 23.8
1976 6 1 16 16
1976 7 1 18.8
1976 8 1 19.2
1976 9 1 17.2
1976 10 1 23.2
1976 11 1 16.1
1976 12 1 22.5
1977 1 1 27.9 27.9
1977 2 1 25 25
1977 3 1 38.6
1977 4 1 21.3
1977 5 1 23.2
1977 6 1 20.8
1977 7 1 22.6
1977 8 1 18 18
1977 9 1 14.8
1977 10 1 17.7
1977 11 1 18.8
1977 12 1 23.1
1978 1 1 25 25 25
1978 2 1 34 34
1978 3 1 25.7
1978 4 1 28.4
1978 5 1 25.6
1978 6 1 20.8
1978 7 1 16.5
1978 8 1 15.1
1978 9 1 18 18
1978 10 1 20.5
1978 11 1 25.6
1978 12 1 23.2
1979 1 1 25.9 25.9
1979 2 1 21.6
1979 3 1 23.6
1979 4 1 24.3
1979 5 1 17.3
1979 6 1 20.2
1979 7 1 14.3
1979 8 1 16 16
1979 9 1 15.2
1979 10 1 20.2
1979 11 1 17.6
1979 12 1 26.7
1980 1 1 27.7 27.7
1980 2 1 23 23
1980 3 1 24.2
1980 4 1 23.8
1980 5 1 23.3
1980 6 1 22.7
1980 7 1 17.4
1980 8 1 16.4
1980 9 1 16 16
1980 10 1 21.7
1980 11 1 26.5
1980 12 1 20.5
1981 1 1 24 24 24
1981 2 1 26.4
1981 3 1 20.2
1981 4 1 21.1
1981 5 1 21.9
1981 6 1 17.3
1981 7 1 13.2
1981 8 1 16.8
1981 9 1 20.1
1981 10 1 16.3
1981 11 1 31.3
1981 12 1 26.1
1982 1 1 21 21 21
1982 2 1 35.8
1982 3 1 23.5
1982 4 1 19.8
1982 5 1 21.6
1982 6 1 18.8
1982 7 1 10.9
1982 8 1 23.2
1982 9 1 17.7
1982 10 1 27.3
1982 11 1 35 35
1982 12 1 22.3
1983 1 1 24 24 24
1983 2 1 22.4
1983 3 1 23.4
1983 4 1 25.9
1983 5 1 25.5
1983 6 1 26.3
1983 7 1 17.2
1983 8 1 18.6
1983 9 1 21.4
1983 10 1 19.3
1983 11 1 30.6
1983 12 1 15.8
1984 1 1 23.7 23.7
1984 2 1 38.6
1984 3 1 31.6
1984 4 1 36.8
1984 5 1 21.5
1984 6 1 21.9
1984 7 1 15.7
1984 8 1 22.1
1984 9 1 18.4
1984 10 1 16.5
1984 11 1 21.2
1984 12 1 20.7
CONCLUSIONS count 108 9 13
4 whole nos 13 whole nos
in 9 YRS in 108 months
EXPECT <1 EXPECT 10.8
ends padded, not cherry picked
I pasted the mean monthly Mauna Loa co2 record into the Wikicheck and got the following result:
RESULT: Extremely Significant management detected.
Interesting!
Pamela said
“You guys sometimes make me giggle. You are quibbling (and I do sometimes as well because when I haven’t had a glass of red wine and a bit of chocolate, accuracy is important) over whether or not winters were actually .02 degrees colder than the official data claims.
Its obviously a boy thing. Mind you what makes me laugh rather than giggle is when everyone parses ‘global temperatures to 1850’ to fractions of a degree as if these figures had the slightest touch with reality. The trouble is that those doing the parsing the most are the IPCC who are influencing the economy of the world. Could you distract them by taking them to help choose your shoes?
TonyB
Since I am somewhat new to the forum, please excuse me if this is redundant!
If the issue is that Jim Hansen is not releasing key information about how or why he does things, and he works for NASA a federally funded organization, wouldn’t a freedom of information request compel NASA to release the relevant information?
Likewise for those issues that drive public policy, I would think that a freedom of information request to release the methodology of the data that formulation of new public policy depends on would either compel release of the information, or provide grounds that that data/methodology not be used as the basis of new policy until such info is released for public review.
Larry
Anyway, what kind of “missing values” in 1900 are being calculated from averages which include data from 2008? Are we talking of specific days in specific stations, or monthly averages of specific stations, or monthly averages of entire regions, or global monthly averages? Obviously all of them will be changed, but where does the problem start?
Best regards.
So there is not a full resolve to the questions regarding the GISS data yet. Anthony is leaning toward Steve McIntyre’s analysis of the data indicating that nothing is really askew. There is, however, a larger dynamic present in this situation. The openness and sharing of methodology, software used, formula’s, etc. It is the ethic which those running the different analyses in this matter continually demonstrate in all of their work. They seek the proper solution, the truth, even if the truth means their initial approach is ultimately revealed to contain error of one nature or another. Their comments and work were and are available for all to see. There to be challenged. That is what science is supposed to be about.
Man’s understanding of the climate would be much farther along if that same openness existed with all scientists involved in climate research. Science realizes no true gains when one small group is permitted to claim the final and proper results and do so without openness of data, methodology, codes, etc. Essentially, some feel their work should go unchallenged and they, to a large extent, are permitted to get away with it. That is not good science and they damage science in doing so. It is a shame that they lack the integrity, ethic, and courage of those participating here in the GISS data issue.
Pamela Gray:
Solution: WWID? What Would Imelda Do? The answer is obvious. She would buy both pairs.
Actually there should be a slight bias to certain ending digits due to the instrumentation. Digital equipment would have a random tenth degree digit, but analog equipment typically is read to the nearest 1/4 degree.
Smokey wrote:
Solution: WWID? What Would Imelda Do? The answer is obvious. She would buy both pairs.
LOL Actually, a lot of Imelda’s shoes were gifts from the manufacturers for bragging rights–free promo. Kind of like all the freebies that are given away to actors/actresses attending the Oscars?
John Philip (16:07:46) :
Well, I’m sure Joe D’Aleo would just point out that this is another example of the AMS board having been taken over by people better at playing politics than most meteorologists.
I didm’t record their comments at a Climate Task Force meeting (see http://wermenh.com/climate/index.html ), but Joe and Fred Ward (retired TV Met from Boston) had rather harsh words about the AMS position statement on global warming. It was passed over the objections of several board members and the rank and file membership had no say.
I commented on that earlier, see http://wattsupwiththat.com/2008/12/03/cleveland-area-tv-meteorologists-disagree-with-prevailing-attitude-about-climate-change/#comment-61147
Joe would likely call it another example of the AGW community patting each other on the back.
Honestly me thinks this whole debate will probably be redundant if temps continue down for the next 10-20 yrs`as predicated by sun (of course they could go up) LOL
Dr. Roy Spencer has a new blog entry on how the IPCC models treat cloud feedbacks…
Does Nature’s Thermostat Exist? A Global Warming Debate Challenge
http://www.drroyspencer.com/2009/01/does-nature%E2%80%99s-thermostat-exist-a-global-warming-debate-challenge/
Boy did Benford’s law drag some cobwebs out of the back filing cabinets. That was so many years ago that Carter was still in office. However, to those that want to “bury” this issue I say no. All questions should be open and up for discussion, if they are reasonable. Benford’s law is a well documented test for manipulation of data either from the system, fabrication, or manual adjustment. To dismiss it out of hand is not appropriate given the errors that have been made in many data sets regarding this issue. The manipulation may be valid such as unit conversion, manual data entry errors, or other unknown issues. But, it is better to know of it’s existence than to bury it and ignore what is happening. This is not a PR game, but a scientific inquiry and both data and analysis must be shared. Otherwise we are not following scientific methods which with the proper feedback lead to the correct answer.
In my investigations I found an interesting piece of software from Kirix. It allows for the pulling of data from a web page directly for analysis. Looks cool, but is a bit pricey. Follow the link for a look:
http://www.kirix.com/
Nylo:
“I have been making numbers with an excel sheet. ”
Might be interesting, where you find missing numbers in a recent year, see if you can identify the station from another source. Weather Underground, for example, keeps data back a few years for a lot of stations. See if you can find the missing value there, plug it in and see if it makes a difference in the outcome.
“This gives an idea of the order of magnitude of the FRAUD we are just watching.”
I wouldn’t attribute to fraud what can be explained by sloth combined with a need to validate one’s own hypothesis. Remember, the entire purpose of the GISSTEMP is to validate Hansen’s climate model.
John Philip (16:07:46) :
More on this via Joe D’Aleo’s http://icecap.us/ (see his comments there) is at http://dotearth.blogs.nytimes.com/2009/01/14/weather-mavens-honor-climate-maven/ but be sure to read the comments, e.g.
http://community.nytimes.com/blogs/comments/dotearth/2009/01/14/weather-mavens-honor-climate-maven.html?permid=17#comment17
“Remember, the entire purpose of the GISSTEMP is to validate Hansen’s climate model.”
To clarify … GISSTEMP wasn’t designed “to see if the Earth was warming”, it was designed to “show that the Earth is warming”.
If the AGW promoters were open about their sourcing and methods this would not be a credible issue. They are not, however. They are secretive and defensive, and so the issue will remain.
The deviations in the data records are interesting and should certainly be explored but I feel the notion that someone is deliberately manipulating the data is way off base. Moreover, getting excited about it and suggesting conspiracies undermines the significance of more trenchant criticisms such as siting, infilling, UHI and simply poorly specified GCMs.
Richard P — However, to those that want to “bury” this issue I say no.
It’s one thing to look at numbers to see if there’s some sort of systemic bias (i.e. equipment issues) but something else entirely to carry on as if there is wrongdoing. The post should have been oriented to looking at a queer set of interesting numbers to the effect of asking how they got there naturally rather than premising nefarious intent.
The former makes sense. The latter doesn’t, especially in that the overall effect seems negligable.
REPLY: If there were not so many odd adjustments or mistakes identified in the GISTEMP record, or if Jim Hansen had decided not to come to the defense of vandals in England, then most certainly this would have been looked at with less suspicion, maybe even not at all. But the list of precedence of odd things seen thus far cause questions like this to be raised in a context that questions the credibility of the dataset and the keeper of it. I gave the benefit of the doubt on this specific issue immediately with my first posting, and like you I think that this is likely an artifact of little significance, but I still have concerns for the overall dataset integrity. – Anthony
I posted this at CA, but Steve may snip. So:
“Luis,
You are wrong to belittle this exercise and the surface stations project. You seem to have failed to grasp what the real issue is — credibility. And there are two aspects to the credibility issue. The first is honesty and the second is competence. The possibility that data has been manipulated goes to honesty. It matters not one bit whether the possible manipulation has a significant impact on trendlines, etc. Even if the impact is tiny, if data’s been manipulated, the parties involved are dishonest and all their work should be regarded as unreliable. After all, climate scientists don’t bother to check or replicate each other’s work. If someone’s untrustworthy, their work is untrustworthy. Period. [Note, this standard is especially appropriate for one who thinks that those who disagree with him should go to jail.)
Competence, the second aspect of the credibility issue, is directly addressed by the surface stations study you disparage. Once it was shown that hundreds of stations violate basic scientific standards for placement, the burden was no longer on Watts to demonstrate some quantifiable way to correct the temperature record. The burden properly rests upon those who consider the record authoritative to demonstrate why such incredibly shoddy work has any scientific credibility at all. And further, why the people in charge of such shoddy practices should be given any credence with respect to the rest of their scientific work. Most people expect that those who endeavor to build sophisticated scientific structures using the temperature record ought to first bother to find out if the thermometers are accurate. [maybe it should be a law that climate scientists demonstrate minimal proficiency with a thermometer before receiving a government grant.]
This climate science is the driver for an extraordinary array of political policies. As Steve noted, he first got interested in the hockey stick because the “findings” were being used to drive public policy in Canada. Of course, Hansen has been at the forefront of using this science for political purposes. We may decry the politicization of the science, but we cannot deny that the two are now inextricably intertwined. So your belittling of a statistical exercise which may shed light on the honesty of a central figure to the debate and a database crucial to the scientific arguments reflects either a misunderstanding of the issues or an attempt at obfuscation.”
Bernie:
The problem is that climate science is such a mess, that we don’t know what set of numbers is dubious… unless someone asks ALL the questions- questions dumb, questions old and questions “unscientific” even.
To rely on one group’s take on the climate (even if that group is a group of scientists) is a fallacy. The possibility does exist that someone is “cooking the books” to enhance the stature of themselves or their particular group they like.
So ALL the questions need to be perpetually asked, including who and who may not be biased (on purpose) in one direction or another. Do you see what I’m getting at? Perhaps one day if some actual evidence is offered, some questions can actually be answered.
Andrew ♫
Bernie 09:23. I agree with you!
Andrew:
The paranoid will always find someone who is “after” them. I understand the history and the need to ask questions … but asking questions is different from essentially accusing people of wrongdoing when there is, at best, inconclusive evidence. In the scheme of things and without an estimate as to the size and direction of any possible effects – this issue can be seen as a curiousity or another example of skeptics grasping at straws. David and Lubos should look more closely at the data but without the attributions as to intent – at least for now.
I gave the benefit of the doubt on this specific issue immediately with my first posting, and like you I think that this is likely an artifact of little significance, but I still have concerns for the overall dataset integrity. –
Are you saying that was your intention with the headline? (“Distribution analysis suggests GISS final temperature data is hand edited – or not” – incidentally, was the “or not” there originally?). If so, I think your words failed you. Your headline insinuates that GISS may be responsible for “editing”. Whatever the strengths or limitations of the distribution analysis, it offers no evidence of that whatsoever.
Saying “X may have done something improper” is not a good way of expressing the benefit of the doubt over such a matter. I am very surprised that you are seemingly so unaware of the implication in your choice of words.
G Alston: “It’s one thing to look at numbers to see if there’s some sort of systemic bias (i.e. equipment issues) but something else entirely to carry on as if there is wrongdoing.”
It was not my intent to imply any wrongdoing by anyone. There are very few things that I ascribe to malice or fraud especially on complex issues. I do believe in making sure that the data is accurate, and use tests even on data I generate as a double check of accuracy and quality. My point was to make sure that we understood that it was not a problem rather than dismissing it out of hand.
My first rule of engineering for interns fresh from school was do not make any assumptions without acknowledging what they are. Many times problems were resolved because what we thought was happening turned out not to be true. By questioning those assumptions you sometimes find the problem much faster rather that going along fat dumb and happy. Now you don’t question basic physics, ascribe to conspiracy theories, or assume that the aliens did it, however, reasonable questions should be allowed especially if errors have been seen in the past.
My prediction for what it is worth would be that this is either a non issue or minor property of the system and will have no effect on the outcome. Of course I am making some assumptions that may be wrong, so now my interns may get some payback ;-).
“” Pamela Gray (21:02:34) :
I should stop trying ti type afyter a nicr glass of gnarlay head zingfandel windee. I wanted them so I went with them.
And now back to science. “”
Lordy lordy ! lady do you also arm rassle down at the local selloon ?
That there zingfandel is lumberjack rocket fuel. Can’t you find a nice lady like wine like Tawny Port or something. I betcha dring Guinness Stout too !
I’d buy the raw leather boots; with the studs on them, ‘case you gots to kick some hooligins outa that place !
We’ll mind our manners a bit more round here, case we draw on yer ire !
George
Bernie,
In climate science, the “evidence” is dependent on what you personally believe. Accusations of wrongdoing should be pervasive and plentiful in a place where the general population claims to know something it dosen’t.
Andrew
If you want to know if the title was edited, use the time machine functions of google or simply google search. Someone copied the original post: and there is no – or not in there
http://www.bolsanobolso.com/showthread.php?t=26859
In fact, someone posted the original link elsewhere on a :
http://wattsupwiththat.com/2009/01/14/distribution-analysis-suggests-giss-final-data-is-hand-edited/ and this is how it still shows in the browser address bar.
Editing? No, Not here and certainly not without audit trail, something Steve McI would get aghasted over.
Sekerob,
Yes the “-or not” was indeed added a bit later. Note that there were also two updates added at the top right under the headline about the same time. I believe when Update 1 was posted is when I added the “- or not”.
Given the fact that the story evolved very quickly after the initial post, within about 1-2 hours if I recall, and that there were commenters suggesting that the lack of certainty should be more prominently displayed than the doubts I expressed at the end of the article, I opted to add that to the title in the spirit of fairness. At this point, since there is evidence both ways we don’t really know and I think the title reflects that.
If I left it “as is” I would be getting complaints also. I opted for a fairer title in this case.
Unfortunately WordPress.com (my free hosting service) removes any formatting attempts to the title, so the use of strikethoughs or colors to delineate such a change is removed. In retrospect I probably should have made a note in one of the updates, and I’ll do that should such a situation arise again.
Thanks for bringing this to my attention.
I’m aghasted too… NOT.
At this point, since there is evidence both ways we don’t really know and I think the title reflects that.
What evidence is there of GISS having hand edited their final data? None whatsoever. Saying “though maybe not” after the event does not make your insinuation “fair”. If I were to insinuate that your post is “evidence” of malicious denigration, then what would you think of that? I suspect you would find such an insinuation offensive. The fact of something being a possibility does not mean that it gives evidence of itself!
Actually, distribution analysis is evidence. It is not proof, and whether it is conclusive remains to be seen.
And as Steve McIntyre says, “…after refusing for a long time and under protest, Hansen did archive his source code, which is a mess – which is probably why he didn’t want to archive it.”
So we have a source code that’s a mess, indicating at the very least a lack of competence, and we have evidence that the data was manipulated.
Maybe there’s an innocent explanation for the distribution analysis findings. But there is no excuse for a shoddy source code. How can that result in good science?
And how can Hansen justify altering the past record — based on new temperature measurements?
Actually, distribution analysis is evidence. It is not proof, and whether it is conclusive remains to be seen.
It’s evidence, but we don’t know what it’s evidence of. Insinuating that it’s evidence of GISS having hand-edited their final results is simply unfounded. Very obviously, the distribution analysis could be evidence of all sorts of things that have nothing whatsoever to do with GISS. It’s not even evidence yet of the data being “manipulated” as you state, let alone manipulated by GISS. You are simply expressing confirmation bias in your post, IMV, by referencing your other beefs with GISS. That’s humanly understandable, maybe, but you should distinguish it from scientific enquiry.
Steven Talbot, a question:
Have you made similar complaints over at Lubos Motl’s site about his choice of the word “cheating” in the title?
Or to David Stockwell for use of the word “fraud” in his title while referencing GISS in the story?
I’m just wondering if your issue is specific to this blog or to the issue of choice of words in titles about the story in general.
Anthony,
Have you made similar complaints over at Lubos Motl’s site about his choice of the word “cheating” in the title?
Or to David Stockwell for use of the word “fraud” in his title while referencing GISS in the story?
No, I’d not been aware of those sites. I will follow up tomorrow and post my views. I think it is entirely improper to suggest fraud or cheating on the basis of what has been presented. If it were to turn out that anyone were engaged in such practices then they should face the severe consequences of their actions. However, I take it as a basic principle of human decency that one should presume innocence until guilt is proven.
I’m just wondering if your issue is specific to this blog or to the issue of choice of words in titles about the story in general.
No, my issue is to do with the suggestion of possible cause in respect of evidence which may have no relationship to the posited cause. If Hansen (or whoever) were to do this I expect you would be all over him with your criticism, and you would be right in that.
I think this ‘story’ may have contributed to doubts about GISS in ways that turn out to be entirely unjustified. You have responsibility in what you say, as do I, and as do Motl and Stockwell. I will post my opinions on their sites. I do not think it is proper to suggest malfeasance without proof of the matter.
Steven,
I’m surprised that you were “not aware” of those sites, since both were heavily referenced through my article with links provided, and were in fact the two major sources. In a court of law, the DA “suggests” malfeasance or commission of a crime to the judge, who will then grant a trial if the initial evidence reaches a level of preponderance defined by law. In the trial phase is where the proof is presented.
I’ve listed my doubts in the article, and the doubts of others. – Anthony
Methinks someone may be scared.
Andrew
Steven Talbot (17:00:31) you are missing the underlying point. If Dr. Hanson’s methodologies were transparent, there would not be the accusations about which you protest.
Anthony,
Well, regardless of whether or not you are surprised, I didn’t realise they were sources where I could post comments. I have already posted a comment at Stockwell’s site but am struggling with Google passwords at Motl’s. In future I guess I’d better make sure that if I make any critical comment here I have already made it at the sites of those whom you reference.
You are not presenting a case in a court of law, ‘in camera’ before a judge. You are publishing your views on your own public web site. If you do not consider that you have responsibility to avoid the insinuation, without proof, of base motives to others in such a forum, then we must accept that we have different standards.
Andrew, what do you think about this before and after graph showing two GISS temperature dataset releases that have been made public, differing only by date:
http://zapruder.nl/images/uploads/screenhunter3qk7.gif
Do you think it is OK the change temperatures in the past? Do you think that type of behavior would be tolerated in say, a medical study? A vehicle safety test? Or how about financial records?
Lets hear what you think about it.
I think it’s a joke that past temps are changed. I think changing information as you go should not be tolerated period, by anyone. You’d think that science would police and eliminate such a practice, were it discovered.
I knew AGW was a fraud all along, but the day Governator Ahnold came out and said, “The science is in… man has created the global warming” I had an extra piece of pie for dessert that night and slept like a baby.
Andrew
Sometimes data analysis can produce a false positive. Been there done that. But the only way to know is to do a different analysis to back up or contradict the first analysis. It seems that the kind of analysis done here would call for a closer inspection of how data is handled.
To tell the truth, even seasoned and educated scientists can make biased choices, all the while swearing truthfully they believed they were being objective. “Facilitated Communication” is one such phenomena that was finally given sufficient scrutiny to discover it was not what it seemed to be and was actually fraught with human tinkering, though done with the best of intentions. What was eventually uncovered was that the human facilitators had no idea they were the ones causing the phenomena to happen, but indeed they were doing exactly that.
Those that have a fanatical belief in the dire consequences of global warming and are intricately involved in the data surrounding global temperatures and CO2 measures, to the point that they are lauded for it, are at high risk of doing the same thing; facilitating the phenomena to occur, even though intentions may be honorable and they faithfully swear that they have been “objective” in their data handling.
This analysis should be taken seriously by global climate change scientists and should be followed up with in-house analysis to see if indeed, hands of been at work.
geez, just three sips of a nice warm sherry. “…hands HAVE been at work.”
My personal take on this whole thing:
The data can not be expected to pass a random test because it isn’t random. The final numbers after calculating fills, averaging, and then “adjusting” will show significant artifacts from those processes. In fact, I would expect those artifacts to change from month to month because, for example, if you add a new month, when calculating a new “average” for a fill value, you divide by an additional number of months.
I would expect the raw input data to be more random over a long period of time (but not over short periods of say, less than 10 years).
This particular issue is, I believe, a non-issue and tends to distract the focus from important measures such as the positive feedback built into the calculations where recent warming increases past warming (and cooling back past the “hinge” point). For example, lets say we get a temperature for a month from a station and it includes all daily readings; no missing values. Lets say it is warmer than “average”. That number impacts all the calculated fill values for previous months’ missing values. It will also impact the future too because future temperatures that might be missing will be calculated with this warmer temperature. Missing values become extremely important when they happen during record warm and cold periods.
The impact of all of this is to exaggerate trends. One might say that this method of using “averages” doesn’t matter because 50% of the time the temperature will be above average and 50% below. That isn’t really true. First of all, we are using an AVERAGE not a median. Secondly, there are natural cyclic modes that cause climate to change in the same direction for long periods of time. The 50% rule would probably only apply over periods of longer than 50 years. For about 30 years you can expect temperatures above “average” and for about 30 years temperatures below “average” with very little time at “average” temperatures. GISS happens to select the coldest period in the past 60 years as its “baseline” further inflating the appearnce of “warming” as their “average” is actually a very cool period.
This is why I believe that our using 30 year periods of time as the “normal” baseline is incorrect. I believe we should use a period that captures the most recent entire PDO cycle (both the warm and cool phase) for Northern Hemisphere temperature baselines. And we should expect 30 years of above “average” and 30 years of “below average” temperatures as normal climate variability. This should end up being something around 60 years but it will vary in duration as cycles vary in duration.
In the meantime, what I believe would be a more responsible way for GISS or even NOAA to provide their product would be first to attempt to see if a missing value from the standard source of data is available from a different source of the same data. If so, that value should be filled from the actual measurement and not from calculated “fills”. Secondly, stations that have dropped out of the record should be reinstated *or* the ratio of urban/rural stations should be adjusted. Third, as a sanity check, the deviation of GISSTEMP from RSS and/or UAH should be monitored and if that deviation changes greatly, particularly over time, it should act as a flag for some QA to see what is going on and why the deviation has changed. Of particular concern to me are months where the GISS monthly change differs from the satellite measurements in both magnitude and direction.
Has anyone ever done an RSS / GISS anomaly plot over time to see what the differences are, if they have changed at certain points in time since 1979 or if they are diverging?
I think it should be ok for me to change the numbers on my speedometer, odometer, bathroom scale, driver’s license, birth certificate and paycheck… retroactively even.
“what do you think about this before and after graph showing two GISS temperature dataset releases that have been made public, differing only by date:”
Looks to me like there is a significant “hinge” or “pivot” point at about 1965. Temperatures after that date are adjusted warmer in the newer picture, temperatures before that date are adjusted colder.
Steven Talbot, same question that I posed to Andrew.
You can do that on bathroom scales??????? WHO KNEW????????
Steven Talbot,
I am not as certain about previous references to Mr. David Stockwell, but I know that Luboš Motl’s interesting website has been referenced here many, many times. And not just in the recent past. I really wonder how you could visit here so often and still be unaware of his site.
Like WUWT, Dr. Motl’s The Reference Frame should be required reading for AGW proponents, so they can at least understand the opposing arguments.
I don’t believe they had no idea at all. Perhaps a very small number didn’t, but I think the vast majority knew exactly what they were doing.
Crosspatch, I think everyone who lived in 1965 knows that it really was a “hinge point” as you call it. Things are actually much cooler now than they were before 1965, and that’s hot, to quote Paris Hilton. So I believe that Hansen may have been a resident of Haight-Ashbury. Of course this conjecture could be wrong.
Thanks for the blink temperature comparator, 1996, 2007… Steve resurrected 1934, but Big Jim buried it alive again…
Anthony:
Mr. Talbot says above — “You are not presenting a case in a court of law, ‘in camera’ before a judge. You are publishing your views on your own public web site.”
This is what I’d been thinking; mostly I was concerned about the PR aspect of all of this. It smacks of nitpicking so as to go after Dr. Hansen. It’s just not YOU. This isn’t the sort of mistake you make.
Now while I wholeheartedly agree that Hansen OUGHT to be gone after, I’d much rather see him in a wrestling smackdown (complete with folding chair blows) for things like changing past temps or encouraging criminals.
Going after him for what appears as an instrumental or algorithmic artifact is like ignoring the egregious stuff and settling for gossip about him and his intern.
And make no mistake, in the mile high view this can (and will) be seen as going after Dr. Hansen. A cheap shot. Even if that wasn’t the intent. You know that Tamino et al are going to eat this up, and this can damage hard won credibility.
That’s my view. I’ll shut up now. Thanks for listening. 🙂
That would explain the combover.
Nylo (02:50:29) :
There was a post several months ago detailing how USHCN adjustments are made, and IIRCC, Steve McIntyre made some forays into seeing how many datapoints are missing.
I didn’t find the link I wanted, but http://wattsupwiththat.com/2008/09/23/adjusting-pristine-data/ is informative.
Too late for more digging.
Temp near Concord, NH: -11.2°F. Cold, and 3rd coldest in my five years of
detailed records:
mysql> select dt, hi_temp, lo_temp from daily where lo_temp <= -10;
+————+———+———+
| dt | hi_temp | lo_temp |
+————+———+———+
| 2004-01-10 | 7.2 | -11.0 |
| 2004-01-14 | 0.9 | -13.5 |
| 2004-01-16 | 10.4 | -11.0 |
| 2005-01-22 | 10.2 | -13.0 |
+————+———+———+
Like MattN noted, "If we get any more global warming, we’ll all freeze to death!"
Yay – Concord NH set a new low temp record of -22F this AM. I figure if you have to endure extreme weather, it might as well be record setting.
Here at home, about 10 miles from the Concord weather station, I reached only -17.5F. I’m in a valley where cold air pools on clear calm nights, but trees, and buildings block some sky exposure.
The Concord station is at small airport (no regular commercial traffic) on a flat (duh!) plain near the Merrimack river valley. Under really good radiational cooling they usually get colder than I do.
My last 48 hours of data are at http://home.comcast.net/~ewerme/wx/current.htm
Steven Talbot, same question that I posed to Andrew.
That is:
what do you think about this before and after graph showing two GISS temperature dataset releases that have been made public, differing only by date:
http://zapruder.nl/images/uploads/screenhunter3qk7.gif
Do you think it is OK the change temperatures in the past? Do you think that type of behavior would be tolerated in say, a medical study? A vehicle safety test? Or how about financial records?
Lets hear what you think about it.
Firstly I think it’s not the subject of this thread and, as I have suggested to Smokey above, referencing other beefs with GISS is indicative of confirmation bias at work, IMV.
However, since you ask what I think –
1. I am unclear as to whether you are concerned about the GISS process only or about the USHCN data with which they work. I’ll presume you are considering the whole ‘package’, though I think it relevant to recognise different responsibilities.
2. As you are well aware, the most significant adjustments were described in Hansen et al. 2001: http://pubs.giss.nasa.gov/abstracts/2001/Hansen_etal.html. In biref, they encompassed USHCN adjustments for time of observation bias, station history, instrumentation changes and urban warming. The main difference between USHCN and GISS adjustments was GISS’s greater negative correction for urban bias (about -0.15°C over 100 years, compared with a USHCN urban adjustment of -0.06°C).
3. You ask if I think it’s OK to change the data from the past. I certainly do. For example, if the influence of UHI can be analysed, should it not be adjusted for? If a station has a warming bias because of its location, should that not be adjusted? Or if time of observation has systematically changed from afternoon to morning, then how can we assess a meaningful view of temperature change over time without adjusting for that?
Given that the whole thrust of your surface stations project is to examine the extent to which temperature readings are uncorrupted by spurious influences I find your questions here rather puzzling. The USHCN (GISS) adjustments have been done in order to remove spurious biases. I think that is entirely proper and, yes, I would expect exactly the same from a medical study, safety test or financial record (to give you an example of the last, I would expect the performance of financial products to be assessable in the context of inflation adjustment).
If it were to be recognised that UHI effects have been underestimated for the last thirty years, say, would you object to those temperature records being adjusted downwards to give us an unbiased assessment of the situation?
It was, of course, an incorrectly computed time of observation bias (satellite drift) that had corrupted the UAH records up to 2005. Did you object to Christy & Spencer recognising their error when it was pointed out to them and thereafter changing their own past data?
Something just struck me this morning. It is -30F at my house in Norway, IA, and Debuque, IA tied a record at -30F for today. This record for today was from 1888. Have these early records been adjusted in similar ways to the GISS data? Or, are only the records for “climate change” analysis purposes changed, and not he actual readings. For if there was truly a problem with the data from these early periods then should not the actual records be adjusted?
Just a question.
Incidentally, this entry on the GISS ‘Updates to Analysis’ page may be relevant to the distribution issue (apols if mentioned before) –
Aug. 11, 2008: Nick Barnes and staff at Ravenbrook Limited have generously offered to reprogram the GISTEMP analysis using Python only, to make it clearer to a general audience. In the process, they have discovered in the routine that converts USHCN data from hundredths of °F to tenths of °C an unintended dropping of the hundredths of °F before the conversion and rounding to the nearest tenth of °C. This did not significantly change any results since the final rounding dominated the unintended truncation. The corrected code has been used for the current update and is now part of the publicly available source.
http://data.giss.nasa.gov/gistemp/updates/
Steven Talbot,
I understand but disagree with many of your points, particularly that the GISS adjustments are correctly applied.
There have been several examples where both GISS and NCDC (USHCN) data shows that completely missed the mark in corrections of stations moves, UHI, and micosite biases. For example:
http://wattsupwiththat.com/2008/02/17/how-not-to-measure-temperature-part-52-another-ufa-sighted-in-arizona/
The GISS UHI methodology of counting nightlights around stations is now terribly out of date, since (as far as I am aware) the last DMSP sat image they used for the purpose was done in 1995. Now 13 years out of date.
in some cases, stations that have no reason whatsoever to be adjusted by the nightlights method, are in fact being adjusted by GISS:
http://wattsupwiththat.com/2008/07/18/cedarville-sausage/
Also both NCDC and GISS datasets have no adjustment for long period biases, such as urban creep, deforestation, etc., nor do they have a method for detecting it.
Finally, it has been shown by NCDC’s own graph, that their sum of adjustments for TOBS, FILNET, SHAP, and the miniscule UHI, impart only a sum positive bias to the record.
I find your faith in the methodology being representative as puzzling also.
Anthony,
I am not asserting the absolute accuracy of the methodology, though I would probably go along with your word ‘representative’. Clearly there is error in the process, though whether or not that exceeds the GISS error bands I do not know.
I am well aware there are examples of poorly sited sensors, and I am sure that you report on microclimate biases that introduce spurious cooling as well as those that introduce spurious warming, such as instances of sensors influenced by tree canopy. I understand from R.P. Sr’s papers that the influence of asphalt is seasonal – cooling in winter, warming in summer. The quantification of that is tricky enough, let alone a total quantification of all biases. I do think that the GISS methodology seeks to dilute the influence of anomalous readings, but whether that is effective or not is a reasonable line of enquiry.
I look forward to a time when there are more satellites in space which have been designed specifically to assess our climate. They will doubtless have their own teething problems and systemic errors, but the closer we get to fuller and more accurate information the better.
So, I wholly agree that there is evidence of inaccuracy in the record. I don’t find that very surprising, really, but I agree that work should continue in the direction of improvement. What I do not share with many commentators here is the ready presumption that inaccuracies and adjustments are evidence of human bias in favour of showing a warming climate. I am not aware of any specific evidence of that, though it seems to be established in the minds of many as if it were a matter of fact. That, it seems to me, is more a matter of ‘faith’ than anything I have said. My view is that we should not presume cause unless it is evidenced, and that, I suggest, is a sceptical stance.
When you change the record of anything, you are changing the history. Temperature is history. History is supposed to be what happened and what people thought at the time. When you give yourself the right to change the record you are giving yourself the right to change history. Anyone who wants to do this wants to hide the past for some reason. It doesn’t matter if they have a good reason or bad to want to change it. It doesn’t matter. Changing it at all is a bad idea.
I work for financial institutions for my day job, and we document everything within reason that is job related. The documentation has to reflect the history, so we can continue to learn from it, for one reason. If you change it, there is nothing for you to help you remember what actually happened. If a mistake is made, or something is wrong, the good and bad are all recorded and we go correctly from there. If I ever went back and tried to change the history of what happened to my company, I would be fired, not to mention my personal relationship with the people that run the places would be destroyed. They would consider it a betrayal.
Andrew ♫
Andrew,
I agree that original records should not be destroyed or overwritten, but they have not been. GISS (and others) present an analysis of those records that remain exisiting.
If your financial institution discovered an error from the past, would they not correct it in their analysis (my experience suggests that if I have been overpaid I don’t stay overpaid once the error is discovered!)?
Biased temperature readings are not the history of temperature but the history of recording error. It seems to me that, by your argument, UAH should still be reporting their faulty pre-2005 data, even though it’s known to be wrong.
In all fields there will be records from the past that are either unintentionally or intentionally biased. If we want to get closer to the truth then our analysis must take account of any known biases, in my view at least. 🙂
Steven Talbot:
Niccolo Machiavelli wrote to a naive prince: “Men are evil, unless compelled to be good.”
Who can compel James Hansen to be honest? Hansen has accepted at least three quarters of a million dollars [that we know about] from entities with a strong global warming/AGW agenda — while the Best Science site is run on a pretty much voluntary basis.
Who are you gonna believe?
There isn’t much difference between believing what someone on the green payroll says, and believing what tobacco company front groups say. They are equally credible.
Finally, I would like to hear an explanation of why it is A-OK for someone in Hansen’s taxpayer-paid job [a job which allows him to arbitrarily “adjust,” “homogenize” and otherwise alter past climate data], to accept huge amounts of cash from groups that want him to push their agenda.
James Hansen is bought and paid for. He has endorsed lawbreaking to achieve his ends, therefore he is unethical; QED. So the presumption that he has deliberately corrupted GISS is warranted, IMHO.
Stephen,
You can do whatever analysis you want and change whatever numbers you want in your analysis. As long as everyone knows that what you have is not the history but the analysis. Is that how it is perceived by everyone? Do the purveyors of the analysis disclose that and make sure everyone knows it? Or are you trying to say that analysis and history are the same thing? You may have a different analysis tomorrow or next week in light of new information. Do you see what I’m saying?
Andrew ♫
Sorry, I spelled Steven wrong, my apologies.
Analogy: I got paid $20 instead of the $25 a week I’m supposed to get, from 1970-1990. Payroll discovered the mistake yesterday and I got retro paid and am getting the correct weekly salary *now*.
Still doesn’t change the fact that I got paid $20 instead of $25 all that time. If someone said I got paid $25 a week from 1970-1990 they would be wrong. They could say I was *supposed* to get $25 from 1970-1990.
Andrew ♫
Andrew,
Yes, I do see your point, and agree that a casual observer is unlikely to realise that the temperature analysis has been subject to processing. Of course, this is true of all the analyses, not just GISS. The satellites don’t even directly measure temperature in the first place, and the fact that RSS, UAH and others come up with different figures from the same input data shows us that there is no ‘absolute truth’ to be had. The satellite records are themselves subject to ongoing corrections, of course. Why are many people so concerned with USHCN/GISS corrections and not with those others, I wonder? The sum total of corrections to the global analysis does not seem to me to be of great consequence in terms of how it affects our judgment as to what to do, if anything (that’s JMV, of course, and I realise that others will disagree). Anthony’s link above is, of course, to the US analysis. Either way, it’s hardly evidence of deception, since the adjustments are openly explained in Hansen’s papers. I think we may have to agree to differ as to whether or not it offers us a more realisitic view of the true history of actual mean temperatures, as opposed to the history of recorded observations. 🙂
Smokey,
Hansen has accepted at least three quarters of a million dollars [that we know about] from entities with a strong global warming/AGW agenda
Do you mean in terms of personal remuneration? Do you have links in respect of that? I certainly didn’t know of it.
while the Best Science site is run on a pretty much voluntary basis.
Who are you gonna believe?
Well, I certainly don’t believe something because it’s voluntarily funded. There’s all sorts of stuff on the internet of that kind that I don’t believe! Actually, I know nothing of however Anthony Watts makes a living and think that’s none of my business. I try to assess and check out what I read regardless of such knowledge.
There isn’t much difference between believing what someone on the green payroll says, and believing what tobacco company front groups say. They are equally credible.
So therefore you don’t think that Fred Singer, for example, is credible? Personally I think it’s best to look at the science rather than making presumptions of that kind (I actually don’t think Singer is credible, but that’s because of what he has said rather than because of his associations with tobacco).
James Hansen is bought and paid for. He has endorsed lawbreaking to achieve his ends, therefore he is unethical; QED.
If you mean his testimony in respect of the Kingsnorth Six then no, he has not endorsed lawbreaking. They were found not guilty, and therefore did not break the law. You may think they should have been found guilty, but a UK jury decided otherwise. To suggest that justifies “the presumption that he has deliberately corrupted GISS” only explains your personal attitude.
Steven Talbot:
Hansen’s payola has been widely reported. Google “David foundation, Hansen” to start. Or “Hansen, Gore”. Or check this out:
http://www.canadafreepress.com/index.php/article/3671
It’s interesting that of the thousands upon thousands of public servants being paid solely by taxpayers, this particular individual gets so much outside loot — and only from organizations with a very heavy, one-sided, pro-AGW agenda.
Big money like that is a corrupting influence; that’s its unstated purpose. It’s like a local hood paying off the beat cop on the side. Justice takes a back seat. Every time Hansen has taken a big chunk of cash he has ratcheted up his wild-eyed AGW scenarios. That should tell you all you need to know about what’s going on.
If Hansen hands that cash to the U.S. Treasury or to charity, I’ll retract. But Hansen appears to be bought and paid for by outside interests. Where does that leave honest science? Where does that leave the taxpaying public?
Steven,
My follow-up about any analysis is this: Does it sound like good judgement to say the recorded past temp is not good enough to use by itself, but it is good enough to stack adjustments on? That it’s wrong to use but right enough to use?
It would seem to me that you would have to know what the temp is supposed to be before you can do any reliable adjustment for your analysis.
Andrew
The gold standard study on facilitated communication used untrained volunteer college students. The subjects were told to only allow the person with autism to lay their forearms in their open hands above a desk. They were not told why or what the person with autism was supposed to do and did not see the keyboard until they entered the room. They were then given a question along with the person with autism, and then “supported the forearms” of the person with autism while they typed out the answer. The answers were clearly connected to the question. A second random group of untrained volunteer college students were given the same task of “facilitating” the person with autism to type out the answer to a question but were not told what the question was, only the person with autism was given the question. The typed answers were nonsense. Unwittingly, but with good intentions, the facilitators were internally motivated to help produce what they thought was the desired outcome, but only if they had come to some conclusion in their own mind of what the outcome was supposed to be beforehand. It is human nature to be biased.
Scientists must always struggle, daily and even every hour, to remain relatively free of bias. Trust me, I know this feeling. Hansen does not appear to even try to guard himself against it, and because of that lack of diligence, I believe is at high risk of biasing his own research.
Pamela,
Indeed. We need to have a disclaimer attached to everything from climate science that says:
*All temps are subject to change at a later date. The data we used to reach that conclusion is subject to change at a later date. The conclusion you just read is subject to change at a later date. D’oh… Never mind.
Andrew ♫
I still seriously doubt the majority did so unwittingly. There’s no way to tell of they were doing something on purpose or not, except their word.