Guest post by John Goetz
The GISStemp Step 1 code combines “scribal records” (multiple temperature records collected at presumably the same station) into a single, continuous record. There are multiple detailed posts on Climate Audit (including this one) that describe the Step 1 process, known affectionately as The Bias Method.
On the surface seems like a reasonable concept, and in reading HL87 the description of the algorithm makes complete sense. In simple terms, HL87 says that:
- The longest available record is compared with the next longest record, and the period of overlap between the two records is identified.
- The average temperature during the period of overlap is calculated for each station.
- The difference between the average temperature for the longer station and shorter station is calculated, and that difference (a bias) is added to all temperatures of the shorter station to bias it – bringing it in line with the longer station.
- The two records can now be combined as one, and the process repeats for additional records.
In looking at numerous stations with multiple records, more often than not the temperatures during the period of overlap are identical, so one would expect the bias to be zero. However, we often see a slight bias existing in the GISS results for such stations, and over the course of combining multiple records, that bias can be several tenths of a degree.
This was one of Steve McIntyre’s many puzzles, and we eventually figured out why we were getting bias when two records with identical overlap periods were combined: GISStemp estimates the averages during the overlap period.
GISStemp does not take the monthly data during the overlap period and simply average it. Instead, it calculates seasonal averages from monthly averages (for example, winter is Dec-Jan-Feb), and then it calculates annual averages from the four seasonal averages. If a single monthly average is missing, the seasonal average is estimated. This estimate is based on historical data found in the individual scribal record. If two records are missing the same data point (say, March 1989), but one record covers 1900 – 1990 and the other 1987 – 2009, they will each produce a different estimate for March, 1989. All other data points might match during the period of overlap, but a bias will be introduced nonetheless.
The GISS algorithm forces at least one estimation to always occur. The records used begin with January data, but the winter season includes the previous December. That December datapoint is always missing from the first year of a scribal record, which means the first winter season and first annual temperature in each scribal record is estimated. Thus, if two stations overlap from January 1987 through December 1990 (a common occurance), and all overlapping temperatures are identical, a bias will be applied because the 1987 annual temperature for the newer record will be estimated.
Obviously, the bias could go either way: it could warm or cool the older records. With a large enough sample size, one would expect the average bias to be near zero. So what does the average bias really look like? Using the GISStemp logs from June, 2009, the average bias on a yearly basis across 7006 scribal records was:
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
“Ron de Haan (03:28:29) :
CNN Biaha Blanca Argetina, winterconditions, snow, temp -18 degree Celsius?”
And yet, yesterday here in Syndey, Australia, it was the warmest July day in 19 years, about 7C above “average”, it was quite balmy TBH. According to the BoM however, 1990 was the highest max on record for Sydney in July.
These “hot/cold” swings are quite intriguing indeed. Mind you when Lt Cook did his observation of Venus crossing the Sun in Tahiti in 1769, temperatures rose to on the day to a max of 119F.
Curiousgeorge hits nail on head.
This is not how responsible, accountable people would handle data.
Would it be naive to suggest that a policy think tank, like Hoover or Heritage, start an intensive program to systematically review all the temperature-station data and their manipulation over the past 150 years?
It’s wonderful that volunteers like Anthony and his colleagues devote countless hours to this effort, but it’s clearly such a huge task that it cries out for someone with the bucks to hire a team of analysts who can undertake a thorough ‘stem to stern’ review of the US and world data underlying the claims of AGW.
These claims are now the basis of major public-policy legislation (e.g. ‘cap-and-trade’), so it is way past time for them to be assessed by professionals without a policy ax to grind. You could argue that a conservative organization like Heritage would have such an ax, but a temperature-data-review program could be insulated from biases, or established at an institution without an ax in the ‘AGW’ dispute—Rand?
Or maybe a non-political foundation could be approached with the aim of funding an independent review effort, called perhaps “The World Temperature Audit.”
One thing for sure: it should not take any government money.
/Mr Lynn
@Curiousgeorge (03:35:41) :
“Would anyone like their bank account or 401K handled in this manner? I don’t think so.”
Yes, if they have the same upward bias :o))
No, if the earlier balances are adjusted downwards :o(
“The GISS record is highly suspect. I said so on a warmists blog and have now been banned from there. They do not tolerate dissent.”
Is there any warmist blog which tolerates dissenting views? I haven’t found one yet.
“liars, D@mn liars and statistics” needs to be changed to “liars, D@mn liars, IPCC liars and statistics” IPCC makes used car salesmen look honest and a lot less dangerous to our pocketbooks.
OFF TOPIC:
I have not seen anyone addressing the effects of irrigation on “Global Warming” and sea level rise. IPCC climate models ignore water as a greenhouse gas so the effect of “Modern Farming Techniques” characterizing “the Green Revolution” would not be studied. The Green Revolution refers to the transformation of agriculture beginning in 1945. It packaged specially bred High Yield Varieties with fertilizers, pesticides, and irrigation.
http://www.crystalinks.com/greenrevolution.html
http://www.copperwiki.org/index.php/Green_Revolution
http://www.i-sis.org.uk/announcingSIS36.php
Irrigation is a significant use of water world wide. “…Agriculture, especially irrigated industrial farming and
livestock production, typically put the single largest demand on surface and
aquifer resources. In water-scarce regions, irrigation can consume well over
three-quarters of total water withdrawals….” http://unfccc.int/resource/docs/napa/com01e.pdf
http://www.whirledbank.org/development/sap.html
http://www.stwr.org/imf-world-bank-trade/creating-poverty-world-banks-latest-passion.html
“….Under the USAID program, farmers set aside a small
amount of their land for fish farming and either dig a pond
or build one inside earthen walls….In a typical situation, farmers produce 1,500 kilograms of fish
per hectare a year, providing food for their families as well as
a new cash crop. The ponds help farmers cope with drought
and enable them to raise crops like cabbage and tomatoes that
require irrigation during the dry season….” http://www.worldwatch.org/press/prerelease/wwp172.pdf
African irrigation projects: http://africanagriculture.blogspot.com/search/label/irrigation
The World Bank/IMF SAP has intentionally bankrupted third world farmers, the land has then been bought up and turned into plantations using modern farming techniques. “…According to a study by Jose Romero and Alicia Puyana carried out for the federal government of Mexico, between 1992 and 2002, the number of agricultural households fell an astounding 75% – from 2.3 million to 575, 000….The vacuum created by retreat of the Mexican state from agriculture was filled by large US and Mexican agribusiness…..Transnational agri-business tends to have much closer links with larger farmers and producers, who have better access to land, irrigation and credit, all of which are scarce commodities for small farmers…” http://www.countercurrents.org/mohanty230608.htm
Europe has also been the target for modernization of farming. According to an article, 2001 Polish entry into the European Union: “…a meeting with the Brussels-based committee responsible for negotiating Poland’s agricultural terms of entry into the EU…the chair-lady said: “I don’t think you understand what EU policy is. Our objective is to ensure that farmers receive the same salary parity as white collar workers in the cities. The only way to achieve this is by restructuring and modernising old fashioned Polish farms to enable them to compete with other countries agricultural economies and the global market. To do this it will be necessary to shift around one million farmers off the land …a lady from Portugal, who rather quietly remarked that since Portugal joined the European Union, 60 percent of small farmers had already left the land. “The European Union is simply not interested in small farms,” she said….” http://www.i-sis.org.uk/savePolishCountryside.php
Here in the USA all we have to do is look at all the irrigated lawns and golf courses, not to mention farms in Arizona, New Mexico and California to see the major effects of irrigation. Not only is water brought in direct contact with the air, but plants transpire also putting water in the air. The “lake effect” seen around bodies of water show that water DOES have a climate effect.
Have there been any studies done on the effects of irrigation on climate? If 50 to 100 ppm of CO2 is supposedly cause for alarm what has all those millions of gallons of irrigation water done? Since much of the water is “fossil water” what is its effect on the sea level? “Modern farming” also cause a major loss of soil that then ends up in the sea. On my farm alone we’ve lost over 2 feet of topsoil as well as an unknown amount of subsoil. I have seen 4 inches plus my seed disappear in one rainstorm before I learned not to smooth the soil when I planted grass for my pastures.
Speaking of sea level, they now estimate there are 3 MILLION volcanoes under the seas!
“..The team estimates that in total there could be about 3 million submarine volcanoes, 39,000 of which rise more than 1000 metres over the sea bed….” http://www.newscientist.com/article/dn12218
How come no one ever points to volcanoes as a major source of CO2? “…Volcano Outgasing of CO2. The primary source of carbon/CO2 is outgassing from the Earth’s interior at midocean ridges, hotspot volcanoes, and subduction-related volcanic arcs….” http://www.columbia.edu/~vjd1/carbon.htm
Basil (19:22:24)
Very, very few North American stations have multiple scribal records that are combined. The bulk of the stations come from Asia, primarily Russia, although there are a number from Europe, Africa, and the Middle East. So the biasing really affects them and is not a world-wide phenomenon. Furthermore, without running the GISStemp process through to the end, it is impossible to know how the cell gridding impacts the result. It might increase or lessen the effect.
It seems the bias and just plain bad science has been noted by the several university geology departments.
“…It has also been established in the literature that biases, inaccuracies, and
imprecision have been introduced to the climate monitoring systems because of
meteorological station moves, instrument changes, improper exposure of instruments, and
changes in observation practices …It has become clear from various studies (e.g., Pielke et al. 2007a) that data used
in existing long-term climate assessments including the U.S. Historical Climatology
Network (USHCN) have undocumented biases that have not been corrected using data
analysis and data adjustment techniques….”
Impacts of Land Use Land Cover Change on Climate
and Future Research Priorities
: http://climate.agry.purdue.edu/climate/dev/publications/J91.pdf
John Goetz
One quick question. How many stations (what proportion) does this affect. I assume if a complete (e.g. 1880-2009) station record exists then the “Step 1” procedure is not used.
John F. Hultquist (19:24:08)
I don’t believe a method was selected to create a warming bias. As I said, the bias method in and of itself seems like a reasonable approach. Its when the estimation process is thrown into the mix that the whole process starts to fall apart.
The insistence on using the monthly / seasonal / annual averaging and estimation process prior to calculating bias is the real problem. Would it not make far more sense to simply take the monthly data in common between the two records and simply average those values without introducing a convoluted estimation process? Manual inspection of the records tells me that far more often than not there is no difference between the periods of overlap, so no bias would be introduced.
Often times where there is a difference it points to an error in an upstream process that can be corrected. For example, I have seen the occasional records where, say, January 20 is recorded in one place as being 5 C and in another as being -5 C. When monthly averages are calculated for each of the two scribal records, they differ slightly – and that will introduce bias when they are combined.
Furthermore, I have seen instances where an entire month will be dropped from the GHCN record if a single day in that record is suspicious. In the above example, 5C and -5C might be plausible January temperatures and not raise suspicion. But -24C in June is suspicions. The GHCN quality-control process assumes the entire month might be bad and so they drop the value, forcing GISS to later estimate it. However, manual inspection might show that the day before had a temp of 21C and the day following 25C. You think there might have been an error with the sign that could easily be fixed, allowing the month to be “saved”?
John Goetz “multiple temperature records collected at presumably the same station”
How can there be any overlap if the temp records are from one station? There could be gaps, but no overlap. WUWT?
More blatent Climate fraud:
ul 23, 2009
Pacific Northwest Snow Pack – the True Story
By George Taylor
Washington Governor Gregoire recently sent a letter to the Washington House delegation in which she stated that the snow pack has declined 20% over the past 30 years: “Last month, a study released by the University of Washington shows we’ve already lost 20% of our snow pack over the last 30 years.”
Actual snow pack numbers show a 22% INCREASE in snow pack over the past 33 years across the Washington and Oregon Cascade Mountains:
image
Larger image here. See post here.
ICECAP NOTE: In this story on Sustainable Oregon, George shows how choosing start and end dates makes all the difference in trend analysis. This is true because precipitation trends in the northwest are linked to the PDO cycle of 60 or so years. In the cold phase, La Ninas and heavy snowpacks are common (like the last two years) and in the warm phase, El Ninos and drier winters (as was the case from the 1970s to late 1990s). By cherry picking his start data as 1950 at the very snowy start of the cold PDO pahse from 1947 to 1977 and ending in 1997 at the end of the drier warm PDO phase from 1979 to 1998, Mote was able to extract a false signal which he attributed to man made global warming.
Arguing this point made George Taylor, state climatologist for decades in Oregon a target (he took early retirement) and cost the assistant state climatologist in Washington, Mark Albright, his job. Phil Mote, the alarmist professor and author of a discredited work on the western snowpack for the Bulletin of the American Meteorological Society doesn’t accept criticism lightly. He ironically was appointed to the state climatologist position George Taylor held in Oregion. It was Phil who fired Mark for challenging his findings. That is the way it is in the university climate world today, real data doesn’t matter so don’t bother to look and if you need to pick and choose carefully. Anyone who disagrees publically and risks funding need look elsewhere for employment.
George shows the 1950 to 1997 trend and the longer term trend analysis for several stations with good records showing no discernible long term trends.
The story doesn’t end there as this post by Jeff ID called SNOWMEN tells, another climate schiester, Eric Steig who made the headline last year when he worked with Michael Mann, the king of data fraud to eliminate the antarctic cooling of the last several decades. Eric chimed in against Taylor and Albright defending Mote and making false or at least uninformed claims about trends. It is clear from Steig’s Real Climate post never even looked at the whole data trends. Jeff correctly notes “These plots are of specific stations, however they demonstrate that at least for the above locations the 1950-1997 trend is a cherry pick, nothing more.”
Unfortunately this bad analysis has gotten people promoted and been used by state governments to make unwise decisions like supporting the flawed and costly and totally unnecessary WCI (Western Climate Initiative), which Paul Chesser writes about in this American Spectator story here. Climate frauds like Mann, Mote and Steig have a lot to answer for, if the governments measures inflict major pain on the citizens and the globe continues to cool in its natural rythym.
http://www.icecap.us
Climate Fraud gets a face:
Jul 22, 2009
Science Czar, John Holdren’s Goldman Sach’s Connection
The Liberty Journal
As I was doing some research in some non-profit’s literature, appeared before me was a 2006 picture of John Holdren, Bill Clinton, and this other guy (name not mentioned). So what, you say. Well the caption indicates, John Holdren’s Woods Hole Research Center Director excepts $1mil check from Goldman Sachs Center for Environmental Markets (CEM).
image
Woods Hole Research Center describes themselves: “The Woods Hole Research Center is an independent, non-profit institute engaged in fundamental environmental science, applied policy analysis, local and regional capacity building, and public and policy-maker education aimed at clarifying the interacting functions of the Earth’s vegetation, soils, water, and climate in support of human well-being and promoting practical approaches to their sustainable management in the human interest.”
In other words, they’re another rich environmental think tank 501(c) non-profit with rich members, well connected to the corporate world, who use their income to influence public policy to further increase their wealth. The Woods Hole Research describes this venture with Goldman Sachs and Bill Clinton:
“A new partnership between the Woods Hole Research Center and The Goldman Sachs Center for Environmental Markets (CEM) announced yesterday at the Clinton Global Initiative will develop new market-based approaches to value the sustainable uses of forests for marketable products and ecosystem services.”
John Holdren is obviously excited as stated:
“It’s particularly gratifying that we developed this project with Goldman Sachs as part of the Clinton Global Initiative – a farsighted effort of the former President to stimulate new partnerships among businesses, researchers, and public-interest organizations to address the great challenges in global health, environment, and economic development. This is not only a grant but also a partnership, in which insights from the Woods Hole Research Center about how forests work and what is needed to keep them working will be linked with expertise at Goldman Sachs about the economic forces and incentives that affect how forests are used and managed.”
Maybe Holdren and Goldman Sachs share ideas while they are at the Council on Foreign Relations meeting. Maybe it’s that John Holdren speaks at Goldman Sach’s conferences, like the “Energy, Environment and the Financial Markets: The Global Opportunity” in London.
Well, they want to make sure they know the value of every last tree and forestland on the earth. Ok, Goldman Sachs is your company if you want to figure how to equate everything with some monetary value as to create an investment from it.
Now to enforce that idea. From their website, here is how Goldman Sachs describes the CEM: “The Environmental Markets Group manages the Goldman Sachs Center for Environmental Markets. The Center works with independent partners in the academic and non-government organization communities to examine market-based solutions to environmental challenges. Two of their primary goals are : (1) Market-making in carbon emissions and other climate related commodity markets and (2) Launching GS SUSTAIN, a global equity strategy that incorporates environmental, social and governance issues into fundamental analysis of companies
Well, it’s merely the rich using tax free big bucks through non-profits to grease palms and divy up the spoils of their pillaging of tax-payer coffers. Only in American politics. See post here.
See SPPI’s new paper on climate money here by Joanne Nova. It starts out: “The US government has spent over $77 billion since 1989 on policies related to climate change, including science and technology research, administration, education campaigns, foreign aid, and tax breaks. Despite the billions: �audits� of the science are left to unpaid volunteers. A dedicated but largely uncoordinated grassroots movement of scientists has sprung up around the globe to test the integrity of the theory and compete with a well funded highly organized climate monopoly. They have exposed major errors.”
**********************
The Morality of Climate Change: (Uploaded 18 July 2009)
One has to know all the facts to determine the morality of an issue. John Christy on CO2Science.org, below and enlarged here.
See this town hall attack on Mike Castle, one of the Republicans who voted FOR Cap-and-Trade below.
“Well its now snowing in Ezeiza airport Buenos aires so much for AGW
http://www.perfil.com/contenidos/2009/07/22/noticia_0033.html spanish”
For those of you that do not read Spanish, the article speaks of an intense cold wave covering a large region of Argentina, and accompanied by below zero (Celsius) temperatures, strong winds, downed electrical lines, road closures, and damage to structures.
Continued falling temperatures are forecast for Buenos Aires.
This is pretty significant, and I would expect it to be picked up by the International news media pretty soon.
There are a few comments posted; One mocks global warming, and says that it is not happening, and one expresses concern that thousands of indigenous people may die of the cold.
Thank you, Mr. Goetz, for your analysis. So far, it appears to be very relevant; and again — this is only one step that introduces a suspicious result. There are many others. I am intrigued by your observation that multiple scribal records is overwhelmingly a non-U.S. issue; many GW pessimists point out that the U.S., representing only 2% of the earth’s surface, may not be experiencing warming but the rest of the world is.
John Goetz (05:46:56) wrote to John F. Hultquist (19:24:08)
I don’t believe a method was selected to create a warming bias.
However E.M. Smith writes (22:25:21) :
“It’s even worse than that… The first step in GIStemp is actually STEP0, and that step does a couple of suspect things all by itself. (Cherry picks 1880 and deletes any data older than that. Takes an “offset” between USHCN and GHCN for up to 10 years from the present back to no earlier than 1980 then “adjusts” ALL past data by subtracting that “offset” – supposedly to remove the NOAA adjustments for things like TOBS, UHI, Equipment, etc., …)”
And: “Those kinds of Logical Landmines are scattered through the whole thing.”
And: “IMHO, this code is more of a glorified “hand tool”. Something that someone cooked up to let them play with the data. There are lots of places where you can insert “plug numbers” to see what happens.”
IMHO the entire global temperature record MUST be reexamined by real committed scientists. Ever since Anthony’s invaluable expose on what kind of temperatures our surface stations are measuring we have known that a reevaluation is required.
Thanks to John Goetz for these great efforts and to E.M. Smith for going into more depth. And Steve McIntyre deserves kudos for being among the first, as far as I know, to take on the “evil geniuses” (stupidheads or manipulatormaniacs).
Mac at 19:48 has given us the article that follows the money.
http://scienceandpublicpolicy.org/images/stories/papers/originals/climate_money.pdf
It’s time to throw the [self-snips] out. No disrespect to mothers and their great efforts at discipline, but real science and real representative democracy also requires the “no” of the fathers. This bunch is on the take and is trying to take us to the cleaners.
This is a surprising result-a seemingly innocuous if strange methodology actually leading to a warm bias.
Some commenters wanted to test if the method ALWAYS leads to warm bias but I don’t think that’s the right question. The question should be, is the result leading to a warm bias in REALITY?
It probably is pure chance but man, what a ringer!
Correction: Steve McIntyre was among the first to expose the “evil geniuses'” many computer programming tricks. I will never be able to believe that they began in innocence, their mistakes just ran away with them, and now they must save face. Too much money is floating around for the climate scam.
“Richard Sharpe (20:44:13) :
I don’t know about GissTemp whatever, but something is up with the weather in the San Francisco Bay Area.
We are having very cold nights and cool days in the middle of summer.”
I know it’s called weather….but Ky just had it’s first July ever without a single day in the 90’s….or at least that’s the forecast. Going down on record, if it holds, the coldest July since records were started.
I’m no statics expert but the end of the graph seems very different than everything up until 1982. The early changes seem to be much more gradual and follow a general trend. Have recent temps really been that different that the extreme changes up and down have not occurred prior? It looks to me like either temps have been fluctuating much more wildly or the data is being treated differently. Can someone explain this for me?
Thanks, John
This off topic but may be of interest. Dr. David Evans points out a new trick that the Alarmists are trotting out: that the atmosphere may be cooling but the oceans are warming.
“Senator Steve Fielding recently asked the [Australian] Climate Change Minister Penny Wong why human emissions can be blamed for global warming, given that air temperatures peaked in 1998 and began a cooling trend in 2002, while carbon dioxide levels have risen five per cent since 1998. I was one of the four independent scientists Fielding chose to accompany him to visit the Minister.
The Minister’s advisor essentially told us that short term trends in air temperatures are irrelevant, and to instead focus on the rapidly rising ocean heat content…”
http://www.globalresearch.ca/index.php?context=va&aid=14504
“Sanity is not statistical.”
– George Orwell, 1984, Book 1, Chapter 9
The net effect is a tenth of degree in more than 100 years, that’s not much.
It’s a fair percentage, though. USHCN1 raw data from NOAA stations shows a 0.14C warming average per station (equally weighted) over that period. Fully adjusted USHCN1 is +0.59C per station. TOBS-only is +0.31. (Fully adjusted and gridded USHCN2 is c. 0.72C.)
Consider the following possibility (my own musings) in the average global temp. Warm oceans produce a narrower band of lows and highs without extremes but the band is higher up on the thermometer. Colder oceans produce a broader band of highs and lows with extremes on both ends. Now consider that warm and cold conditions occur regularly along with neutral conditions, with warm occuring more frequently during El Nino oscillation, and cold occuring more frequently during La Nina oscillations. What would the average turn out to be in these two cases? It would be interesting to look at the range of temps as well as record events and then correlate with ENSO to see if this idea of an average global temp could be misleading.
John Goetz, I’m sorry but I couldn’t resist when I got to the bottom of AnonyMoose’s post. I know that what you are doing takes a huge effort, and I applaud that effort more than anyone. I did not intend to detract from the posting in any way. Your work is very important, and appreciated.