To answer Paul H.’s question indirectly: Here is the HACRUT3 data for NH, SH and global, moderately smoothed:

http://www.woodfortrees.org/plot/hadcrut3vgl/mean:60/plot/hadcrut3vnh/mean:60/plot/hadcrut3vsh/mean:60

So yes, it has warmed a bit, and may still be warming a bit, but there’s no evidence in these records yet of anything that looks like runaway positive feedback.

On the question of data manipulation, I personally think the different datasets tally pretty well, and since they’re all derived differently any adjustments are not having a great effect on long-term changes, which is all that really matters. So as an engineer if I ask myself “can I rely on this data?”, my answer would be “Yes, to a first approximation over long term trends”.

More notes on comparing different datasets, and in particular the baseline issue: http://www.woodfortrees.org/notes#baselines

Cheers

Paul (R) Clark

http://data.giss.nasa.gov/cgi-bin/gistemp/do_nmap.py?year_last=2008&month_last=6&sat=4&sst=1&type=anoms&mean_gen=06&year1=2008&year2=2008&base1=1951&base2=1980&radius=1200&pol=reg

Excellent questions and the crux of the debate. Sorry to say that the answers are highly subjective and time sensitive. There are so many variables to take into account that a definitive answer to “are we warming” is impossible or incomplete at best. With the difference in measuring devices, locales, times of observations, changes in any of the these, and throw in the accuracy of the observer, to make any judgment is premature. As Anthony has pointed out with his auditing of stations and Steve McIntyre’s overall Climate Auditing, it becomes clear that the strict discipline needed in Science is regretfully absent in the field of Climate Science and makes answers highly suspect and open to question.

I’ll have a shot at answering this.

First of all, there are 2 questions that need answering. One is, is the world’s climate warming and the second is, is it due to a global effect. Because we know peat fires in Indonesia, coal fires in China, etc, are excerting a warming influence on the climate.

The answer to the first question is, on average, probably yes.

The answer to the second question is more important. A global effect should show up equally in both hemispheres (north and south). Even though, ex tropics, the SH has less than 5% of the population of the NH.

There has been no significant warming over the entire satellite record for the SH (about 30 years).

So the obvious conclusion is that whatever warming that has occured in the NH is not due to a global effect and therefore greenhouse gases cannot be the cause because they are distributed moreorless equally across the hemispheres.

“But I can’t see adding data where none exists is constructive here. Anything added that changes a trend (or affects the existing record in any way) would be pure speculation; weather hardly ever does what we tell it to, even if before it happens, but especially after. Perhaps I’m still missing something here, but it doesn’t sound like sound science. How far could you stretch this vacation (30 days… 3 years), how would you set limits on “accuracy”?”

There are certainly limits, but science has always dealt with measurement errors and omissions. You certainly don’t want to throw out the entire 100 year history of a good site because someone lost a monthly data sheet or the tanscriber couldn’t make out a poorly written digit.

Stepping well beyond my statistics knowledge, one reason to fill in missing data with a good guess is that there are statistical methods that need a full history. One of them, (Principle components) was used in producing Mann’s hockey stick graph, though it had more problems with extrapolating data beyond a sample period than filling in holes. And many, many other problems to boot.

Some times the missing data can be omitted without much trouble, e.g. gaps in USHCN and COOP records during equipment relocations, Anthony frequently posts graphs with breaks in them. The breaks are convenient markers to see what short of shift in data occurs with the relocation.

Given all the other error sources in the data, filling in missing data with something close to neutral provides a lot of computational convenience at little cost.

Here’s an example – suppose a site had that April 1933 data missing. How would you compute the average temperature for that year? If you don’t, then that would suggest you couldn’t compute an annual average for the country that year because there ought to be at least one missing month somewhere for each month of the year. By splicing in neutral data, then all the simple averaging math (e.g. add up the monthly averages and divide by 12) still work.

When dealing with issues in my working life I have always found the best question at the start is: “What is the problem we want to fix?” (BTW – the end game should always answer clearly the question – “why is this strategy/policy/plan the answer to that problem?”)

Therefore – my first question is – “Is the earth warming?” (Please stay with this point and do not get onto secondary issues such as – is it dangerous? can we forecast what will happen? – and what is causing it? – question ‘1′ has to be is the earth warming?)

Now, with all such issues as this my basic logic is to say there are 2 key factors (to me as a layman please understand)
1). what is the appropriate start point?
2). what is the data that shows this?

(Please understand I am trying to be purposefully simplistic here)

Now, depending on which side of the argument you are on you will naturally choose a start point that fits with the thesis you are trying to make. I would like to put that to one side for the moment and concentrate on getting clear in my mind an understanding of the data.

I have been singularly horrified by how much data manipulation may be going on in this area. I am not a scientist but I do suffer the consequences, as a citizen in my country, of decisions made by politicians on the basis of government policy formulations founded on the data produced about global warming/climate change.

So all I ask is:

1). ‘Is there a clear set of data that shows the average temperature of the earth over the last 100 hundred years?’

2). ‘Is this data available/reported (in its raw form as well as its adjusted form) in order that scientists around the world can evaluate it and have such assessments peer reviewed with rigour?’

Anthony: Odd. GISS goes down while UAH and RSS go up slightly.

Could just be noise, but it might slightly confirm my hunch that the satellite datasets react quicker and more strongly to short-term events (e.g. La Nina).

I keep a local copy of Hadley/GISS/UAH/RSS at home. I also update the entire dataset, instead of merely adding the latest month. I was about to make a fool of myself by crowing about 12 consecutive months of temperatures falling year-over-year at GISS. Then I did a double-take. The GISS dataset with data to May 2008 showed March 2007 as 0.60 and March 2008 as 0.58, a 0.02 decline over 12 months. The GISS dataset with data to June 2008 shows March 2007 as 0.59 and March 2008 as 0.60, a rise of 0.01 over 12 months. They’re the only one showing a rise over the previous year for March 2008. Anything to deny the skeptic community a propaganda point.

The GISS 12-month running mean anomaly to June 2008 is 0.421. The GISS 12-month running mean anomaly to March 1998 was 0.422 (at least on the current GISS dataset :p ). So we’re back to where we were over 10 years ago, according to GISS.

Dr. Hansen should note that in the real world, corporate executives do not get thrown into prison for disagreeing with him, but rather for fiddling with corporate numbers. GISS restates previous years’ numbers more often than Nortel, fercryinoutloud.

This is a case where the computational algorithm is determining the data required, where it should be the other way around. And the algorithm should handle the data available. Calculating means and trends where data is missing from a time series isn’t a hard problem.

Amateur is too weak a word for this.

“Ric, how do you adjust missing data? I must be missing something.”

http://wattsupwiththat.wordpress.com/2008/04/08/rewriting-history-time-and-time-again/
describes the process pretty well.

Suppose the observer for Penacook was on vacation for part of April 1933 and hence wasn’t able to come up with the 30 days of data to compute the average for the month. Averaging the days he was around for doesn’t work, escpecially if he was away during the beginning or end of the month when temperatures are usually furthest from the mean. Replacing the missing data with daily averages might work pretty well, and may be what should be done.

Instead, the missing month’s data is filled in by the other two months in the season and the average temperature of the month in that season for all other years. As the temperature record gets longer, the amount of data gets longer.

It would make a lot more sense, at least to me, to use something like the average of the preceding month, the following month, and the previous year’s month, and the following. Maybe a few other years too, but certainly no more than a few.
Probably some use of averages for the month and adjacent ones too.

There are some algorithms for doing this, I have one that’s used to take a collection of data samples and spread them out over an even matrix. If I ever have time, I want to try going from GPS traces to topographic map and I need that step to make some contour drawing software happy.

http://nsidc.org/data/seaice_index/images/daily_images/N_timeseries.png