All the curves that are fit to print

Now if only the mainstream media will pick up on this…. Hah!

For a layman, could you explain the difference between 1st order and 5th order curve fit in a 30 year period? Obvoiuosly, the 1st takes a longer time frame, but is it 5 times longer?

Hansen postulated global cooling first, before big Al sucked him into the warming camp, where his investments are. Now, for the long view, why is no one saying that we are actually in a very long term cooling trend, and warming is just a temporary glitch? The opposite argument sure gets a lot of traction with nothing more than some impenetrable climate models which seem to have less and less connection to reality. The refusal to admit COO rises after warming occurs, and is not the cause of warming is just one problem.
Still no more cycle 24 sunspots. Hope everyone has their warm clothes ready.

Thanks Anthony,
As I said, I’m a layman to this. I meant no offense by asking you to be “fair”.
This helps me, I appreciate it.
Happy Independance Day!

It seems to me that the fitting of a polynomial curve to this data is inappropriate. Although you can get a polynomial to fit almost any data, it has no meaning other than it seems to fit. My preference for this kind of trending is to use an exponential moving mean that weights the most recent data heavily and earlier data less. The result is a curve that tends to follow the current change in the data which is what we want in this case whether it is going up or down.
Love your site. Read it every day.

First or fifth order curve fits have nothing to do with time frames. A first order curve fit is one of the form y=mx+b. You need at least two points to make such a fit (which would be a wast of time). A fifth order curve fit is of the form y=ax^5+bx^4+cx^3+dx^2+ex+f. You need at least six data points (ie, xy) to perform that. A first order curve fit is a linear (straight) line. A fifth order fit is useful for making a smooth fit through a lot of data, but has little physical meaning (ie, try to project it beyond the last point in Anthony’s graph and it might project down to -infinity or swing around and go to +positive infinity. Now, Anthony isn’t trying to pull the wool over anyone’s eyes by doing that – its just that making a nice smooth fit using a fifth order is easy and fast in excel.
The other “curve fits” that are often discussed are actually statistical deconstructions of wave forms, and assume that temperature follows a complex combination of long and short waves that are combined. The waves are assumed to have a basis in reality in that they could be caused by sun spots or some other periodic influence, and, unlike a fifth (or even first order curve fit) can be projected into the future. THey are used often in financial and other types of forecasting. Those types of deconstructions and take more time (and a good stats package like “R” – climateaudit.org discusses R quite extensively and is a good place to go if you want to see how stats are applied. The R CRAN site is another…
You can do the same – I believe that a few posts back someone made a link to all the data for the last few decades in excel. THere are a lot of good quickie instructions on the web as to how to do the first and other curve fits using excel. I strongly encourage those who don’t know how to fit data (or even how to graph it!) to learn…
Oh, and Neilo – don’t be lazy. It took me three seconds to find the IBD article that confirms what George M stated. Hanson has apparently been trying to find the limelight since the 70’s….
http://www.investors.com/editorial/editorialcontent.asp?secid=1501&status=article&id=275267681833290
Happy Independence Day!

Bob Zorunkle,
If memory serves, first-order and fifth-order refer to the degree of polynomial to which the data has been fitted. Both first-order and fifth-order curves provided by Anthony encompass ALL the data.
A first-order curve fitting is an assumption that the data are following a straight line and that deviations from that line are just random variations. This strikes me as a fantastically stupid assumption on something that is coupled, nonlinear and chaotic, like climate. But hey, most members of the media don’t even understand the concept of slope.
A fifth-order curve is one of the form fx5 + ex4 + dx3 + cx2 + bx + a, and is much more robust for fitting temperature trends.

Bob, in this case, first order and fifth order refer to polynomials of the form:
y=a0 + a1*x**1 + … + an*x**n
which describe a best fit curve to the data. A first order polynomial is a line.

Thanks for the graphs, Anthony. It’s pretty clear that anyone looking at the first two graphs back in 2004 would think things were getting warmer.
I sure hope all the fuss will slack off, and people will start thinking about what they will do long-term to handle climate change, both colder and warmer. It’s a good start that more people are paying attention, and more detail is being developed on the potential effects of both warming and cooling. Long range contingency planning, efforts to better inform our representatives, and acquiring more knowledge on these topics would be a lot more helpful than knee-jerk political bashing.
Tnx again – Tim (www.timprosserfuturing.wordpress.com)

Why not a 200 day moving average? That would show more of the short term moves while smoothing out the peaks and valleys. The 5th order isn’t bad, at least it shows a little bit of the variation. (just eyeballing it, it looks like about 1,000 day moving average, but I’m guessing)
With a little bit of logic, I think I can demonstrate that the first order curve fit is of no analytical value. It appears to be simply a straight line fit through the available data. Here’s the problem: the slope of that line is dependant on the starting point, which was a completely arbitrary decision by the observors. I realize that that is all of the data we have for these measurements, but the underlying temperature function stretches back to time immemorial. The choice of starting point was a human decision that has no relationship to the underlying function. When an arbitrary decision by the obersvor which has no relationship to the underlying function can dramatically skew the results of the curve fit, then that curve fit tells you next to nothing about the underlying function. The fact that the 11 year fit shows a very different slope illustrates this point.
Moving averages may not tell you much about the underlying long term function, but at least they show you relative movement over a defined time period, which seems quite appropriate when dealing with a cyclical phenomena.

neilo:

On July 9, 1971, the Post published a story headlined “U.S. Scientist Sees New Ice Age Coming.” It told of a prediction by NASA and Columbia University scientist S.I. Rasool. The culprit: man’s use of fossil fuels.
The Post reported that Rasool, writing in Science, argued that in “the next 50 years” fine dust that humans discharge into the atmosphere by burning fossil fuel will screen out so much of the sun’s rays that the Earth’s average temperature could fall by six degrees.
Sustained emissions over five to 10 years, Rasool claimed, “could be sufficient to trigger an ice age.”
Aiding Rasool’s research, the Post reported, was a “computer program developed by Dr. James Hansen,” who was, according to his resume, a Columbia University research associate at the time.
So what about those greenhouse gases that man pumps into the skies? Weren’t they worried about them causing a greenhouse effect that would heat the planet, as Hansen, Al Gore and a host of others so fervently believe today?
“They found no need to worry about the carbon dioxide fuel-burning puts in the atmosphere,” the Post said in the story, which was spotted last week by Washington resident John Lockwood, who was doing research at the Library of Congress and alerted the Washington Times to his finding.

From this article.

Out of interest (and I’ve seen it before but just can’t find it anywhere) what is the current “background” warming level?
I seem to remember it is somewhere in the region of 0.6 degrees centigrade per century since the “end” of the last ice-age (end is in inverted commas as I’m was always taught in my geology lectures that we are still in an ice-age as we have icecaps – it is just a warm period inbetween cold ones).

Higher order polynomial regression curve fits are great for smoothing data, and to see trends within the scope of that data. But if you have ever tried to use them to extrapolate outside that scope, you will know the perils of that exercise. Once you exceed the boundary conditions of that fit, all bets are off, and the higher order the polynomial, the less useful it is for predictions beyond the scope of the data, especially when the so-called “dependent variable” (the y axis value) has little dependence on the “independent variable” (the x-axis).
Heck, even first-order regressions are subject to this limitation. Just because you can get a line or curve to fit doesn’t prove a causal relationship.
My $.02
DaveK

Brendan.
We have wind here in Denmark (around 20% of electricity needs are supplied that way) and there is nothing wrong with it at all, as long as it doesn ‘t “stand alone”. In Denmark we have swedish hydro-electric, natural gas and of course oil as back-up, so no problem. Wind is also looking more and more economical compared to the rising oil prices, so don’t knock it.
Question to all:
Isn’t all this talk about “surface temperature” a waste of time. Surely the only
temperature that effects climate is the temperature (not at the surface) of the oceans, as witnessed by PDO, AMO, El nino etc. ???
Surely surface temperatures are too sporadic to gain any real meaning on climate from them ??

Much as I love to see the temperatures giving grief to the alarmists, we might find the temperatures start to turn upwards soon. According to http://www.cdc.noaa.gov/people/klaus.wolter/MEI/mei.html we are probably coming to the end of the La Nina. If we go into an El Nino this can mean a rapid rise in temperatures. Of course if we are in a period like that between 1950 and 1975 when La Ninas seems to follow one after the other, then we may be in for a prolonged period of lower temperatures.

The Engineer:
I don’t agree that surface temperature is a waste of time, since it’s what most people key off of and understand about global warming. The time spent illustrating and discussing that there has been little to no warming for 11+ years will eventually pay off.
I do agree that surface temperature fluctuations are periodic and driven in the most part by oceanic oscillations: the AMO and ENSO, but not the PDO, which is not the simple residual of global SST subtracted from the North Pacific SST, north of 20N. The magnitude of the “North Pacific Residual” is more on the order of the AMO, but the timing of its oscillation is different than the AMO, and the PDO for that matter. The PDO is an aftereffect of ENSO, or at least one paper describes it as such. In “ENSO-Forced Variability of the Pacific Decadal Oscillation”, Newman et al state in the conclusions, “The PDO is dependent upon ENSO on all timescales.” Refer to:
http://www.cdc.noaa.gov/people/gilbert.p.compo/Newmanetal2003.pdf
I’ve illustrated and discussed the difference between the PDO and the North Pacific Residual here:
http://bobtisdale.blogspot.com/2008/06/common-misunderstanding-about-pdo.html
I have 17 posts on Smith and Reynolds SST data, including instructions on how to download it from NOMADS, over on my blogspot. For the Smith and Reynolds posts, I’ve tried to keep my AGW skepticism to myself and to report only what I found.
http://bobtisdale.blogspot.com/2008/06/smith-and-reynolds-sst-posts.html
Regards

Bob Tisdale.
Thanks for the information. I’ll get onto it during the weekend.
My main point was that its the sea that drives the climate, not land.
The sea stores energy for long periodes of time, the earth warms
or cools very quickly.
Trying to compare, or average temperatures in the desert with temperatures
in the antartic, makes no sense to me at all. Climatologists are attempting
to compare thousands of different situations that exist today with thousands of different situations that existed yesterday in chaotic systems
and then trying to extrapolate to some kind of average figure in a stable system. The noise must be almost suffocating.

Anthony,
The mindset of climate science is linear or polynomial functions but they represent a limited class of functions and ones that are hardly seen in nonlinear systems. The climate is highly nonlinear and we see things like the PDO which causes a climate shift, not a simple rise or fall. You could achieve just as good a fit to the data with a step function: flat from 1979 to about 1998 and then a second level from about 2000 on connected by a smooth transition at the 1998 El Nino. The rise in the step is about 0.2C. It’s not what the modelers would predict, but so what.

Engineer –
I love knocking wind. Wind is a backup, and a moderately expensive one at that. My point in showing the article (did you even read it?) is that even if there was a supernetwork of electrical tie ins across Europe, the correlation of wind from the sea to the urals is too strong – and there are too many days during peak demand where you see calms of up to a week or more, making it impossible to rely on wind as your main source of power. I won’t repeat what was said very well in that article – but they even mention the british off shore wind going through extended levels of calm that make them useless.
If you look at any of the specific positions (ie, “areas”) as Anthony has place here, you would see that its not a simple matter to even eliminate noise there.
As for tying all the data together, there are multiple good ways to do so, that will not only statistically tie together the suffocating data, but will also give good estimations of standard deviation. They do however take some skill, and if the underlying data is corrupt or has been “changed” your mileage results may vary…. Kriging is perhaps the most common, with a sound underlying basis to it. There is some “art” to it in creating some of the input search parameters, and kriging the whole world using one set of these may not in fact be appropriate… There are others, but that would be the first tool I would reach for. (See the R CRAN pages – as an engineer, I would expect you to be able to pick up these methods fairly rapidly).
I should mention that, if properly taken, we probably have enough points to apply simple averageing to the data without resorting to kriging. Kriging itslef would not take into account mountains, or seas, or deserts – just the data. You could create a numerical surface and tie together correlations on how you expect the surface to react, but now we are into numerical modeling – and, well, that’s another crazy cat lady house of cats that we could get into…
FFT has been mentioned, but again, it is just one of several tools. I think we all can agree that curve fits are quick and dirty assessments that have little meaning in reality for this application – although a first order is by far the most appropriate, but not a predictor. FFT’s and some other stats methods are much better, and “honor the data” as Andre Journel used to say…

Just a brief reminder…
One of the “shifts” that occurs every 10 years on temperature averages (Normals)
by the NOAA NCDC is that they compute a “new normal” series of temperatures for each observational climate station, using a 30 year period set on each 10th year (1970, 1980, etc), then curve fit it by some polynomial function to create the “normal” daily highs/lows/averages output for each station. You will have to consult the National Climatic Data Center for the particular function used…
We were using the 1960 to 1990 “normals” through 2002, then switched to
1970 to 2000 “normals” which will be in effect until about 2011 or 2012, when
another set of “normals” will be issued…
Looking at the ‘history” of using these 30 year ‘normals,” it becomes obvious that
as you get away from the 1960-1970 “global cooling” and toward the 1998 warming peak, the normals are going to show substantial “warming”…
Just a thought for your discussion and analysis…
-JH-

I should mention that although wind is looking more economical as compared to oil, the connection between them currently is weak. That is, wind cannot be a replacement for oil, oil is not used to create electricity (for the most part – only crazy cat ladies use oil to create electricity)… The much waited for electric car will bring its own issues as rare earth prices will begin to rise when battery demands begin to go through the roof… I have mentioned before that I think nuclear based methanol is an appropriate susbtiture fuel, in that our current infrastructure doesn’t need to change at all. I am open to solar if the promised breakthrough from Nanosolar reaches $1/W – right now, solar is a toy, and an expensive one at that.
Boy, I really thought my sticks would stir up more bees.

Nice write up Anthony!

I’d like to make two basic points:
1. The logic used to suggest that Mr. Hansen believed in global cooling is incorrect. The evidence is that a man who DID argue in favor of global cooling (Mr. Rasool) used a computer program written by Mr. Hansen. The fact that Mr. Rasool used Mr. Hansen’s program provides zero evidence about Mr. Hansen’s beliefs. Mr. Hansen is no more responsible for the global cooling hypothesis than the manufacturer who supplied the hardware on which Mr. Rasool performed his computations. The statements made here about Mr. Hansen are slanderous. Moreover, they are classic ad hominem attacks: arguing that the AGW hypothesis is incorrect because one of its advocates is an evil person. If Osama bin Laden endorses AGW, that doesn’t make it wrong. If Mother Teresa opposes AGW, that doesn’t make it wrong. Let’s dispense with this National Enquirer gossip and concentrate on the facts.
Second, on the matter of the most useful curve to which to fit the data: we should rely on Occam’s Razor to guide our considerations. We want to go with the simplest hypothesis. If we represent our data with a polynomial, then the complexity of the polynomial is directly commensurate with the degree of the polynomial. In other words, y=ax+b is simpler than y=ax + b(x**2) + c. Occam’s Razor argues strongly in favor of the first order least squares fit over the fifth-order fit. This works in favor of the AGW hypothesis.
REPLY: I’m sorry, your story in number one is just absolutely ludicrous. You’d have to assume that they never talked about it or collaborated on it. If Hansen didn’t believe in global cooling at the time then he’d likely not have allowed his connection to it in any way. University research departments are tight centers of collaboration, and if one scientist is in disagreement of others works in the department, they usually don’t collaborate to create a finished product. These two people were just office doors apart, not on other sides of the world where one found a tool to use created by the other and there was minimal collaboration.
It doesn’t much matter what James Hansen, other commenters, or you believe for that matter, nature will be the final arbiter on the AGW issue.
There were lots of university meteorology departments that started research into global cooling at the time, so it would be no surprise that Hansen and Rasool were following the same path as many other departments at the time. It is your logic that is flawed.
– Anthony

I am extrapolating from Oph’s views that the AGW models assume that CO2 is the mechanism. From what little I know about scientific investigation of modeling (I know more about the gold standard scientific method, having used it myself), I would think that the group that came up with the AGW model understood that “A” model is built to explain a theory, not test it. If a theory is being tested, a control model (or even better, several competing models) would be advantageous here (I am also extrapolating from Oph’s view that the CO2 theory has not been proven yet and they are still in the testing mode). That would lead me to suggest to the AGW investigation group that they need several models going at the same time with different mechanistic theoretical assumptions for each.
The research literature is awash with climate theories, not just one climate theory. I don’t understand the eggs in one basket method unless the group is not an investigative group and is instead an agenda group.

Regarding the temperature time series, it is clear that there are a number of variables that have to be factored in to explain it properly.
First and foremost, there is an ENSO index variable that greatly impacts the global time series. Major El Nino in 1997-98, 1987-88, 2000-2005 and 1983-84, minor La Nina events in 2007-08, 1989 and 2000.
Secondly, there are volcanoe impacts with Pinatubo in 1991and El Chichon in 1982.
If one could factor these out in some manner, the time series would have a much different slope etc.

Occam’s razor argues for the simplest but ‘simple’ needs to be clarified. A linear trend, while ‘simpler’ is not always at the top of the list of preferred trend decriptors in nature. Exponential/Logarithmic or periodic curves are more common and hence might be a better fit as per Occam.
Great blog! I read it often but this is my first post here.
~peace~

Brendan: I looked at your link to the work of T.J. Nelson, reading some of it and skimming the rest. He is to be commended for good organization of material, impressive gathering of relevant data, and clear writing. However, his scientific logic has some weak points. See the following quote:
QUOTE:
“Between 1900 and 2000, atmospheric CO2 increased from 295 to 365 ppm, while temperatures increased about 0.57 degrees C (using the value cited by Al Gore and others). It is simple to calculate the proportionality constant (call it ‘k’) between the observed increase in CO2 and the observed temperature increase:
ln (365/295) = k * 0.57
k = 0.3735
ln (2) = 0.693 = k*deltaT
deltaT = 1.85
This shows that doubling CO2 over its current values should increase the earth’s temperature by about 1.85 degrees C. Doubling it again would raise the temperature another 1.85 degrees C. Since these numbers are based on actual measurements, not models, they include the effects of amplification, if we make the reasonable assumption that the same amplification mechanisms that occurred previously will also occur in a world that is two degrees warmer.”
END QUOTE
Several problems: First he is assuming that 0.57 is the actual increase — but UHI is a concern, not to speak of data manipulation.
Second, he is assuming that all increase is due to CO2, but there could be other causes for temperatures to rise in the 20th century such as land use, solar variation, recovery from the LIA, . . . .
Third, he would get a different result if he based his calculation on the last 20 years rather than the 100 years in the 20th century. Or the results would be even more scary and dramatic if he took 1900 to 1940 to establish his k.

Okay, this “global cooling” discussion is getting way out of hand. Obviously, I’ll refer everyone to RealClimate’s “Global Cooling” article, but another comment is important:
What the hell does it matter if, in the infancy days of climate science, climatologists mistakingly believed that there was going to be an imminent period of cooling? Seriously, what does it matter? I was cleaning out my basement the other day and I came across a PC World article from the month when the very first 1.0 GHz machines went on the market; in that article, the writers suggested that it wouldn’t be long before we hit the 10 GHz benchmark. Obviously, they were wrong, as we are now fully aware that heat issues prevent clocking of single core CPU’s that high, which is why we have moved to a multi-core paradigm in computing.
Should the PC World writers from that month forever hold the stigma that they made an erroneous prediction based on a more primitive knowledge base than we have now? Of course not. Yet you’re doing the same thing.
Referring to the Global Cooling mole is an ad hominem meant to attack the scientists involved in AGW research, and does nothing to refute or further any argument against the AGW theory and its predictions.

The “pack of lies” which is your opinion is contradicted by at least 7 papers in the 70’s that you cite above. So are you going to claim those papers were lies now?
No, they were a small minority. You seem to be assuming that if a single scientist advocated global cooling, then the idea had currency. There are always weirdo ideas floating around the scientific community, largely because (despite the claims of those with a political agenda who are unaware of the truth) scientists are extremely open-minded about scientific issues. If you can make a good case for something, you’ll get it published. And if you can make a compelling case for something that everybody else rejects, then you get the Nobel Prize.
Remember, there were also 42 papers — six times as many as the global cooling papers — promulgating global warming. If you want to base a case on these papers, then you end up concluding that scientists were 6:1 against global cooling. Therefore, stories that scientists believed in global cooling are a crock.
The tendency that you have to use terms on this forum such as “deniers” and “pack of lies” suggests that you are more about passion and less about facts.
Perhaps you did not read my post on this subject, but I used the term “denier” on my first post and as soon as somebody objected to it, I promised not to use the term again — and I have not used it. Meanwhile, many posters here continue to use derogatory terms towards those they disagree with. Are your standards of civility dependent upon the editorial claims of the writer?
I think “pack of lies” is a correct characterization of a story that is false and yet is often repeated despite proffered evidence that it is false. When people make an honest mistake, they stop doing it once they have been corrected. However, the skeptics keep repeating the global cooling falsehood over and over. I think we’re not talking about an honest mistake here, but rather willful disregard for the facts.

Oops, I didn’t see this earlier:
REPLY: just because NAS issued a report in 1975 doesn’t mean anybody paid attention to it or dropped all the research about it. Global cooling research went on at many universities into the early 1980’s, NOVA did a show about it in 1979.
You seriously believe that nobody paid attention to an NAS report? Wow… all I can say is… careful… well, I suppose that there were skeptics even then who refused to pay attention to the NAS. But the great majority of scientists hold the NAS in very high esteem. To be invited to be a member of the NAS is a major career plum. So YOU may not have paid attention to the NAS report, but just about everybody else who had any interest in the topic certainly did. Jeez, even *I* remember reading a story about that report.
As to “global cooling research”, I think that you’re conflating climatology research with global cooling research. Many of the factors that were cited in the original Science paper, such as aerosols, were legitimate subjects of scientific research. Indeed, they are STILL be studied as part of global warming research. The IPCC AR4 report includes 25 pages on aerosols. Are you suggesting that the IPCC believes in global cooling because it is reporting on research on a topic raised in conjunction with global cooling?
REPLY: No I’m suggesting that I had firsthand knowledge of scientists at universities in the USA that were still pursuing the idea of global cooling as late as about 1981.
In fact I know of one university researcher who dropped what he was doing in synoptic and mesoscale meteorology after the hard winters in the USA in 1977/78 and stared focusing on global cooling issues.
The 1975 NAS report is like the IPCC report is this way; some paid attention, some took it with a grain of salt, and other just flat ignore it as “science by consensus”.
A good example of science by group consensus that turned out to be flat wrong was the consensus of the solar science panel convened by NASA lat year that claimed we’d be seeing the activity of solar cycle 24 by now.
From this story on space.com where they talk about the opposing views solar scientists have for cycle 24 they offer some opinions. NOAA Space Environment Center scientist Douglas Biesecker, who chaired the panel, said in a statement:

[…] despite the panel’s division on the Sun cycle’s intensity, all members have a high confidence that the season will begin in March 2008

Well, obviously March 2008 didn’t happen. We are still waiting. So much for that group consensus. Data, calculations, models, expert panels, and even SWAG wasn’t able to predict natures whims.
Nature rules, nature confounds, nature outwits.