Joe D’Aleo and Don Easterbrook have produced a new paper for SPPI. This graph of US Mean temperature versus the AMO and PDO ocean cycles is prominently featured:
I particularly liked the regression forecast fit:
They have this caveat:
Note this data plot started in 1905 because the PDO was only available from 1900. The divergence 2000 and after was either (1) greenhouse warming finally kicking in or (2) an issue with the new USHCN version 2 data.
Hmm. I’m betting USHCNv2.
Abstract:
Perlwitz etal (2009) used computer model suites to contend that the 2008 North American cooling was naturally induced as a result of the continent’s sensitivity to widespread cooling of the tropical (La Nina) and northeastern Pacific sea surface temperatures.
But they concluded from their models that warming is likely to resume in coming years and that climate is unlikely to embark upon a prolonged period of cooling. We here show how their models fail to recognize the multidecadal behavior of sea surface temperatures in the Pacific Basin, which determines the frequency of El Ninos and La Ninas and suggests that the cooling will likely continue for several decades. We show how this will be reinforced with multidecadal shift in the Atlantic.
Here’s the paper you can download:
UPDATE: The goodness of fit, seems almost too good. There may be a reason. I’m reminded in comments of this article by statistician William Briggs – (thanks Mosh)
Do not smooth times series, you hockey puck!
Where he points out:
Now I’m going to tell you the great truth of time series analysis. Ready? Unless the data is measured with error, you never, ever, for no reason, under no threat, SMOOTH the series! And if for some bizarre reason you do smooth it, you absolutely on pain of death do NOT use the smoothed series as input for other analyses! If the data is measured with error, you might attempt to model it (which means smooth it) in an attempt to estimate the measurement error, but even in these rare cases you have to have an outside (the learned word is “exogenous”) estimate of that error, that is, one not based on your current data.
If, in a moment of insanity, you do smooth time series data and you do use it as input to other analyses, you dramatically increase the probability of fooling yourself! This is because smoothing induces spurious signals—signals that look real to other analytical methods. No matter what you will be too certain of your final results! Mann et al. first dramatically smoothed their series, then analyzed them separately. Regardless of whether their thesis is true—whether there really is a dramatic increase in temperature lately—it is guaranteed that they are now too certain of their conclusion.
Perhaps Mr. Briggs can have a look and expound in comments. I only have the output, not the method. But let’s find out and determine how good the “fit” truly is. – Anthony
UPDATE: Statistician Matt Briggs responds in depth here. He says:
I want to stress that if D&E did not smooth their data, the correlation would not have been as high; but as high as it would have been, it would still have been expected. All that smoothing has done here is artificially inflated the confidence D&E have in their results. It does not change the fact that AMO + PDO is well correlated with air temperature.
Unfortunately, the PDO and AMO are not similar datasets and cannot be added or averaged. The AMO is created by detrending North Atlantic SST anomalies, while the PDO is the product of a principal component analysis North Pacific SST anomalies, north of 20N. Basically, the PDO represents the pattern of the North Pacific SST anomalies that are similar to those created by El Niño and La Niña events.
In response to this and others, the weighted influence of the oscillations on the regional climate data can certainly be applied either sequentially or as an average as long as the weighting is consistent with the degree of influence.
Terrible science. You should never smooth before calculating R2 or any other key metric. The chances of creating a spurious correlation between unrelated data sets is dramatically increased if you do.
Did Mann produce that graph?
Very interesting, not so much the paper as such but in the “sceptics” publicly discussing and showing possible errors in a paper which apparently proves their case. This appears to be a more valid scientific discussion than that shown by the calamatists. Perhaps the amateurs have something to show the professionals, much like the amateur physicist A. Einstein (a clerk who only held a teaching diploma in mathematics). He should not have had anything of interest to say but only changed the basis of physics. One wonders how he would have been received if he had needed to be “peer reviewed” before publication.
I am not a fan of smoothing so I don’t use it unless the underlying data series is just too noisy to be useful (and then I only smooth it the least amount possible).
US temperatures, however, are very noisy. It is so noisy that one can’t do anything with it.
Here is the US monthly temperature anomaly (in degrees F).
http://img148.imageshack.us/img148/6950/usmonthlyanomf.png
Putting in place a 12 month moving average now results in something useful and probably provides a pretty good representation of the US Climate (now in degrees C). Note the US has not warmed much over the last 100 years and there has been a substantial downturn in the last 4 years.
http://img844.imageshack.us/img844/3128/ustempscaugust10.png
If one uses the unsmoothed AMO and PDO figures, we see the AMO and PDO do a pretty good job of matching the general cycles of the US Climate, but …
– the underlying temperature series (12 month moving average) still varies but quite a bit more than can be explained by the ocean cycles;
– and the PDO is not really significant in influencing US temperatures.
http://img215.imageshack.us/img215/2496/usmonthlymodelpdoamo.png
Basically, I think we need a new measure for what the PDO is supposed to reflect – the influence of the north Pacific on climate. Maybe a longer smoothed series is better but the current PDO index doesn’t actually match up with any temperature series.
Ric Werme says: “Oh cool. From my Guide to WUWT (see button in top right), my first ‘WUWT Classic’ is 2008 Jan 28: Warming Trend: PDO And Solar Correlate Better Than CO2…”
The TSI data used in that post by Joe D’Aleo…
http://wattsupwiththat.com/2008/01/25/warming-trend-pdo-and-solar-correlate-better-than-co2/
…is the Hoyt and Schatten data, and it’s obsolete. If memory serves me well, Hoyt and Schatten created that dataset to explain the rise in global temperature from 1900 to 1940, so they should correlate.
“The divergence 2000 and after was either (1) greenhouse warming finally kicking in or (2) an issue with the new USHCN version 2 data.”
Because as scientists we all know that if our predicted results don’t match up with the observed data, there must be something wrong with the observed data.
This isn’t my area of expertise, but is that seriously what they’re saying?
Maybe a longer smoothed series is better but the current PDO index doesn’t actually match up with any temperature series.
There’s more than one influence, Bill!
By the way there’s nothing wrong with “smoothing” as long as the variations about the smoothed graph can be shown to be statistically “random.”
Thanks to Joseph and Don for this and now back to work for me
OK. Let us suppose that what is called the PDO is, as has been suggested by many and is supported by careful analysis, a long-term integration of ENSO tropical anomalies. All well and good.
It still remains the case, though, that we have a sixty-year cycle of dominant El Niño / sardine events alternating with dominant La Niña / anchovy events, which is reflected in differing SST patterns in the North Pacific, and which remains unexplained. This is the point that was being made, I believe, by a number of commenters on various PDO articles.
Thus the PDO still remains to be explained, whether it is “independent” of ENSO or not.
Anthony,
I have responded in depth over at my place:
http://wmbriggs.com/blog/?p=2952
REPLY: Thanks Matt, for taking up the call. – Anthony
I COMPLETELY agree about not smoothing the data. It is the real data that enlightens us about climate, not the smoothed data. At the most, I would agree that a three month running average, for each geographic biome, can and should be compared to three month running averages of strategic geographic SST’s (ENSO style). Atmospheric pressure systems and major trade winds should also be so compared. I repeat, no more than a 3-month running average should adopted as the standard for all atmospheric/oceanic known systems.
The Australian BOM update showing the current La Nina is growing stronger.
A La Niña remains well-established in the Pacific. Given the current strength of the event and the outlook from long-range models surveyed by the Bureau, this La Niña is expected to persist into at least early 2011.
All indicators remain firmly at La Niña levels. The central Pacific Ocean is cooler than the long-term mean both at and below the surface, the Southern Oscillation Index (SOI) remains strongly positive, trade winds are stronger than normal and cloudiness over the central tropical Pacific continues to be suppressed. Such consistent signals indicate the tropical atmosphere and ocean are now clearly reinforcing each other.
The SOI graph is looking very strong.
This will be bigger than 2008.
Bob Tisdale says:
October 1, 2010 at 6:13 am
The TSI data used in that post by Joe D’Aleo…
…is the Hoyt and Schatten data, and it’s obsolete.
Plus he calculated an R^2 on heavily smoothed data, so the whole analysis is worthless.
There are no current solar threads active….so I will ask the question here.
Did anyone notice the huge jump in F10.7 flux when negative sunspot 1108 left the face 2 days ago?
“smoothing induces spurious signals—signals that look real to other analytical methods”
Thanks for the reminder. Ought to be worn as a phylactery by those flogging stats.
Geoff Sharp says:
October 1, 2010 at 7:06 am
Did anyone notice the huge jump in F10.7 flux when negative sunspot 1108 left the face 2 days ago?
Huge and huge? Small jump. Huge F10.7 is 300.
Sunspot 1108 was not ‘negative’ or abnormal. A ring of opposite polarity often forms about lone spots. The increase in F10.7 is not related to the disappearance of something that would raise F10.7. The Sun is messy. Don’t over-interpret the data.
Leif Svalgaard says:
October 1, 2010 at 7:37 am
Maybe with all your wisdom you can tell us why the F10.7 record jumped from roughly 80 to 90 while in the background the the sunspot area is diminishing with no flare activity?
Funny!…now, what did you say temperature was? 🙂
Yes indeed. And here is a lagging indicator:
http://articles.latimes.com/2010/sep/30/food/la-fo-wine-harvest-20100930
This is definitely more in the climate category than the weather category.
This has been kicked around for some time:
http://www.ukweatherworld.co.uk/forum/forums/thread-view.asp?tid=17838&start=1
The point of which is there is a natural signal and an anthro signal in the global data. Neither signal is known to be anything but cyclical, and the science is far from settled regarding positive or negative feedback. We know from the historic record that climate swings from one condition to another, and that no condition has been permanent, and that no swing was ever unidirectional.
Regarding the feedback loop, we don’t now the system gain, the period (20 years, 100,000 years?), if it is critically damped, or undamped (of so expect wide variations). And we don’t know if there are tripping points in the loop – reachable limits that will destabilize the loop forever without other influences.
Most people have never had the opportunity to watch an Anschüetz gyrocompass start up cold. This is a heavy sphere about the size of a bowling ball and which has two internal gyros fitted. It floats top side up in a temperature controlled electrically conductive fluid inside a spherical chamber. There is no mechanical connection between the sphere and the container it rides in. It was designed by Hermann Anschütz whose cousin was Maximillian Schuler. Schuler’s name is affixed to the curve generated by the gyrocompass as it stabilizes on startup. See more at Schuler tuning for his work in pendulums and gyros. Until Schuler discovered the natural periodicity of the system, practical gyrocompasses were impossible. I think there are natural periodicities to be discovered.
And when you watch one spin up, you’d swear the wide swings will never stop. They do – Schuler guarantees it.
Proclaimeth the great Svalgaard:
the whole analysis is worthless.
Well then, Ladies and Gentlemen, I guess that settles it, doesn’t it.
Smoothing is the wrong way to go. If y is temp and x is PDO, then a regression of y on x is y = a + bx. If you do, say, a 5-year moving average of PDO to smooth it, then you are running y = a + b(x(t) + x(t-1) +x(t-2) + x(t-3) +x(t-4))/5. This is equivalent to
y = a + .2bx(t) + .2bx(t-1) + .2bx(t-2) + .2bx(t-3) + .2bx(t-4). The fit is better because you are regressing on current PDO and past PDOs, but not in a correct way.
There may be a cumulative effect of PDO on temp, so maybe the model should have past PDOs included in it. The correct way to do this would be
y = a + bx + cx(t-1) + dx(t-2) etc. Then the regression would separately estimate the effect of each past year’s PDO value on temp. The fit of the regression would be better because you are allowing the regression to choose the coefficients b,c ,d etc. instead of forcing them. It would also tell you which years have the largest effect on temp.
@ur momisugly Pamela Gray, Mattstat and others.
I agree that computing r2 comparing smoothed signals without allowing for the reduction in degrees of freedom imposed by additional serial correlation is plain wrong.
However, if the hyopthesis is that the temperature is related to the AMO and PDO as a liner system, simple correlation is not the best way to look at the problem because it obscures the temporal relationship between the two signals, i:e does temperature follow the oscillations or lead it? This is, from a point of view of mechanisms, rather important and the lead/lag relationship in the graph shown is rather difficult to interpret because the PDO+AMO leads temperature when temperature is increasing, but lags it when temperature is decreasing. I would guess that this is because the averaging (smoothing) process is a filter which has eliminated the high frequency components in both signals that determine their relative phases – a well known problem. I would therefore go along with the commentators who have said that one should work with the raw signal and performed a more formal correlation that involves a consideration of phase shifts and extracted the degrees of freed om of the signals from their serial correlations.
jim hogg says:
October 1, 2010 at 5:31 am
That ‘trick’ has been used before.
1.) set up the ascribed proof argument
2.) adjust the data to support the prescribed outcome
3.) Make sure the proof window is short, to accomodate the natural fluctuations of climate about the mean.
4.) Cherry-pick the time window
5.) As soon as the fluctuation hits the target, run away to the next agenda item but continue to cite the ‘frozen in time’ proof.
6. Never update a “proof point” that has turned sour.
Leif Svalgaard says:
October 1, 2010 at 7:37 am
Leif: Geoff is talking about a certain ‘type’ of sunspot where, when it round the limb to the backside, the flux rises abruptly. The converse is true when this type of spot first appears. It’s a current behavior of the Sun that we have been watching.
Enjoy.
Well since I’m not a climatologist, so I don’t exactly understand who or what AMO and PDO are; but I accept that those in the trade probably do; so I am just looking at graphs of data.
Well actually I guess I’m really looking at graphs of what is left over when the original data was thrown away with “smoothing”.
I’m not a fan of throwing away real data that was presumably obtaiend with difficulty and at great expense; and then replacing it with faux data.
If you complete the smo0othing process, you end up with just a single number for the whole data set (each one you are trying to compare. So you will eaither get the same number from each set; or you will get a difefrent number so you can comment on that difference (or sameness.
But I get the gist of what they are trying to do.
For those who say the fit is “almost too good”; well I’m seeing enough differences that I wouldn’t describe the fit as too good.
But certainly enough sameness to make one wonder if they are somehow related.
The second curve; the AMO/PDO regression fit does not to me look like the best fit of these two curves. My eye says that these two curves were force fit to a perfect match at the start; and then diverge (increasingly) thereafter.
If I assume that these two cuves are extracti0ons from an even longer data set; that the authors do not have; but fear not Mother gaia has all that data; then I would not just accept that they should exactly match at the starting point where the authors come into the picture and start logging data.
so my suggestion is; Can you get an even better match of the same raw data; if you let the two data sets “float” so that you don’t force a good fit at the start at the expense of a bigger divergence at the end of the frame.
I notice that the Temp is just for the USA; which conveniently sits almost exactly positioned between the Atlantic and teh Pacific; how likely is that ? Well I’m just shaking your cage here. You’re suggesting that the combination of whatever PDO does and AMO does, is pretty much sufficient to describe to some fairly respectable degree what will happen over the USA. Izzat the first time any such relationship has been hinted at ?
Rather interesting; but I still don’t like throwing away data and trying to convince one’s self that you are gaining information rather than losing information.
Reminds me of the chap that predicted the value of the fine structure constant to better than parts in 10^8; by completely throwing away all information about the entire physical universe; and making the number up out of whole cloth, mathematically.
Watch out; it could happen to you.