Guest Post by Willis Eschenbach
In a recent post here on WattsUpWithThat, the claim was made that the El Nino influences rainfall. They showed a correlation between various historical proxies and El Nino/La Nina. So I thought I’d take a look at the modern correlation between rainfall and the El Nino. As a measurement of the El Niño, I’m using the Oceanic Nino Index (ONi).
So, what is the ONI when it’s at home? NOAA says:
Oceanic Nino Index
The Oceanic Nino Index (ONI) is one of the primary indices used to monitor the El Nino-Southern Oscillation (ENSO). The ONI is calculated by averaging sea surface temperature anomalies in an area of the east-central equatorial Pacific Ocean, which is called the Nino-3.4 region (5S to 5N; 170W to 120W). Also, a 3-month time average (running mean) is calculated in order to better isolate variability closely related to the ENSO phenomenon.
OK … although I don’t like the 3-month boxcar filter (running mean). It is known to do things like reverse the peaks and valleys of a dataset, and is a horrible choice. So I’ve used the un-averaged version of the ONI
Now, I’ve written about the relationship between temperature and rainfall before, in “Cooling and Warming, Clouds and Thunderstorms” and in “How Thunderstorms Beat The Heat” The TLDR version is that over the ocean, where most of the rain falls, the rainfall amounts go up with increasing temperature. The evaporation of the water to make rain is one of the major mechanisms that keeps the ocean from overheating.
But I digress, I’m here to discuss the El Niño and the Oceanic Nino Index. The paper made a curious claim, that La Nina conditions were wetter, and El Nino conditions were dryer. Here’s the graphic from their paper:
Figure 1, with Original Caption
Now I was born yesterday, but having lived in the South Pacific I do know that there is no general rainfall rule for El Nino and La Nina. Some places like Southern California get wetter in an El Nino year, and some places like Australia get drier. So I found it odd that they identified “wetter” with La Nina and “drier” with El Nino. I thought I’d look at the correlation between the amount of rainfall and the Oceanic Nino Index (ONI). Figure 2 shows that result. A positive correlation (yellow to red) means that a high ONI index (El Nino condition) is accompanied by increased rain. A negative correlation (green and blue), on the other hand, means the opposite—there is less rain during El Nino conditions. The correlation (both positive and negative) with rainfall is at a maximum three months after the change in the ONI.
Figure 2. Correlation of the monthly Oceanic Nino Index (ONI) with the amount of monthly rainfall, 2000-2015. Red box shows the area which is represented by the ONI. Red/white circles show the locations of the proxies shown in Figure 1.
This shows what I started out by saying, which was that an El Nino increases the rain in Southern California and decreases the rain in Australia. In other words, we cannot say “Wet/La Niña-like” or “Dry/El Niño-like” as the authors do.
It also shows the idiosyncratic and convoluted nature of the area of positive and negative correlation. For example, the ONI is generally positive correlated with the northern hemisphere rainfall, and negatively correlated with southern hemisphere rainfall.
Now, the authors of the study say:
The composite record shows pronounced shifts in monsoon rainfall that are antiphased with precipitation records for East Asia and the central-eastern equatorial Pacific. These meridional and zonal patterns are best explained by a poleward expansion of the Australasian Intertropical Convergence Zone and weakening of the Pacific Walker circulation (PWC) between ~1000 and 1500 CE.
I see no need to invoke any such special mechanisms to explain rainfall shifts that are “antiphased” to rainfall shifts in other areas. From an examination of Figure 2 above, such antiphasing is the rule rather than the exception. In La Niña times California gets drier, Australia gets wetter, and the world goes on.
Regards to all on a sunny evening,
w.
My Usual Request: Misunderstandings can be avoided. If you disagree with me or anyone, please quote the exact words you disagree with, so we can all understand the exact nature of your objections. I can defend my own words. I cannot defend someone else’s interpretation of some unidentified words of mine.
My Other Request: If you believe that e.g. I’m using the wrong method or the wrong dataset, please educate me and others by demonstrating the proper use of the right method or the right dataset. Simply claiming I’m wrong about methods doesn’t advance the discussion unless you can point us to the right way to do it.
Her in the far northern portion of California, the El Niños do correlate with wetter times as a rule; however, sometimes southern California can experience an El Niño one fall/winter/spring ahead season of us.
I have noticed that pattern for the last thirty-six years.
So, to break it down for an idiot like me, is there more rain globally, during El Ninio or La Niña conditions?
Its climate Willis, you have to turn some of the graphs upside down to get the right result… 🙂
Thanks Willis for pointing out that a running mean is horrible in time series analyses. What has fooled so many people is that a running mean can induce periodic behavior where there is none, it is called the Slutsky-Yule effect. Even a series of random numbers can have induced periodic behavior when a running mean is used to smooth the data.
Yes, there is some growing awareness of how bad running averages are. Willis is spot on with this. There is no need to ‘smooth’ the data before doing this analysis.
Running means have frequency responses with sizable sidebands of both polarities. While they may, in pathologically extreme cases, reverse the polarity of significant high-frequency oscillations, that is almost never the case with geophysical data, whose spectral content is generally concentrated well below the cut-off frequency of the boxcar filter. And no matter what running mean is used, it is mathematically incapable of generating a strictly periodic signal from random noise. The Slutsky effect refers to IRREGULARLY oscillating data emerging from a low-passed sequence of random numbers. Let’s not get caught up in the weeds of overblown theoretical caveats.
The first and strongest negative lobe is at about 1.4 times the base frequency. That’s about 8.5mo for a 12mo moving average. Hardly “high frequency” , nor improbable “extreme” case as you state.
The effect is real and pervasive in just about any data you may be looking at including geophysical data.
Look at how a 5y running mean manages to turn the 1998 El Nino into a slight trough !
https://climategrog.wordpress.com/2013/05/19/triple-running-mean-filters/
Hardly an “overblown theoretical caveat”. I suggest you read the above article and inform yourself about the nature of the problem.
Greg:
There’s nothing new or unexpected in your contrived example. The frequency response of moving averages has been known in geophysics for at least half a century. Given the spectral structure of the RSS global average temperature, only blind “climate scientists” would misapply MAs as badly as you indicate. In any event, there’s precious little spectral content at 8.5 months in the monthly series and the MA at issue here is the seasonal average of 3 months. Stop riding a hobby horse in field where you’re the patent novice.
Willis,
“This shows what I started out by saying, which was that an El Nino increases the rain in Southern California and decreases the rain in Australia. In other words, we cannot say “Wet/La Niña-like” or “Dry/El Niño-like” as the authors do.”
The authors are well aware of those variations. Here is their Fig 1 showing correlation of temperature with Nino3.4:
The article is about W Pacific monsoons etc, and I think that annotation on the fig you show is to be interpreted relative to that.
Ah, so they weren’t talking about the whole globe, which renders my other question irrelevant to what the paper is actually discussing.
Nick, you say the authors are well aware but then provide a graph showing temperature correlations not temp-rain correlation. If the paper is about W Pacific only, why are half their proxy sites in northern South America?
Greg,
I used the wrong word in describing the plot – it is rain-temp correlation, as the caption says. And it is about the W Pacific – the title is:
“Western Pacific hydroclimate linked to global climate variability over the past two millennia”
Oops – Fig 1 shows correlation of rainfall with Nino3.4 SST.
I thought it was an Australian paper e.g. Wet for them is dry for us.
As shown in Figure 2, the east coast of Kenya is also wetter as I found out first hand in 1997 when working there in the “El Nino Floods”. The flood damage to roads between Malindi and the Tanzanian border was very extensive.
Good Essay. Thank you again.
Is there more rainfall overall in the regions of negative correlation, so that the overall correlation between the rainfall and La Nino index is negative?
Willis, this N/S difference is quite striking. I have often read that ENSO affects global temperature with a 3mo lag but this graph shows why attempting to remove “ENSO” variability by subtracting a globally correlated weighting is so unsuccessful. For a large part of the globe it will ‘removing’ by adding to the actual ENSO effects.
Typical climatology nonsense of trying to reduce the whole planet’s climate to single number indices.
This graph looks a lot like an indication of Pacific wide oscillation.
This makes me ask whether the max correlation for the SH is also at 3mo lag. Where does the max. POSITIVE correlation for SH lie. Maybe it even leads Nino3.4
Yes, the graph shows NH rainfall correlates strongly positive, while SH is a negative association. WUWT? Since the SH is mostly ocean, it may be getting a lot more rain in absolute terms.
An El Nino is rainfall. That is how it was originally defined in the 1400’s by the Spanish Fleet. if you want to rename it to what your model is doing with the ocean, give it another name. We can model all El Ninos on the West Coast by looking at basin discharge data. That is the social relevance. This shows almost no ENSO to rainfall correlation. No problem. Redefine El Nino. Whoa, what a match.
We can study historical data to `1930 in Southern California by studying basin discharge data. No modelling used.
I was under the impression that this name was given by spanish speaking fishermen in Peru much more recently. What is your source for this claim that it was the spanish fleet?
Sounds very improbably because to link this to Christmas would require many years ( decades ) observation of local climate.
I doubt that this has anything to do with the Spanish except that they imposed their language on the whole continent of S. Am.
Mirriam Webster lists the first known use of El Nino to describe a weather pattern as being in 1896.
Do you mean 1500s or 1600s? Surely not the 1400s. The Spanish weren’t in the Pacific until the 1500s. ENSO was supposedly first noticed by fishermen from the Viceroyalty of Peru in the 1600s, as the appearance of warm water around Christmas time.
Unless you are thinking of the rain in Spain, which doesn’t always fall in December.
My original comment on the 1400s v. the 1500s or 1600s appears to have gone missing.
Correct me if wrong, but wasn’t the arrival of warm water around Christmas time off the tropical South American coast first noticed by fishermen from the Viceroyalty of Peru in the 17th century?
“This graph looks a lot like an indication of Pacific wide oscillation. ”
It is known to be and east/west oscillation. My point there was that it seems that the whole SH is showing opposite correlation. My guess is that SH is leading ENSO.
Is this negative correlation for SH actually largest at 3mo lag ?
Willis, You wrote “This shows what I started out by saying, which was that an El Nino increases the rain in Southern California and decreases the rain in Australia. In other words, we cannot say “Wet/La Niña-like” or “Dry/El Niño-like” as the authors do.”
This reads (to me at least) as if you are inadvertently, perhaps, attributing a causal relationship between the sets of observations.
Did you intend to do this?
Another aspect of ENSO is sea level pressure difference between Tahiti and Darwin. It seems very likely that there will be correlation between SLP and rainfall. What is your objection?
“El Niño conditions in the tropical Pacific are known to shift rainfall patterns in many different parts of the world. Although they vary somewhat from one El Niño to the next, the strongest shifts remain fairly consistent in the regions and seasons shown on the map below”
http://iri.columbia.edu/wp-content/uploads/2016/05/ElNino_Rainfall.pdf
Something must be seriously wrong with the map. El Nino emphatically does NOT cause decreased precipitation in the Galapagos.
could be the 3mo lag ! That is why I was asking whether this lag also applied to SH. #
When the data shows such a clear N/S split, a one size fits all lag is very unlikely to be appropriate. What is needed is lag correlation for at least 6mo either side of zero for SH only. Actually it would be best to look 18mo either side.
Perhaps you could carry out the necessary calculations and post the result here.
It is not clear which map you are referring, the map I linked to (see just above) shows that Galapagos is ‘Wet’ from January to May
I know, I meant the top map. Possibly the three month lag causes this very odd result, and the map really shows what happens after a nino.
Yes, I suspect this is the case. It may well be that SH peaks at or even before the Nino3.4 region shows its warming peak. Willis’ correlation plot is rather interesting but rather raises the question of what it would look like with other lags. Just because 3mo is what shows the largest correlation on a global scale does not mean that this is the lag with greatest correlation for all regions.
I would at least like to know what lag gives the best correlation in SH.
I suggested this earlier but a lot of my comments seem to be getting lost today.
ISTR that part of the El Ninyo and La Ninya effect at least was due to the wind strengths off the Pacific coast of South America (if you cannot put the tilde on the second ‘n’ suggest you add, as I have done, the ‘y’ to represent the proper pronunciation). When winds tended to blow east, warm water was driven east, and this resulted in humid air and high precipitation in SA (La Ninya). And when the winds blew westwards this resulted in upwelling of cold water, and dry air on the coast (El Ninyo).
But it appears that the wind directions for El Ninyo and La Ninya are reversed for the East Coast of Australia, so that we get the more easterly tending winds with an El Ninyo and dry weather. And when we get westerly tending winds we get the La Ninya and rain. (See your map showing the same correlation for both Eastern Australia and West Coast of SA.)
Query – if there is a three month lag, is this not the same as a three month lead when you are looking at the six monthly change in winds as the sun moves north and south?
Though this last week we had a beaut “East Coast Low” which dumped plenty of rain from Southern Queensland right down to Tasmania over the course of several days – some places about 300 mm over 36 hours, plus 160 km/h winds. One of these (‘weather’, not ‘climate’) certainly stuffs up any correlations between anything unless you have the local knowledge to detect it and take it out of the data. Though I suppose some would call this “Cherry Picking!
No, I’d call it falsifying the data. If it happened it is part of the data. Why one earth would you need to ‘take it out’?
If it screws up your correlation there probably was no correlation.
‘Removing disagreeable’ data is a standard scientific technique in climate studies……
years of experience tells me that when the enso hits neutral or the minus on wuwt pages I can be happy to know decent or above avg rains are on the way in Aus.
when we copped the floods a couple of yrs ago it damn near hit -2.
and since its moved even to near midneutral we ARE getting the rains starting albeit late for the season.
bloody marvellous
it can rain all it wants:-)
Atmospheric water vapor goes up with El Nino’s, so I would think less rain when the El Nino is increasing in strength, but when it declines and cools that water vapor condenses hence more rain.
That would explain your observations
Your figure 2 shows India wetter with El Nino?
The Southern Oscillation was defined by the 1st head of India Meteorological Department and used in the long-range rainfall forecast. The 126 years [180-2006] Indian southwest monsoon rainfall amounts versus Southern Oscillation part are gien as:
SO 1 2 3 4 5 6
El Nino 07 05 05 00 01 18
Neutral 14 13 37 14 16 84
La Nina 00 00 07 07 10 24
Total 21 18 49 21 17 126
SO = Southern Oscellation Part; 1 = Deficit, 2 = below normal, 3 = normal
4 – above normal, 5 = excess and 6 = total
This was one of the parameter [out of 16 parameters] used in the long-range rainfall forecast. Mosst of the predictions come within the normal condition. The forecasts never come close the real when they are in deficit or excess groups.
Dr. S. Jeevananda Reddy
Dr Reddy,
How are this year’s Indian monsoon’s? what are the forecasters projecting for Indian monsoon compared to non El Nino years?
joelobryan — That is exactly what I presented above. During El Nino 18 years, deficit in 7 years, below normal in 5 years and normal in 5 years and above normal in one year. The same in the case of Lanina with 24 years, in 7 years the precipitation was normal, in 7 years, it was above normal and in 10 years it was excess.
I neutral years, they varied between deficit to excess. However, in the 60 year cycle, in the above the average part of 30-years on more years the precipitation was in excess over the deficit years and vice versa is the case with the below the average 30 year period. In the southeaster parts it is quite different as this region receives rainfall in pre-monsoon, monsoon & post-monsoon seasons. In 132 year annual rainfall cycle, during the above the average 66 years period, excess in 24 years and deficit in 12 years was recoded. Vice versa was the case with the below the average 66-year period. Now, this part is in the below the average part since. In the case of 60 year cycle [in the southwest monsoon precipitation at all India level] the below the average 30 year period starts from 2017.
All forecasts were within the plus or minus the average for the country. None of them predicted the deficit or excess precipitation condition. In this the statistical regression is carried out on 16 parameters and the best correlated are used in the analysis.
This year SKYMET [a private forecaster] and IMD forecasted above normal. Though the monsoon was predicted on normal date, it was delayed by 8 days. It moved northward and stopped there. However, a cyclonic system looming above this zone.
Dr. S. Jeevananda Reddy
correction — plus or minus 10% of average
sjreddy
I think the issue withis study is what the data really is and what they have done with it.
There is dO18 isotopes, dueterium proxies, some LLpc1 index and some varve density and color lake cores. Are these rainfall proxies? dO18 and deuterium are primarily temperature proxies.
And there the base data is shown in the paper in various ways but appears to be highly random while the main chart shown appears to be some smoothing function that now shows the data in a different way.
I thought dO18 was a precipitation proxy, that had been used by some Climastrologists as a temp proxy.
It is a precipitation-based proxy but temperature is what drives the variation. Technically, it is the best temperature proxy that there is and this goes back hundreds of millions of years. It just seems to always match what we know about the climate in history on all time scales even including decade-specific. But there are region-specific formula to use rather than global averages and there is a drift over time so that it must be detrended as one goes farther back in time etc. One has to work with the data to see how important these issues are. But it really works compared to tree rings or even deuterium. It is the best temperature proxy that there is.
Bill. I cant help thinking that there is some circular reasoning being indulged in. Sorry
Thanks Willis. Something seemed off about that paper’s trying to tie cold with wet with La Nina, and the inverse for El Nino. From a climate stand point, cold seems to be dry, and warm, wet. As you point out it is regional.
Thanks, Willis.
Has anyone studied the cause of the drought in Venezuela which it’s President blames on the US, causing global warming. They are highly dependent on Hydro for electricity which is also rationed due to the “drought”.
Venezuela is about to collapse with the left leader’s policies, and people are dumpster diving for food.
https://www.yahoo.com/news/anti-maduro-recall-halted-violence-erupts-venezuela-officials-163224076.html?ref=gs
The left government failure is not reported much in the MSM, could educate voters of socialism failures.
Interesting map. Maybe instructive to try other nino zones individually and the whole lot averaged together. Methinks our nino definitions are too arbitrary.
I am very much aligned with your rainfall and cloud cover cooling ideas, to me it makes such very good sense. So surely how dry/wet California is versus Australia is not particularly relevant as this is generally due to the state of the trade winds in any one year.
So surely it would be most interesting vis a vis this argument what the total rainfall in the El Nino areas themselves is rather than the periphery land masses.
All statistics. No physics. No models. A good illustration of a “Climate Change”.
Willis writes
Never underestimate an individual’s capacity to be self centered. Many people on forums think that Winter see’s a decrease in insolation too.
Here’s the graphic from their paper:
====
Honest question….
On their graphic, they have boxed in the LIA…..
…but according to their graphic, the LIA does not end a little past 1800
Their own graphic shows the LIA ending around 1950
First they claim when the LIA ended…
…and then blame all of the recovery after that on man’s fault
Would there be different/more noticeable conditions such as rain/dry during the change over between the two since that (perhaps) is when the cold/warm air masses react together more often?
In looking back at the effects of El Nino since 1946 I was surprised that I could not find a consistent effect on precipitation in Northern Colorado/Southern Wyoming. Some El Ninos leave this area wet and sometimes dry–no clear pattern. During this most recent event I think I observed what explains this. During this recent El Nino we experienced a persistent northerly flow over Wyoming and Colorado in which the clash of warm/cold air occurred over the border region (from the I-70 corridor to I-80). Thus, Northern Wyoming was cold and dry, the border region wet and cold, and southern Colorado/northern New Mexico was dry and warm–the boundary between regimes being quite abrupt. Depending on circumstances, perhaps El Nino strength or location E/W in the Pacific, this pattern might shift north or south.
The patterns can move hundreds of kms from event to event. The general pattern is still there but it might have shifted by a few hundred kms north south east west.
I believe I read somethng on this site a few weeks ago that might be relevant. If I remember accurately the post said that in some el nino years the mass of warm surface water doesn’t make it all the way to the coast and as a result the rain tends to fall off or near the coast rather than more inland.
Obviously there are other major weather systems that interact but the el nino/LA Nina systems are beginning to be better understood and provide a good starting point for untangling the fantastic intricacies of global weather. Heat goes in at the tropics and drives a chaotic process in three dimensions while finding its way upward, poleward and back out.
Climate scientists should pull the plug on computer models and do some basic work on the slightly simpler subsystems.
A running mean does more than just distort locally. The “irregularities” it is used to suppress are not noise but information. More specifically, the sharp “noise” peaks it wipes out are the ENSO oscillation, manifesting itself as El Nino peaks with La Nina valleys in between. People simply don’t think about that because “out of sight, out of mind.” applies. It never occurs to them to ask why the entire hundred fifty year global temperature graph is covered by shark teeth spaced five years apart. I had a few things to say about that in a comment I attached to their paper.
Does not El Nino happen at different times of the year?The impact of each occurance must have to take account of this.
Wills wrote, “The paper made a curious claim, that La Nina conditions were wetter, and El Nino conditions were dryer.” I don’t agree! The figure Willis labeled as Figure 1 (it’s actually Figure 4) only shows that, for the locations indicated, La Niñas tend to be wetter.
Nick Stokes pointed out correctly that their real Figure 1 shows that the authors were well aware of the fact that Los Niños make it drier some places and wetter other places. They are not claiming that there is less rain globally.
Willis missed the whole point of the paper, which is that the paleoclimate record shows that there are large climate fluctuations that occur on century time scales and which can effect global temperature. In the discussion at the end of the paper the authors state, “Our analysis of multicentury hydroclimate variability suggests that projections of tropical rainfall patterns, and global temperature extremes, will remain uncertain until paleoclimate records and models consistently capture the lower-frequency variability, and associated feedbacks, in the tropical Pacific.”
This implies that it is possible that all, or almost all, of the current warming (mild though it is) could be due to low-frequency variability that climate models do not capture. I think that’s an important finding, although it’s not the first paper to show such evidence.
The whole thing is now a funding vehicle with advanced running gear to the1982/3 model.
Previously El Nino only referred to the Australian dry season which encompassed autumn through early spring. It was only called off-on at the end of any given year and there was no such thing as an El Nino summer, nor an El Nino for New Zealand, North America or Britain – in fact it only drove to Peru!
The Mustang GT350 is not the old ’82 Cortina.
The latest El Nino model purrs along at a billion dollars a day; with sea-level stabilisers, a global-warming gearbox and a climate-change clutch. It is fuelled by alarmism, driven by politicians and self-adjusts to inflation.
From Lu et al 2007 http://dx.doi.org/10.1175/JCLI4227.1 “The ENSO events generally do not impact the tropical total rainfall, but rather induce significant anomalies with opposite signs over tropical land and ocean”
hence:
Boening et al 2012 DOI: 10.1029/2012GL053055 “ Global mean sea level (GMSL) dropped by 5 mm between the beginning of 2010 and mid 2011”, “This temporary shift of water from the ocean to land is closely related to the transition from El Niño conditions in 2009/10 to a strong 2010/11 La Niña, which affected precipitation patterns world-wide.”
and
http://www.drroyspencer.com/2016/03/record-rainy-cloudy-humid-february-over-the-oceans/
In La Niña times California gets drier, Australia gets wetter, and the world goes on.
Regards to all on a sunny evening,
w.
—————————————————–
Enjoy the sunny evening.
For our viewing pleasure tonight, a cloud, that is being called, “da mutter ship.”
http://www.spaceweather.com/images2016/11jun16/lenticular.png
Larger version available at http://www.spaceweather.com .
THE ‘MOTHERSHIP’ OVER MT. TOM: On the evening of June 9th, a spectacular array of UFO-shaped clouds appeared above California’s Eastern Sierra mountain range. The display was a sensation on social media as people in nearby valleys began posting pictures of the sunset-colored armada. One photographer, however, was in the mountains; ultrarunner Jeff Kozak of Bishop CA captured this edge-on view from 11,100 feet:
http://www.spaceweather.com/images2016/11jun16/lenticular.png
I was topping out on Morgan Pass at sunset when I looked south to see what I thought was The Mothership hovering over Mt Tom!” says Kozak.
In fact, it was a lenticular cloud. Lenticular clouds form downwind of mountain ranges where the air organizes itself into starship-sized waves. Although they appear stationary, moist air is constantly moving through them, condensing at the apex of the wave. Wind scults the clouds into giant saucers and voilà–spaceships in the sky.
Larger version available at http://www.spaceweather.com .
Note: Still looking for a rotation mechanism, as one of the contributing mechanisms, for ENSO changes.
1972-1980 9 Leap seconds added.
1980-1990 7 ” ” ”
1990-2000 6 ” ” ”
2000-2010 2 ” ” ” (1 in ’05’ and 1 in ’08’)
2010-2016 2 ” ” ” (1 in ’12’ and 1 in ’15’)
Earth was in a slowdown period before 1990’s. Since 1990’s has gathered some momentum.
Taken from: THE IERS BULLETIN C
AND THE PREDICTION OF LEAP SECONDS D. Gambis and the IERS Bulletin C notices given at their website.
Vuk, posted a pretty good reference to the variability of El Nino impacts around the world. I have correlated it for many decades for the Gulf Coast and far West Texas.
Thanks Willis. Very interesting, very informative (as usual). I had always thought of El Nino patterns as being East-West. Now I see they are North-South.
An ocean driven climate cycles of about 1000 years duration is shown convincingly in the linked study. This study shows an oscillation of about 1000 years wavelength. There appear to be approximately 500 year periods of alternate warm/cold and wet/dry conditions. The oscillation is a Lorenz type butterfly wing oscillator / attractor which alternately hangs in one of two states, or wings – the el nino dominated and the la nina dominated.
https://en.m.wikipedia.org/wiki/Lorenz_system
What is interesting is that these Lorenz millenial oscillations are in different phase in different places. Inter hemispheric bipolar seesawing is evident. The paleo records seem to fall into three categories:
1. North hemisphere like:
Flores, Sulawesi; Warm/dry in MWP, cool/wet in LIA.
2. Southern Hemisphere like:
Galapagos; Cool/wet during MWP, warm/dry in LIA.
3. Chaotically switching between NH and SH regimes: Peru, Ecuador
1sky1 June 11, 2016 at 1:31 pm
Thanks, 1sky1. Not sure what you are calling “geophysical data”, but a 10-year or 11-year boxcar filter is often used with sunspot data, leading to significant reversal of oscillations. See my post “Sunny Spots Along The Parana River” for one among many examples.
In terms of the Oceanic Nino Index that I was discussing, they are using a 3-month boxcar filter on data that will perforce contain a strong signal from the Madden-Julian oscillation (MJO). The MJO varies in period from 30-90 days … I’m sure you can see the problems with using a 3-month boxcar filter on that data.
Regards,
w.
An 11-yr MA will NOT produce any “significant reversal” of the sunspot series itself, whose narrow-band power spectrum is peaked right around 11yrs. Your Parana example shows such reversal only for the RESIDUALS of several prior operations, whose spectrum is peaked apparently at somewhat higher frequencies.
The polarity reversal problem can occur only when there’s significant spectral content at the side-bands of the MA filter. Sadly, “climate scientists” are seldom aware of the spectral structure of the signals they write about and concern themselves only with the MA’s obliteration of signal components corresponding to the length of the MA.
1sky1 June 13, 2016 at 1:25 pm
Thanks, 1sky1, and sorry you had trouble posting. Moderating is sometimes slow, it’s all volunteers and sometimes things just take time.
Regarding your claim about no significant reversal, you really should actually try the operation before making pronouncements about the result. Here’s the annual sunspot data series itself, along with the 11-year running mean.
As you can see, the 11-year running mean totally reverses the peaks and valleys of the sunspot data. In other words, your claim that “An 11-yr MA [moving average] will NOT produce any “significant reversal” of the sunspot series” is 100% wrong.
w.
Willis:
By cherry-picking the oldest, pre-Maunder segment of the sunspot series, you’ve managed to show some small, out-of-phase oscillations in the 11-yr MA output. But this is obviously the result of abnormally short cycles in this–the most unreliable–segment of the record. You will not find consistently noteworthy reversals in the more reliable, post-Maunder segment of the record, especially if a ~10.6yr MA is employed on the MONTHLY data. (That is not to say that small wiggles will not be encountered after that periodicity is zeroed out by the MA.) Before claiming I’m 100% wrong, you should at least show the complete smoothed record and recognize that the wiggle polarity is critically dependent upon which side of the first zero in the frequency response of the MA do the sunspot frequencies lie. Having used MAs extensively as a tool for removing periodicities and all their harmonics, I’m 100% confident in that matter.
1sky1 June 14, 2016 at 3:26 pm
I don’t know whether you are really that ignorant, or you are just trolling, but NONE of what I showed is “pre-Maunder”. All of it is after the Maunder minimum. So once again you are 100% wrong, we DO find reversals in the “more reliable, post-Maunder segment of the record”, and I showed them above. (We also find them in the time period 1930-1970, in case you want more recent data … and in any case why is the age of the data an issue?)
Finally, I see you are trying to move the goalposts. Before, you said
Actually, it is because the MA is about the same length as the period (11-years) that we get the reversals. Anyhow, first you said no significant reversals were produced. Now that I’ve shown significant reversals, you’re moving the goalposts and saying that there were no “consistent noteworthy reversals” ….
And how is any signal reversal not “noteworthy” or “significant”? If you’ve flipped the signal over, 1sky1, that is by definition significant.
w.
Willis:
My memory lapse about the name of the minimum in the early 19-th century (Dalton) scarcely affects the fact that the data is unreliable until ca. 1820 and has features not seen afterward. Nor does that alter the analytic fact that phase reversal can occur only when there is significant spectral content at frequencies HIGHER then the first zero of the frequency response, which is always equal to 1/T, where T is the duration of the MA. You are badly mistaken that it results from periods very near T, because those periods are almost entirely reduced to zero amplitude. And my reference to consistently noteworthy reversals was in anticipation of your examining the monthly series with a 127-month MA, which produces a reversal of direction every time that the difference between the current point and that 128 months ago changes sign. Had you done that examination, my meaning would be clear.
Following all this lively stuff on smoothing, with strongly held opinions that do not sit well together, is it not fairly clear that the science of “smoothing” is less than settled? some interesting technical aspects have been discussed quite heatedly in this thread, though I don’t recall seeing anything on the problem of starting and end points of time series.
Some postings recommend what seems to me to be rather obvious – to avoid “smoothing” and to use original observations to support ones ideas, or hopes, rather than simply discard, to a greater or lesser extent, observations that do not run in good (sequential) accord with other related observations.
Clearly, careful scrutiny of recorded observations is essential. There are several reasons why. The primary one is a technical mistake in the technology. The next most likely one (in my experimental experience) is digit transposition, like 27 instead of 72. This would probably be rather obvious to the careful experimentalist, but 56 and 65 rather less so.
I think that we have to take it as read that mistakes – and in the case of climate, deliberate ones – could occur, but that we just do not know for certain where they are.
In my analyses I avoid smoothing the original data, unless simply to replicate what others have done and thus assure myself that I am examining the same published data. Unless one knows otherwise an observation is an observation and deserves respect. The diagrams that I produce, always using the original data, but cannot publish here, disclose (I think) features of complex time series that are not apparent to most people who scrutinise the same data, but which I believe are pertinent to the understanding of what can be expected from careful time series analyses of climate related data, and what can not. The important one in my opinion is that climate time series should /never/ be extrapolated beyond very few (perhaps 3) time steps. You can never be sure about what might happen!
At the final stage of many data analyses – climate in particular – smoothing in the form of fitting a (linear?) model is unavoidable, if simply to pre-digest very complex information into something that can be handled by the primitive digestive systems that are typical of the MSM and politicians, before they spew out their heavy type headlines to the general public.
I’m learning interesting stuff from this thread. Let’s keep it running!
1sky1 June 14, 2016 at 6:04 pm
So your bogus incorrect claim is now a “memory lapse”? How hilariously convenient.
Look, I took some data which you said would NOT have a reversal. It has a reversal. So your claim is hogwash. Now you are heavily into damage control. You are only hurting yourself with this flailing.
But if you truly think that there are no reversals after 1820, here ya go. I warned you not to uncap your electronic pen without testing your bizarre theories, but you continue to shoot off your mouth.
As you can see, the peaks in the red line match up with the valleys in the blue line. We call this “reversal”. Reversal that you claimed would not happen. Nice try. Your record of 100% wrongness is unbroken.
OK, I’ve done that. Here’s the monthly series with a 127-month MA. I see little difference between that and the annual series. It certainly does NOT produce reversals as you have described, “every time that the difference between the current point and that 128 months ago changes sign”. Instead, it’s doing what the annual MA did, in the same places. So once again you are 100% wrong.
Finally, I just looked at a simple sine wave with a period of 144. If you take a 154-point moving average of that sine wave, you get a large reversal. As expected, whenever the period of the MA is slightly larger than the underlying frequency that you get a reversal of signal, and when the period is slightly less, you don’t get a reversal of sign.
And this is the problem with the sunspot data. The problem is that the underlying frequency varies above and below that of the 11-year moving average. When the MA is slightly larger than the sunspot frequency you get reversals, and when it is slightly shorter than the sunspot frequency you don’t get reversals.
Which why, as I said, an 11-year moving average simply sucks when applied to a sunspot-related dataset. Because the period is so near to the sunspot period that the filter is sometimes longer and sometimes shorter than the sunspot data, it munges the data badly, reversing some sections and not others. I fail to see how anyone could defend such a monstrosity.
w.
robinedwards36 June 15, 2016 at 2:38 am
Nope. What is clear is that 1sky1 reflexively disagrees with anything I say. He is claiming, for example, that the reason an 11-year moving average reverses sunspot data is because the sunspot data is “unreliable” … say what? Just how would a moving average determine the “reliability” of data? If you think 1sky1’s claims hold water or unsettle the science of smoothing, you desperately need to take another look at his claims and try some examples for yourself. He’s just blowing smoke and hoping to impress the rubes.
Wrong thread. Take a look at my post called Michael Mann, Smooth Operator, which discusses just that.
While smoothing is often misused, so are hammers, but that doesn’t make a hammer any less useful when it is applied correctly.
Extrapolation is generally useless.
And I don’t mind smoothing to increase my understanding of what’s going on, as smoothing makes things clear visually. But I avoid using smoothed datasets for any kind of statistical analysis or as input to further analysis.
Works for me, thanks for your comments,
w.
robin:
The question whether to “smooth” or otherwise filter data is inextricably
related to the intrinsic nature of the data and the purpose thereof. If
the data contain no extraneous, confounding signal components and
demonstrate a high S/N ratio, there is no reason to filter. But that is
seldom the case with real-world geophysical data, where several unrelated
physical processes are often at play and the observations may be very
noisy. Indeed, very much different physical processes are found in
different frequency ranges of the power density spectrum of sea-surface
elevation, where sea level changes, tidal variations, long waves and
sea-swell superimpose upon each other. Frequency discrimination filtering
is often required to isolate the signal of interest in the time domain. And
even when there are no confounding components, strong high-frequency
fluctuations or noise can usefully reduced by judicious smoothing. (See,
e.g.,http://www.sidc.be/silso/dayssnplot.)
Instrument response characteristics usually smooth the highest frequencies.
Sunspot data are gathered daily, but are usually decimated into monthly or
yearly averages by simply subsampling corresponding MAs. Ironically, these
smoothings are ignored by those who preach the naïve mantra of: “NEVER
analyze smoothed data,” while manifesting little comprehension of the
spectral structure of the signal or the effects of noise. One can, of
course, resort to purely spectral techniques of analysis upon the raw data, but there is little demonstrated such proficiency in climate science, where presentation of results and of signal relationships is almost always done exlusively in the time domain. Nevertheless, the key to proper analysis of filtered data lies in fully comprehending the effects of the over-all frequency response.
Willis:
Instead of presuming to know the quirks of my memory, you should examine your own. At the outset I wrote about not producing any “significant reversal” of the sunspot series. You now claim, that I simply said “would not have a reversal”–without any qualification. This after I took pains to explain exactly when a reversal CAN take place. And then you dismiss that mathematically proven explanation as “hogwash.” Ironically, you then have the temerity to turn around and resort to that same explanation, but in different words, as your very own. Finally, only the tendentiously blind would claim that the 127-month smoothed series doesn’t show any, albeit tiny, reversals beyond those seen in the 11-year smoothing.
Such high-school polemical tactics are no substitute for producing the ENTIRE post-Dalton record of the 127-month smoothing., or for comprehending indisputable analytic facts. I’ll try to find more time tomorrow to explain what that smoothing really shows.
Despite the ability of the negative side-bands of the frequency response of
moving averages to invert the polarity of certain frequency components,
there are strong clues that something else is happening with the smoothed
sunspot data.
The first and strongest clue is the lack of any consistency of appearance of
out-of-phase wiggles. Even in Willis’ cherry-picked stretches of record,
and contrary to his earlier claim that “the 11-year running mean totally
reverses the peaks and valleys of the sunspot data,” they appear primarily
over the troughs of the unsmoothed data, often without corresponding
inversion under the peaks. No linear filter can produce such disparate
response at peaks and troughs. Furthermore, it’s apparent (particularly in
the monthly data) that the timing of actual reversal is not exactly
coincident with the troughs. Since no MA can change the phase relationship
by any amount other than 180 degrees, this points elsewhere than simple
inversion of appreciably higher-frequency signal components.
This suspicion is only reinforced by noting that some of the wiggles seem to
exceed ~10% of the original wave-height of the data, despite the analytic
fact that the amplitude response stays below 0.1 at all frequencies within
10% of the first zero of the M-point MA response function
H(f)= sin(M*pi*f)/(M*sin(pi*f)) for normalized frequency f 0.01 can produce a “beat frequency” matching the
visible centennial-scale grouping of Schwabe cycles and their attendant
minima.
What seems to explain the wiggles more convincingly is the persistent asymmetry of the Schwabe cycle, with more rapid rises than falls. Such asymmetry necessitates that there be significant spectral components well out-of-phase with the peak of the waveform. Also, the asymmetric shallowing of the troughs relative to the peaks requires net positive contributions from components concealed from the eye in the original data. They emerge into view, however, as a feature of the well-known Gibbs phenomenon when the major components of the Schwabe cycle are annihilated by the near-zero response of the MA. That inherent aspect of Fourier composition is notorious for producing features not at all visible in data.
I’ll conclude my comments tomorrow.
My comment disappeared again.
Failure to comprehend the essentials of the Fourier composition of signals
and noise leads to much confusion about the effects of smoothing,
particularly by MAs. The Slutsky effect, which is related to the Gibbs
phenomenon, is frequently misinterpreted by signal-analysis novices as
producing spurious features not at all present in the data. In fact, the
output of all filters is a strictly deterministic product of the data. In
the case of linear filters, the frequency of all components is unchanged;
only their and amplitude and phase are changed by the (generally complex)
frequency response function.
This should put to rest Willis’ ludicrous notion that “a smoother (such as
the 11-year running mean that they used on the Parana data above) merely
redistributes the energy present in the signal. A filter, on the other
hand, actually removes energy from the signal.” He then comes to the
multiply wrong conclusion: “The net result is that we end up with some of
the frequency data aliased into the average as amplitude data.” In fact,
there is no inherent difference between MAs and other low-pass filters other than
their frequency responses.
Certainly, MAs are far removed from ideal, positive definite low-pass
filters. But obtaining the latter (e.g., binomial filters) requires far
longer convolution kernels, with attendant greater loss of output to end
effects, to achieve the same frequency cut-off. Similarly greater losses
are incurred when cascading MAs to minimize the side-band ripples. MAs are
best viewed as simple, first-cut low-pass filters. Where they excel,
however, is in totally annihilating strictly T-periodic components by
virtue of their exact zero-response at 1/T and all higher harmonics.
They are thus the filters of choice in eliminating diurnal and annual
cycles.
Despite the negative side-bands of frequency response, the efficacy of
removing the 11-yr Schwabe cycle to reveal otherwise hidden
lower-frequency components is apparent at a glance in all of Wiilis’ figures
presented here. While railing against presumed, but unproven, inversions of
the Schwabe cycle by the 11-year MA, he totally fails to recognize tbat it’s not for that narrow-band cycle, but for these wide-band, lower-frequency components of the sunspot record that the authors of the paper claim high correlation with Parana River flow. That makes for a polemical tempest in a teapot, rather than credible geophysics.
Moderator:
Your delayed posting of my comment at 4:44pm yesterday deleted the following passage immediately after the phrase “frequency <= 0.5" on the previous line : "It can be shown that there's simply too little spectral content in the sunspot series beyond such a narrow frequency interval (centered at ~10.6yrs) to produce such strong wiggles by inversion consistently. Indeed,…" Please rectify this meaning-destroying elision.
Instead of simply inserting the passage I indicated in my comment to the Moderator yesterday, the third paragraph of my June 16 4:44pm comment was mangled even further. In its entirety, that paragraph should read:
“This suspicion is only reinforced by noting that some of the wiggles seem to
exceed ~10% of the original wave-height of the data, despite the analytic
fact that the amplitude response stays below 0.1 at all frequencies within
10% of the first zero of the M-point MA response function
H(f)= sin(M*pi*f)/(M*sin(pi*f)) for normalized frequency f 0.01 can produce a “beat frequency” matching the
visible centennial-scale grouping of Schwabe cycles and their attendant
minima.”.
Try again to post the original third paragraph:
This suspicion is only reinforced by noting that some of the wiggles seem to
exceed ~10% of the original wave-height of the data, despite the analytic
fact that the amplitude response stays below 0.1 at all frequencies within
10% of the first zero of the M-point MA response function
H(f)= sin(M*pi*f)/(M*sin(pi*f)) for normalized frequency f 0.01 can produce a “beat frequency” matching the
visible centennial-scale grouping of Schwabe cycles and their attendant
minima.
The persistent mangling is bizarre! Let’s try posting again, but with the line contining the formula placed at the end here.This suspicion is only reinforced by noting that some of the wiggles seem to
exceed ~10% of the original wave-height of the data, despite the analytic
fact that the amplitude response stays below 0.1 at all frequencies within
10% of the first zero of the M-point MA response function
It can be shown that there’s simply too little spectral content in the
sunspot series beyond such a narrow frequency interval (centered at
~10.6yrs) to produce such strong wiggles by inversion consistently. Indeed,
no frequency separation > 0.01 can produce a “beat frequency” matching the
visible centennial-scale grouping of Schwabe cycles and their attendant
minima.
H(f) = sin(M*pi*f)/(M*sin(pi*f)) for normalized frequency f <= 0.5
The third try seems to the charm. It’s astonishing how WordPress can mangle a comment containing a separate line of mathematical formula.
Why has my response disappeared without a trace?
Don’t have time to replicate my original response. Suffice it to say that an 11-yr MA does NOT produce “significant reversal of oscillations” in sunspot data, which has a narrow-band power spectrum concentrated near 11 years. Your Parana examples apply the MA not to sunspot data, but to RESIDUALS, whose spectrum obviously contains higher-frequency content. Furthermore, MJO is a high-frequency oscillation which doesn’t even dominate the atmospheric record, let alone the Oceanic Nino Index, whose very wide-band spectrum is peaked near 5.5 yrs. A 3-month seasonal data smoothing does negligible harm in examining relationships to ENSO.
My abbreviated duplicate response of a few minutes ago has likewise disappeared. I won’t waste any more time in the face of such pitiful evasions.
ENSO during positive PDO seems to behave more reliably than ENSO during a negative PDO.
Thanks, James. What do you mean by “behave more reliably”?
w.