The Story Told by the Southern Oscillation Index

Guest post by David Archibald

Bob Tisdale’s post on ENSO on 19th November prompted me to see what I could find in the Southern Oscillation Index (SOI) data. The SOI is calculated from the monthly or seasonal fluctuations in the air pressure difference between Tahiti and Darwin. Sustained negative values of the SOI often indicate El Nino episodes. These negative values are usually accompanied by sustained warming of the central and eastern tropical Pacific Ocean, a decrease in the Pacific Trade Winds, and a reduction in rainfall over eastern and northern Australia. Following is a graph of the SOI on a monthly basis from 1876 to 2010. The major El Ninos are discernable, otherwise it looks like a lot of noise.

The graph following shows the cumulative monthly SOI from 1876 to 2010.

The SOI does tell a story. It was non-trending for the last of the Little Ice Age and then from 1920 went into a long La Nina-dominated trend that ended with the Great Pacific Climate Shift of 1976. The planet warmed into the 1930s at the beginning of this trend, but then cooled, as it should have down in a La Nina-dominated trend, from the 1940s to the 1970s.

The subsequent El Nino-dominated trend from 1976 to 1995 was almost three times as fast as the rise. The Climategate emails show that Phil Jones was aware that global warming ended in 1995. The end of the El Nino-dominated trend in 1995 might be the physical cause of that cessation of warming. The SOI has been non-trending since.

This might have been a very neat story if the world had cooled instead of warmed into the 1930s. The 20 years of El Nino-dominant trend from 1976 to 1995 produced the late 20th century warming that got so many people hot and bothered. The story told by the SOI also reinforces how important the Great Pacific Climate Shift of 1976 was. The climate system turned on a dime for some as yet unknown reason.

Advertisements

86 thoughts on “The Story Told by the Southern Oscillation Index

  1. Sorry, clear as mud. I sort of get your point, but I am not clear how you got from SOI to your point. Maybe I am being thick, but maybe not? A little more explanation would be helpful, please and thanks.

  2. The graph showing the cumulative SOI is interesting, and suggestive. However, as you point out, the world was getting warmer during the 1930s despite the La Nina-dominated trend. Are you suggesting that there was 10 to 20 year lag between changing SOI and changing teperatures? If so, have you any ideas why there would be such a long lag period?
    Second question: how can you be sure that the 1976-1995 period of El Nino dominated trend was responsible for the warming and was not just the way warming got expressed? In other words which arguments do you have that it was the primary cause?

  3. The major El Ninos are discernable, otherwise it looks like a lot of noise.

    If you ask me, it is a lot of noise. Here is a suggested test – take the notation off that SOI index graph i.e remove the “1998 el nino”, “federation drought” etc, and hand it to someone who does not know when those incidents occurred. Do a blind test. Ask them to tell you which are Australian drought years…. do you think they will succeed?
    Hasn’t it been suggested that the best correlation comes from the Indian Ocean Dipole , not SOI?

  4. Dr. Archibald
    Recently I plotted diagram of five various events (independent of climate/temperatures) which I characterised as the North Pacific Gateways as shown here:
    http://www.vukcevic.talktalk.net/NPG.htm
    You will note that blue line is in a good agreement with your cumulative monthly SOI, with my graph about 10-12 years in advance to the cumulative SOI.
    Also you may notice that the red line has a close correlation with the PDO, again with about 10-12 years advance.

  5. Antonia says:
    December 9, 2010 at 12:35 am
    “for some as yet unknown reason”
    Sigh. A real scientist speaks. Thank you, David Archibold.

    Well it may not stay ‘unknown for too long’, now that I am aware that one of my graphs (see my post above) matches Dr. Archibald’s cumulative monthly SOI.

  6. Thanks David for soome interesting ideas.
    From memory 1976, was a year of several bad earthquakes and a new volcano, I think. Perhaps worth a look for sudden LOD changes that year?

  7. David Archibald wrote, “It was non-trending for the last of the Little Ice Age and then from 1920 went into a long La Nina-dominated trend that ended with the Great Pacific Climate Shift of 1976.”
    That’s one of the differences between the SOI and NINO3.4 SST anomalies. I recreated the ONI Index table using HADISST NINO3.4 SST anomalies in this post…
    http://bobtisdale.blogspot.com/2010/11/long-term-oni-like-table-of-el-nino-and.html
    I then did a quick count of El Nino versus La Nina events, and it contradicts your statement. Keying off the development year, during the period from 1910 to 1944, there were 10 El Niño events and 6 La Niña events. (For this simple comparison, if an El Niño or La Niña event extends from one winter to the next, it would be considered two events.) From 1945 to 1975, there were only 7 El Niño events, compared to 11 La Niña events. And from 1976 to present, El Niño events dominated again. There were 12 El Niño events but only 8 La Niña events.

  8. Aahh, the summer of 76, fond memories of hazy summer days during endless school holidays, picnics and trips to the seaside.
    Can we have some more please?

  9. “It was non-trending for the last of the Little Ice Age and then from 1920 went into a long La Nina-dominated trend ”
    Please keep in mind that during a La Nina phase, the ocean is actually absorbing heat. So while the surface temperature is lower because of the stronger trade winds, the reduced cloud cover is actually allowing more solar radiation to reach the water.
    So increased La Nina events means the Pacific was actually accumulating energy during that period. Notice how the graph makes a sudden drop at around 1933. That is the ocean giving the heat up to the atmosphere. It then continued accumulating heat until 1975. At that point the ocean began to give up heat to the atmosphere. If I am right, we should see a corresponding slackening of the trade winds since 1975 and an increase in tropical pacific clouds.
    Now the Pacific is dumping heat to the atmosphere. Until about 1998 or so and since then it has been about neutral. And looking at the various SST graphs you can see that. We had a significant La Nina in 2007/2008 followed by a significant El Nino .. they cancel each other.

  10. wilt says:
    December 9, 2010 at 12:31 am
    No, I don’t believe in big lags in the atmospheric climate, so the warming of the 1920s and early 30s may have been due to another factor – the Sun did become more active. On your second question, I am not saying which follows which, and I am not arguing that it is the primary cause.
    I was originally motivated to produce this graphic by Bob Tisdale’s observation that some heat or cooling remains in the system from each ENSO event. In the broad, that observation is borne out by this graphic. Perhaps, in the long term, El Ninos and La Ninas do balance out. In our 134 years of data, we have a non-trending steady state situation, then 56 years of La Nina-dominant followed by 20 years of El Nino-dominant – taking it back to where it came from. It could have stopped short, or it could have overshot – but it didn’t.
    Antonia says:
    December 9, 2010 at 12:35 am
    Thankyou. I was thinking that I wouldn’t be the one to propose a reason for the Great Pacific Climate Shift of 1976. But I am not a believer in random walk in climate. Perhaps the SOI or whatever causes it got locked into a long run state, and 1976 was when the system broke trend and started reverting to normal.

  11. “The story told by the SOI also reinforces how important the Great Pacific Climate Shift of 1976 was. The climate system turned on a dime for some as yet unknown reason. ”
    I don’t know if it was just a coincidence but 1976 was the year that Britain experienced a summer that broke all previous records. According to Wikipedia:
    “In the Central England Temperature series 1976 has the hottest summer for more than 350 years and probably for much longer. ”
    http://en.wikipedia.org/wiki/1976_United_Kingdom_heat_wave
    Since 1976 there have been a few summers that could be compared with it and I think that is one of the main reasons why the majority of the British population came to accept the idea that human activity is changing the global climate.
    Perhaps if the SOI continues along a similar path to the last few years belief in AGW will decline.

  12. David
    Can you link to the original data that you used for SOI and also can you tell us how the “cumulative monthly SOI” was calculated?

  13. This was around the same time the Salinity on the surface of the oceans started accumating.
    Interesting…

  14. David Archibald says: “I was originally motivated to produce this graphic by Bob Tisdale’s observation that some heat or cooling remains in the system from each ENSO event…”
    And the leftover warm water from El Nino events gets spun up into the Kuroshio-Oyashion Extension by the North Pacific gyre, and releases heat once again, but this time at Northern Hemisphere mid latitudes during the La Nina, assuming a La Nina follows the El Nino. And this secondary release of heat helps explain the upward shifts after major El Nino/La Nina events in:
    — North Atlantic SST anomalies,
    — Northern Hemisphere TLT anomalies north of 20N, and
    — Land plus Sea surface temperature anomalies (GISTEMP LOTI) between 20N and 65N.
    http://bobtisdale.blogspot.com/2010/12/enso-related-variations-in-kuroshio.html

  15. Bob Tisdale says:
    December 9, 2010 at 1:42 am
    My graph gets away from counting events and uses the cumulative SOI value. Not all El Ninos and La Ninas are created equal. This way we compare apples with apples.
    crosspatch says:
    December 9, 2010 at 1:57 am
    Thankyou. That may explain why the El Nino trend stopped. The oceans are back to no excess heat.
    John A says:
    December 9, 2010 at 2:35 am
    ftp://ftp.bom.gov.au/anon/home/ncc/www/sco/soi/soiplaintext.html
    That’s the link. If you email me, I will send you my Excel file on the SOI. david.archibald @ westnet.com.au

  16. I don’t know how the graph was plotted but I would guess it is a straightforward cumulative sum of differences (CUSUM) graph which is one of several standard methods of treating a time series. The way I would do it (the author will have to confirm or deny this) is to treat the series as a retrospective analysis.
    That is: obtain a set of data, then carry out a grand overall assessment rather than doing it year by year as a running assessment. Both methods could be used and have different methods of interpretation. The running assessment allows you to determine (to within statistical limits) whether a significant change in the value has occurred while the data is coming in.
    For a retrospective assessment I would find the grand mean. For each annual point subtract the grand mean and add it to the cumulative sum of differences for all previous years. Repeat this for all the annual figures and you will have a series of ups and downs with the graph eventually being forced to return to zero (a little thought will show this has to be the end result).
    The graph can be interpreted as follows:
    If a series of annual values is above the grand mean the graph will climb. If the values are the same as the grand mean for a number of years the slope will be horizontal. If a series of values is below the grand mean then the graph has a negative slope.
    My simple eye-ball interpretation of the graph (assuming it is a retrospective CUSUM) is:
    1876-1916 The SOI values were all close to the grand mean
    Around 1916 something happened to produce a step change in the (local) mean
    1916-1976 The mean of the SOI stayed constant at a higher value than the grand mean. The grand mean can be calculated from the slope
    Around 1976 something happened to produce a step change (reduction) in the mean
    1976-1995 The (local) mean value changed to below the grand mean for 20 years. Again, the value can be calculated.
    1995 Again something changed to shift the local mean of the SOI back to the grand mean.
    1995-end of graph The SOI has returned to the grand mean.
    Thus, it is the slope that matters and what you are seeking is to see whether one or more of the series of (apparently) constant local-means is different from the grand mean.
    There is a formal statistical test that is outlined by the UK BS on CUSUM analysis to determine whether the changes are significant or not.
    My suggested interpretation above is my guess. A proper analysis would need to be carried out but is far from arduous once the data is obtained. The calculations are easily managed by a simple spreadsheet.

  17. I still think it looks like noise, but, hey let’s try a hindcast.

    These negative values are usually accompanied by sustained warming of the central and eastern tropical Pacific Ocean, a decrease in the Pacific Trade Winds, and a reduction in rainfall over eastern and northern Australia.

    Can someone, preferably someone who does not know the answer, look at the SOI graph and tell me what the weather has been like in eastern Australia for the past 10 years.

  18. David Archibald says: “My graph gets away from counting events and uses the cumulative SOI value. Not all El Ninos and La Ninas are created equal. This way we compare apples with apples.”
    Which is why I usually use a 121-month running-average filter of NINO3.4 SST anomalies to show periods when El Nino and La Nina events dominate:
    http://i54.tinypic.com/33di5qr.jpg
    And that agrees with my simple analysis described above.
    Regards

  19. I find it interesting that El Ninos almost always have a stronger influence on global weather patterns than La Ninas.
    I also think that if ENSO was a product of TSI or cloud cover, long term ENSO predictions would have been more accurate than they are now.
    The “usual” conditions in the pacific is for the waters in the eastern pacific, near Peru, to be generally 8DegC COOLER than it’s latitudinal position would indicate. This is because of the cold ocean currents flowing up the coast of Chile, upwelling near Peru.
    When this upwelling slows, the surface waters warm quickly = EL Nino. The more dramatic this slowing, the stronger the El Nino. It’s effects are well known, floods in South America, droughts in Australia and North America and an increase in global temperatures.
    When the upwelling speeds up, the surface waters cool back down, higher than normal speed = La Nina
    The upwelling is the result of the thermohaline circulation, a phenomenon we know little about. I recall reading somewhere that there were many sensors released recently into the oceans to track the thermohaline circulation. If and when we get a handle on what makes this circulation tick, we’ll be able to predict ENSO events quite accurately.
    Caveat: Though solar insolation or cloud cover do not cause ENSO events, they may well modulate the strength of these events. High insolation during El Nino strengthens it, low insolation weakens it. Vice versa for la Nina.
    These are what I accept regarding ENSO events. I got my information from the web site of the late John L Daly HERE and HERE and many many other articles John posted about ENSO
    regards

  20. Dr. Archibald
    In this graph I superimposed your SOI cumulative graph with data for the ‘Pacific Gateway’(independent of climate / temperature events) located along 5 degree South parallel.
    http://www.vukcevic.talktalk.net/SOI.htm
    Only significant disagreement is about 10 year period around 1960, which I do not consider significant for reason of the nature of the ‘pacific gateway’ data.
    Dr. Archibald thank you for bringing the SOI cumulative to my attention.

  21. The great Pacific Shift happened at the same time that the Atlantic also started it’s warm phase. I am at a loss as to why the Pacific and Atlantic seem to cycle together, but the changes in both seem to happen at the same time.
    Whatever is causing both Oceans to cycle in this manner is what is causing the sawtooth in the global temperatures. That is the 1-2C variation each couple hundred years. We are near a peak of that activity and a drop-off will happen in the next 10-15 years, but the question is why?
    John Kehr

  22. John Kehr says:
    December 9, 2010 at 4:00 am

    The great Pacific Shift happened at the same time that the Atlantic also started it’s warm phase. I am at a loss as to why the Pacific and Atlantic seem to cycle together, but the changes in both seem to happen at the same time.

    Could be they cycle together due to the interconnected thermohaline circulation I mentioned in my post at 3:56am

  23. I don’t think there is a physical explanation which allows an accumulation index for the ENSO.
    The high resolution daily/weekly/monthly data shows that the ENSO has a continuous on-going impact on the climate which lags behind it by 2 to 4 months.
    There is also a smaller lag of about 8 months as the warm/cool ENSO waters circulate into other ocean cycles/areas like Kuroshio and the Indian Ocean. Atmospheric circulation patterns will then leave a secondary footprint in other major ocean cycles like the AMO (the AMO peaked in August and is now falling rapidly – down 0.3C in the last 3 months). Since the Atlantic and the Pacific are separated by land, atmospheric circulation is the only way the ENSO can impact the AMO (and landmasses in general I guess).
    After that, the ENSO’s impact is lost to the climate. It is lost to space. It doesn’t hang around for 20 years. The real climate does not operate like that.
    So, today’s climate is always being impacted by the La Nina of 3 months ago (and there is a smaller secondary lag of about 8 months which doesn’t always show up but does about 50% of the time). These lag timelines will explain a large amount of the general climate fluctuations.
    There are at least 5 major climate indices that show this pattern including temperatures, water vapour, cloudiness, outgoing longwave radiation and rainfall. So, the physical explanation for the lag timelines is evident in the climate indices themselves.

  24. David,
    My cumulative chart peaks in June 1976 (+680) with a double bottom(?) May 1998 (-485) and May 2007 (-487).
    Checking the BOM data you linked to I see that they are similar,but different, to the BOM SOI data I used 3 or so years ago.
    Any idea how many versions of SOI data BOM have?
    Are they subject to revision?

  25. Reminds me a bit of when I did a 6 year smoothing (I chose 6 years because it was the difference in time between the big dip in 1991 and the big El Nino in 97/98) of the Oulu neutron monitor. I noticed in that chart that the El Nino’s seemed to arrive durring the ramp up to max, not on a peak, much like what you were describing above.

  26. crosspatch says:
    December 9, 2010 at 1:57 am
    “It was non-trending for the last of the Little Ice Age and then from 1920 went into a long La Nina-dominated trend ”
    “Please keep in mind that during a La Nina phase, the ocean is actually absorbing heat. So while the surface temperature is lower because of the stronger trade winds, the reduced cloud cover is actually allowing more solar radiation to reach the water.”
    Excellent to remember. Many times I hear the statement, “the planet warmed, or the planet cooled, when all that is known, or factual would be, the atmosphere warmed or cooled.

  27. It would be interesting to see the cumulated SOI plottet with the AO, AAO and the NAO. Maybe a more complete story would begin to emerge..

  28. “clear as mud”
    That’s your point of embarcation. Return to basics: SO is Gaia’s heat sink, NH her radiator fins. Solar input runs on(from our vantage point) a 400 year cycle of activity. As earth heats, atmospheric H2O rises increasing albedo. Earth is turning off Solar input just as that input drops.
    Thanks for another salient addition to the puzzle, Mr. Archibald.

  29. David Archibald wrote: “No, I don’t believe in big lags in the atmospheric climate, so the warming of the 1920s and early 30s may have been due to another factor – the Sun did become more active.”
    The difference between your cumulative SOI graph and the variations in global temperature could and should also suggest that the sea level pressure difference between two points in the Southern Hemisphere does not represent the effects of ENSO on global temperatures. And that’s because the SLP at those two points are also impacted by processes that aren’t related to ENSO.
    It could and should also suggest that there may be problems with the early SOI data. SOI data is made up of two Tahiti SLP datasets, before and after 1935. The pre-1935 Tahiti SLP data was “discovered” in the late 1980s. It was incomplete, like many datasets. And those problems have to be understood and presented.
    Again, if we use NINO3.4 SST anomalies and not SOI data, we can use period average NINO3.4 SST anomalies to show that global temperatures rose while El Nino events outweighed La Nina events and that global temperature dropped when La Nina events dominated.
    http://i56.tinypic.com/zxmsg8.jpg
    That graph is from this post:
    http://bobtisdale.blogspot.com/2010/11/multidecadal-changes-in-sea-surface_17.html
    Regards

  30. crosspatch says:
    December 9, 2010 at 1:57 am
    Indeed, remembering that the oceans have 3000 times the heat capacity of the atmosphere, global temps, whether measured at the surface or near the tropopause are just noise.
    I’d bet a differential equation presenting evolved heat would output something approaching the cumulative SOI.

  31. Bill Illis says: “I don’t think there is a physical explanation which allows an accumulation index for the ENSO.”
    Somewhat along those lines, I don’t believe there’s an ENSO index that captures the true effects of ENSO. None capture the release of warm water from below the surface of the PWP and the redistribution of that warm water after the El Nino has ended. None capture the increase in Downward Shortwave Radiation over the Pacific Warm Pool during an El Nino, which helps to supplement the El Nino and recharge the PWP. None capture the increase in precipitation over the PWP during a La Nina, with the increase in precipitation indicating of an increase in heat released into the atmosphere. Etc.

  32. John Kehr says: The great Pacific Shift happened at the same time that the Atlantic also started it’s warm phase. I am at a loss as to why the Pacific and Atlantic seem to cycle together, but the changes in both seem to happen at the same time.
    Whatever is causing both Oceans to cycle in this manner is what is causing the sawtooth in the global temperatures. That is the 1-2C variation each couple hundred years. We are near a peak of that activity and a drop-off will happen in the next 10-15 years, but the question is why?

    Then we have:

    Baa Humbug says:
    Could be they cycle together due to the interconnected thermohaline circulation I mentioned in my post at 3:56am

    YES!
    See: http://mason.gmu.edu/~bklinger/drake.pdf
    Which talks about the Drake’s Passage effect and how it drives the circulation of the ocean currents. (The circulation is NOT just an artifact of salty cold sinking).
    Then these two point out oscillations of many years span in the circumpolar current.
    http://oceancurrents.rsmas.miami.edu/southern/antarctic-cp.html
    http://www.nature.com/nature/journal/v380/n6576/abs/380699a0.html
    Put them together and you get periodic oscillations of the circumpolar current and winds that would cause oscillations of the overturning current that would drive La Nina / El Nino on a short term as well as possibly influencing the strength of the Gulf Stream and temperatures in Europe.
    (No, I’ve no proof for this conjecture as I’ve just made the connection… but the pattern “fits” and is worth a “dig here”…)

  33. Moderator-
    In the sentence containing “as it should have down in a La Nina-dominated trend”, please change “down” to “done”.
    You may delete this message.
    IanM

  34. It is my understanding that it is the Trade Winds that blow the warm surface water westward during a La Nina, permitting the cold subsurface water to rise.
    IanM
    ******
    At December 9, 2010 at 3:56 am Baa Humbug said:
    I find it interesting that El Ninos almost always have a stronger influence on global weather patterns than La Ninas.
    I also think that if ENSO was a product of TSI or cloud cover, long term ENSO predictions would have been more accurate than they are now.
    The “usual” conditions in the pacific is for the waters in the eastern pacific, near Peru, to be generally 8DegC COOLER than it’s latitudinal position would indicate. This is because of the cold ocean currents flowing up the coast of Chile, upwelling near Peru.
    When this upwelling slows, the surface waters warm quickly = EL Nino. The more dramatic this slowing, the stronger the El Nino. It’s effects are well known, floods in South America, droughts in Australia and North America and an increase in global temperatures.
    When the upwelling speeds up, the surface waters cool back down, higher than normal speed = La Nina
    The upwelling is the result of the thermohaline circulation, a phenomenon we know little about. I recall reading somewhere that there were many sensors released recently into the oceans to track the thermohaline circulation. If and when we get a handle on what makes this circulation tick, we’ll be able to predict ENSO events quite accurately.
    Caveat: Though solar insolation or cloud cover do not cause ENSO events, they may well modulate the strength of these events. High insolation during El Nino strengthens it, low insolation weakens it. Vice versa for la Nina.
    These are what I accept regarding ENSO events. I got my information from the web site of the late John L Daly HERE and HERE and many many other articles John posted about ENSO
    regards

  35. I am not certain how Dr Archibald calculated his cumulative graph. I can see that it does not make sense in temperature terms to accumulate value over period of 20 years.
    However, if there is a spiky (noise like signal) than a 20 year moving average (running mean), makes sense. If during the process the sum is retained without dividing by 20 (for 20y m.a.) then you would get a cumulative or 20 year integrated output.
    This helps to visually identify likely source of the variable, which may not be otherwise so obvious.
    To many this may not make much sense, but Dr. Archibald’s graph, in my view, is one of more important recent publication at the WUWT.
    Now it is possible to identify the source and cause of the Southern Oscillation.
    http://www.vukcevic.talktalk.net/NPG.htm
    I have also included the PDO graph, which I mentioned in a post on Dr. Curry’s ‘Climate etc’ blog; her commenthere

  36. E.M.Smith says:
    December 9, 2010 at 7:10 am
    Thnx for those links EM, I’ve added to my collection.

  37. E.M.Smith says: “YES!” in response to Baa Humbug’s question, “Could be they cycle together due to the interconnected thermohaline circulation I mentioned in my post at 3:56am”
    How? There’s a time lag (a couple of years at minimum) between the release of heat from the Pacific Warm Pool until the time that any residual warm water could be carried down to the ACC, then caught up into the Benguela Current in the South Atlantic, and carried north into the North Atlantic. If memory serves me well, an ENSO signal takes 8 years to travel around the Antarctic in the ACC.
    Through teleconnections or atmospheric bridges an El Nino event will impact the North Atlantic in a few months. Or in the case of the 1976 Climate Shift, it would be the transition from a 3-year La Nina that ended early in 1976 to a minor El Nino in 1976/77, and that drastic change may have initiated the change in the North Atlantic.

  38. @wilt December 9, 2010 at 12:31 am:

    Second question: how can you be sure that the 1976-1995 period of El Nino dominated trend was responsible for the warming and was not just the way warming got expressed? In other words which arguments do you have that it was the primary cause?

    As far as that goes, has anyone ever figured out what is the CAUSE of the El Niño itself? Everyone seems to look at it as a cause in and of itself.
    Yes, the central and eastern Pacific SST rises. Yes, to all the things we observe.
    But what is the underlying cause?
    Please don’t anyone tell me it is because the winds changed. That doesn’t account for the addition heat energy. And even with the winds changed, what are the arguments that it isn’t itself just an effect?
    So, that additional heat – where does it come from?
    I’ve looked all over the place and haven’t found a decent explanation – or even much of an effort – to find out where that comes from.
    El Niño is not an underlying cause. It is an effect itself.

  39. @John A says December 9, 2010 at 1:36 am:

    How is the ‘cumulative monthly trend’ calculated ?

    John –
    I scratched my head about that, myself.
    Try this:
    In the article David Archibald talks about the SOI going negative sometimes (and I assume positive at other times). I read that to mean that it all starts from a baseline of zero and negative values deduct from the cumulative SOI, and positive ones add to it. I would suppose Archibald means that – looked at long enough – the overall cumulative total should end up at or near zero.
    That would also imply that – in that long, long term – half the curve would be below zero and half above zero. That doesn’t seem to be the case during the interval of the chart, and that may mean that a negative trend may occur at any time in the next several decades. (But perhaps the starting point was at a low point.)
    For some reason, this graph – and that Great Climate Change of 1976 – makes me think of James Hansen in the 1970s and his prognosis that warming was on its way. All I remember from that winter of 1976-1977 was how utterly arctic the U.S. was. It was a year I moved out to the boonies, and it cured me of ever wanting to do that again.
    But Hansen may have noticed something that he has kept mum on all these years, something that told him a reversal of the cooling trend was imminent. That may be giving him more credit that he is due, but the thought did occur to me.

  40. Those who say, “So what?” to this are missing the point.
    Of COURSE this is just observing an artifact. ALL climate measurements are artifacts. There is not a direct measurement of a cause in any of it. We measure temps and pressures and precip and SSTs and Arctic ice and tree rings and ice cores and coral reef growth, and all of them – without exception – are artifacts. They are all symptoms. Saying “So what?” could be applied to the temperatures just as well.
    The causes are the various energy outputs that reach us from the Sun, whatever heat comes out of the Earth, the tilt of the Earth, the laws of chemistry and physics, the rotation of the Earth, the configuration of the continents and mountain ranges, and apparently the Milankovitch factors. Numbers 4, 5 and 6 in that list are constants; the rest are variable. Warmers would say human activity should be included, but thus far that has really only been the pea under the Princess’ mattress.
    Just as “everything is derivative” in out culture, everything we measure in climate is an artifact.
    I applaud Archibald for finding a new perspective on this. If ENSO is important, it behooves us to find the significant measurements to make. Since we don’t understand climate yet, it is obvious we haven’t found the right things to measure and interpret yet. Our collective light bulb has not yet illuminated us yet.
    So why not rearrange the data a bit, look at it in other ways? I think history would show that we don’t know which ways of observing are important until we get smacked in the face by them. Most of the ways we are looking at it now are not working, are they? Not yet. Maybe fairly soon, but not yet.

  41. Synchronized chaos, folks. There are many factors involved and they don’t all work together at the same. When they do however, that is when climate shifts take place. See Tsonis.
    Are we now witnessing a synchronized event as described by Tsonis?

  42. Bob Tisdale says:
    December 9, 2010 at 8:16 am
    How? There’s a time lag (a couple of years at minimum) between the release of heat from the Pacific Warm Pool until the time that any residual warm water could be carried down to the ACC, then caught up into the Benguela Current in the South Atlantic, and carried north into the North Atlantic. If memory serves me well, an ENSO signal takes 8 years to travel around the Antarctic in the ACC.
    Hi Bob, good question.
    The only answer I can come up with at this stage, and it’s more than likely that I’m wrong, is rather than thinking of it as a “body of warm water” meandering for thousands of miles, I think of it as a channell of flowing water. If this flow slows then thousands of miles at the other end (the North Atlantic end) it also slows thereby allowing the sun to warm the surface. But for this to be true, there has to be an upwelling zone in the North Atlantic. I believe Thermohaline Circulation models show that there is, off the coast of Nth West Canada Sth of Alaska.
    Is just a thought.

  43. Ian L. McQueen says:
    December 9, 2010 at 7:39 am
    It is my understanding that it is the Trade Winds that blow the warm surface water westward during a La Nina, permitting the cold subsurface water to rise.
    Hi Ian, here is the way John L Daly explains it…my bolding
    The El-Niño Southern Oscillation is the result of a cyclic warming and cooling of the surface ocean of the central and eastern Pacific. This region of the ocean is normally colder than it’s equatorial location would suggest, mainly due to the influence of northeasterly trade winds, a cold ocean current flowing up the coast of Chile, and to the upwelling of cold deep water off the coast of Peru.

  44. Feet2theFire says:
    December 9, 2010 at 8:29 am

    As far as that goes, has anyone ever figured out what is the CAUSE of the El Niño itself? Everyone seems to look at it as a cause in and of itself.
    Yes, the central and eastern Pacific SST rises. Yes, to all the things we observe.
    But what is the underlying cause?

    I hope this is useful to you. At the John L Daly site there is hours and hours of reading about the SOI or ENSO. Most of it is by a man called Theodore Landscheidt.
    His equations and charts are way over my head but the gist of it that I could gather makes sense. He claims a high degree of accuracy in predicting ENSO 2-3 years out.
    make a start here if you wish http://www.john-daly.com/sun-enso/revisit.htm

  45. Bob Tilsdate says: http://i54.tinypic.com/33di5qr.jpg
    You say you filter the data with a 121 month running mean. You recognise this is a filter, do you not worry the frequency response of this filter may give misleading results or cause you to miss something?
    What mean did you use here? Just a straight box car running mean?
    I have done some work of filter frequency responses, the number and magnitude of side lobes are frightening.
    Can you link to the data you plotted here? I’d be interested in trying some other filters.
    regards.

  46. Re: DR
    For uncritical consumers, your line of influence may be hazardous. Take a minute to think about how nonstationary spatiotemporal modes alias. When the assumption of randomness fails, Simpson’s Paradox bites HARD with SHARP teeth. Suggested: Consider the possibility of strange nonchaotic attractors and switch to multiscale phase-aware data exploration (ethically resisting misguided conventional mainstream pressure to underpin meaningless statistical inference with absolutely untenable assumptions). Regards.

  47. Bob Tisdale,
    With anomalies, we can see more variation around the average annual cycling in the KOE, but temperature-precipitation relations are driven by absolutes. Seeing this graph [ http://i56.tinypic.com/29e0pvp.jpg ] in absolutes might add even further insight. Certainly we do need to be aware of where the big amplitudes are when looking at aggregated summaries (e.g. global averages).
    Your notes on KOE dovetail with what I’ve been saying about solar cycling & earth rotation rates. For example, see recent notes here [ http://wattsupwiththat.com/2010/12/07/massive-solar-filament-eruption-captured-by-sdo/#comment-547001 ] in response to a comment from vukcevic on the ~1925 X pattern, where seasonal heating/cooling patterns crossed in the NH.
    You had an article awhile back where you succinctly illustrated how the North Pacific is warm during the cold phase of PDO (contrary to widespread misunderstandings about PDO, which are seriously interfering with progress in these discussions). I think readers might benefit from a link to that article. (I’ve lost the link.)

    I agree with Bob Tisdale & Bill Illis that atmospheric teleconnections are the main ingredient. Nonsensical notions about long term ocean lags ignore the fact that oceans have no problem cooling down given a few months without much insolation (which varies seasonally & with interannual cloud cover). People are quick to point out the disparate heat capacities of water & air. That matters on diurnal timescales. Ask any sea-kayaker. The ocean might not be colder at night than during the day, but it most certainly is cooler in winter than in summer. (Decades not necessarily required, just a change in insolation persisting for at least many days.)

    David Archibald, you might next explore the effect of recentering your anomalies on your integral of SOI. The base period matters. Most agencies are using base periods centered on the recent warm period. The early 20th century “issue” with your graph goes away if you recenter to something less biased by the late 20th century. I also suggest using -SOI [i.e. negative SOI] instead of SOI so that up corresponds with warm persistence and down corresponds with cold persistence. The next step (e.g. for anyone trying to understand what vukcevic is hinting at) is to look at the integrals by month to explore variations in seasonal persistence. I recommend doing so via a color-contour plot (available in Excel).
    Best Regards.

  48. Bob Tisdale, in reading your recent article …
    Tisdale, Bob (2010). The ENSO-related variations in Kuroshio-Oyashio Extension (KOE) SST anomalies and their impact on Northern Hemisphere temperatures.
    http://bobtisdale.blogspot.com/2010/12/enso-related-variations-in-kuroshio.html
    … I noticed some musing on dynamics of ~1988 when AO persistence switched decidedly to positive & AO/NAO coupled for about a decade with global patterns (like GLAAM & LOD’). Interannual NPI coupling with interannual AO/NAO was briefly reversed at the beginning of this interval (rare during the 20th century). I detected this using a new algorithm that represents correlations of adjacent derivatives in the complex plane. Subsequently I found this supporting evidence:
    Schwing, F.B.; Jiang, J.; & Mendelssohn, R. (2003). Coherency of multi-scale abrupt changes between the NAO, NPI, and PDO. Geophysical Research Letters 30(7), 1406. doi:10.1029/2002GL016535.
    http://www.spaceweather.ac.cn/publication/jgrs/2003/Geophysical_Research_Letters/2002GL016535.pdf
    Note their speculation on a temporary change in the number of spatial modes. (This could certainly help with explaining why EOF1’s explain so little of the total variance.)
    Data exploration using a high-resolution complex wavelet suggests that the change was driven by an external factor: the sun.

  49. Looking at this 1876-2010 SOI graph, I have a simple and perhaps stupid question.
    The AGW mantra includes the claim of ever more ‘disrupted’ and extreme climate swings, yet that alleged trend does not appear to be evident. If anything, it looks like the opposite trend.
    So what’s up with that, if anything?

  50. P. Solar says: “You say you filter the data with a 121 month running mean. You recognise this is a filter, do you not worry the frequency response of this filter may give misleading results or cause you to miss something?”
    I’m using the filter in this instance to show that there are long-term changes in frequency and amplitude in the NINO3.4 data—that there’s more to the data than just noise. And I use it to show that the data switches from positive to negative at times that coincide with warming versus cooling epochs. That is, El Nino events dominated from 1910 to the mid 1940s (global temperatures rose), La Nina events dominated from the mid-40s to the mid 1970s (global temperatures fell), and El Nino events dominated after the mid 1970s (global temperatures rose again). I’ve used other spans for the running mean with similar results. And when I use the 121-month running mean in a post, I usually preface it with something along the lines of, it’s the same filter used by the NOAA ESRL for their AMO data.
    You asked, “What mean did you use here? Just a straight box car running mean?”
    The value at month 61 is the average of the NINO3.4 SST anomalies for months 1 through 121. The value at month 62 is the average of months 2 through 122. Etc.
    You asked, “Can you link to the data you plotted here?”
    Starting here:
    http://climexp.knmi.nl/selectindex.cgi?someone@somewhere
    Click on NINO3.4 in the second row, which is the HADISST1-based NINO3.4 SST anomaly data. On the next page, scroll down to “Manipulate this time series” and enter 1900 & 2011 in the “select years” fields. Click on “select”. On the next page, scroll down to the third graph, which is anomalies, and click on “raw data”. There’s your data. (From the looks of the graph we’re discussing, I believe that I also used 1950 to 1979 as the base years, which you would select after you limit the years. But as you’re aware, the base years will just shift the curve up and down.)

  51. Paul Vaughan says: “Seeing this graph [ http://i56.tinypic.com/29e0pvp.jpg ] in absolutes might add even further insight.”
    The GISS data is not available in absolute temperatures, only anomalies. Also, the differences between TLT and surface temperatures are great. For example the average TLT for the No Hemisphere north of 20N from 1995 to 2009 is approximately -6 deg C, while the KOE SST is about 16.5 deg C. Plotting them togther on a graph would serve little purpose.
    And thanks for the link to Schwing et al.

  52. Ian L. McQueen says: “Caveat: Though solar insolation or cloud cover do not cause ENSO events, they may well modulate the strength of these events. High insolation during El Nino strengthens it, low insolation weakens it. Vice versa for la Nina.”
    Do you have a link to a paper or data that supports this? Cloud cover follows the warm water east from the Pacific Warm Pool during an El Nino event, so DSR over the central and eastern equatorial Pacific decreases during an El Nino. But DSR also drastically increases over the Pacific Warm Pool during an El Nino.

  53. I’ve been reading this thread with great interest, from the original plot by David Archibald through to all the comments, explanations and queries about cumulative plots of time series. Over the last 16 years I have made literally thousands of cumulative sum plots on virtually every climate related time series that I’ve been able to find, or which have been sent to me. The “technology” is very simple indeed, as has been shown in various postings above. Interpreting the plots is something that one acquires with experience, and I don’t want to embark on a “lecture” about it here, but I would be very willing to go into some of the “subtleties” if pressed!
    I have cusum plots of /loads/ of assorted climate indices, and these readily confirm the main points made by David. The 1976 shift in the PDO is a nice example, and as I’ll claim below, it can be “quantified” in a temporal sense to a particular month or possibly two. Climate step changes in Alaska, derived from the several readily available station data sets turn out to occur a small number of months after the PDO shift.
    The great plus point for cusum analysis is that it produces a pattern, or trace, or fingerprint, that virtually instantly describes the large-scale behaviour of a time series. Attempting to do this from standard data plots, or plots “enhanced” by smoothing processes, is generally rather difficult. For instance, it is almost impossible to identify a step change from a conventional plot, much less attach a good estimate of the date of the step, and still less its magnitude. All these are quite trivial when examining a cusum plot.
    What I have found over the years is that climate data are suffused with step changes, which seem to punctuate long stable periods. By long I mean perhaps 20 months up to two centuries. Of course the departures from a straight line cusum (a period of stability) increase with the length of the series, and with the time resolution one chooses to use. There is also the problem of dealing with subjectivity in relationship to choice of periods that one might believe to be “stable”, but once you’ve looked at several cusum plots you’ll realise that the major features of the plots are usually so obvious that you’ll wonder why other people haven’t noticed them! It is possible to use standard (though often fairly computation intensive) methods to identify step changes from cusum plots. I have tried this but with my limited facilities (and faculties) it is a real chore. For academic purposes I guess one would have to go about it that way, but from the practical aspect I’d expect that it is not worth the hassle. The differences would be trivial, I would bet.
    If anyone is interested in the sorts of things that cusums reveal I would be happy to correspond via email. I could then send plots illustrating loads of fascinating things that climatologists seem to have overlooked – such as the NW Atlantic step change in the autumn (fall?!) of 1922, or the extraordinary behaviour of Central England temperatures.
    Robin

  54. I would like to respond to a number of points made in this most interesting thread.
    I am willing to expand on any of the points that I am about to make.
    But will not do so now, in the interest of brevity.
    I have found that rainfall is a better indicator of climate than temperature.
    At least, when both are expressed as annual totals/ averages (rain/temperature).
    There appears to be no long term tred in the rainfall at a number of individual Australian locations.
    Rainfall like othere aspects of climate is chaotic, in the mathematical sense.
    Hurst examined records of 841 years of river Nile flow and from that developed R/S analysis, which he measured on a scale of zero to one.
    A Hurst number of 1, indicates a series that is fully auto-correlated, which means that if the series starts to wander say upwards, then it will continue in that direction until an external force pushes it back down again. The share market is an example of series with high Hurst numbers.
    A series with a Hurst value of 0.5 is truely random – Einstein’s Brownian motion is an example.
    Series with values approaching zero are trendless and have strong tendence to revert to the mean.
    I have found the rainfall at locations such as Sydney & Mrlbourne, which have long rainfall records (150 years plus), have Hurst values very nearly zero.
    Rainfall in Australia fluctiates wildly on a multi decile basis, but there is no long term trend.
    Now having said all that quite briefly, I turn to my main surmise.
    It is possible that all the trends that you are all calculating, are merely long term fluctiations.
    You cannot find the principal driver, because there may be none.
    The spinning earth, its yearly passage around the sun, fluctiating magnetic force line between earth and sun (11/22 year sun spot cycles) and so on, all kick things around and around – AMO, PDO ENSO and so forth respond – all trying valiantly to catch up.
    Perhaps what goes around just comes around again and again and again; chaotically and on multi different time scales.
    Until there is some basic change in the system and another ice age or hot house era emerges.
    That also could just be due to the random coincidence of independent directionless forces coming togther haphazardly.
    We humans keep looking for pattern, causes, directions, purposes, when none may in reality exist.

  55. Bob Tisdale replying to P. Solar: “[…] the base years will just shift the curve up and down.”
    …by a different amount for each month – and each of these 12 amounts can vary substantially as one adjusts the base period. In some types of analysis, this substantially changes what turns up. I advise making sensitivity analyses a habit.

    Robin Edwards, Looking at integrals & derivatives is among the first things I do to learn the nature of a new time series. I suggest organizing some of your top highlights into a webpage or perhaps a guest post. The note you mention about the North Atlantic dovetails with findings of Harald Yndestad. He suggests there was a fundamental change in the N. Atlantic sometime around 1923 (based on signal analysis, including wavelet analysis of a variety of time series).

    vukcevic, I saw your note on the other thread. I suspect the 45 year pattern relates to alignments of the QBO with the year. It doesn’t take more than a few minutes to write a little program in Excel that will figure out stuff like that for any period you want to enter. The program will also illustrate why periods jump around – for example as one sees in the LOD time series. Everyone needs to realize that due to north-south, polar-equatorial, & continental-maritime terrestrial contrasts (for a few simple examples), nonstationary signals won’t necessarily sit conveniently piled in global FFT power spikes.

  56. ~ Bob
    I’ve grabbed the data you indicated but it does not really look anything like your plot. Could you cast an eye over this snippet and tell me if that’s the data you intended to point me too.
    2009.2500 -0.190559
    2009.3334 0.304281
    2009.4166 0.574663
    2009.5000 0.800828
    2009.5834 0.694503
    2009.6666 0.778035
    2009.7500 0.997765
    2009.8334 1.57316
    2009.9166 1.80899
    2010.0000 1.40437
    2010.0834 1.15557
    2010.1666 1.12739
    2010.2500 0.615990
    2010.3334 -0.235387E-01
    2010.4166 -0.538576
    2010.5000 -0.785873
    2010.5834 -1.25070
    2010.6666 -1.46854
    thanks.

  57. Bob Tisdale, there’s also a decided upturn in the integral of AAO/SAM ~1992 that I suspect (based on multivariate analyses) is related to the upturn in AO/NAO a few years earlier. A question I’ve been meaning to ask you: Where was the ~1940 upturn in SST most distinct? (It seems to me I remember seeing a graph of some section of the Indian Ocean where ~1940 spiked uniquely in a long record lacking other prominent features…)

  58. AusieDan, people see chaos in places where it does not exist (perhaps proving your point). Have a look at this for some stimulating ideas:
    White, W.B.; & Liu, Z. (2008). Non-linear alignment of El Nino to the 11-yr solar cycle. Geophysical Research Letters 35, L19607. doi:10.1029/2008GL034831.
    https://www.cfa.harvard.edu/~wsoon/RoddamNarasimha-SolarENSOISM-09-d/WhiteLiu08-SolarHarmonics+ENSO.pdf
    Russian scientists are arguing that we should be looking for a strange nonchaotic attractor (which would still be very hard to figure out).

  59. Ian L. McQueen says: at 7:39 am
    This is because of the cold ocean currents flowing up the coast of Chile, upwelling near Peru.
    The upwelling is the result of the thermohaline circulation, . . .
    E.M.Smith says: at 7:10 am
    YES and so on . . .
    I’m late to this post and want to express an interest in the – more than usual – ocean circulation comments.
    I appreciate seeing such comments (and links) because lack of this sort of knowledge often leads to very elementary mistakes. One I’ve seen a couple of times is an expression of surprise when someone finds out that sea level isn’t the same across the world or even for the same ocean.
    More concern for the ocean — currents, the size, timing, and shape of the various basins – seems to be required. For example, does not the surface water along the west coast of South America move away from the coast following the principle outlined in Ekman transport?
    http://en.wikipedia.org/wiki/Ekman_transport
    So, the pressure system west of South America contributes to the development of the South Eastern Trades. From the link just above:
    Ekman transport is a factor in coastal upwelling regimes which provide the nutrient supply for some of the largest fishing markets on the planet. Wind in these regimes blows parallel to the coast (such as along the coast of Peru, where the wind blows North). From Ekman transport, surface water has a net movement of 90 degrees to the left in such a location. Because the surface water flows away from the coast, the water must be replaced with water from below.
    Now I’m off to read a few of those links others have thoughtfully provided. Thanks.

  60. AusieDan:
    Re: patterns vs. no patterns in rainfall data, I did a study some time ago of the rainfall over the Melbourne area. Taking the Bureau of Meteorology (BOM) data for the grid box centered at 38.0 deg S: 145.0 deg E, I did a seven period, centered, non weighted, moving average of the annual totals. When plotted, this showed a distinct periodic form averaging roughly an eighteen year period. There were some quite large variations in amplitude, but no noticeable trend. It sounds as if the lack of trend is consistent with your broader observations, however, a closer look at the cyclic pattern revealed an intriguing possibility.
    I superimposed a nominally 18.0 year period sine curve, and by adjusting parameters, closely matched the magnitude to a mean of 900 mm/yr, +/-55 mm. Scanning periods showed that it was not 18.1 yrs.. Rhodes Fairbridge has identified an 18.03 year cycle in tides affecting the Canadian-American east coast from Newfoundland down to the Gulf of Mexico, which he attributed to the Saros cycle of solar-lunar nodes alignment, associated with the occurrence of lunar eclipses. Could it be that the basic cycle also has a tidal influence here? I found that setting the period to 18.03 years, with the first high point at 1902.5, produced a very close temporal match.
    Looking at recent events, an obvious departure from the idealized sine wave occurred around the time of the 1998 El Nino, falling to about 110 mm/ yr below an idealized low in 2002.5, but then instead of then rising, stayed low until around 2008. This has been taken as evidence of a changing climate, but since 2008 rainfall has increased considerably, and the effect of the current La Nina looks like restoring the pattern back into line with the long term 18.03 year period cycle. There is an ideal high at 2011.6, and if the current weather pattern holds it seems very likely that will be achieved. Was the recent dry period really not a sign of long term climate change, but rather an unusually low excursion in a natural and normal cyclic pattern?

  61. Oops, the system did not like my symbols. Third line in second para. should read: Scanning periods showed that it was not less than 17.9 nor greater than 18.1 yrs..

  62. P. Solar says: “I’ve grabbed the data you indicated but it does not really look anything like your plot. Could you cast an eye over this snippet and tell me if that’s the data you intended to point me too.”
    That’s it with the default base years, and the default base years as far as I can tell are the term of the data, in this case 1900 to 2010.
    Regards

  63. After reproducing the Dr Archibald’s graph, it is obvious that from the point of pressure consideration it doesn’t have any physical meaning to it.
    However, that said if it is considered that the pressure time function is a result of a short-term time differential of another physical process, than such cumulative sum makes sense for the purpose of identifying that process.
    Here, that may be the case since it appears that cumulative SOI has a reasonable correlation, during the last 100 years, with a signal I identified as one of five critical ‘Pacific gateways’.
    Correlation excludes the 10 year period 1960-70, possibly related to the pacific islands nuclear tests (ending 1972).
    http://www.vukcevic.talktalk.net/NPG.htm

  64. vukcevic, the 1960-70 correlations/anti-correlations one finds when comparing terrestrial time series relate to rapid changes in persistence of AO/NAO & AAO/SAM and an associated spike in LOD’. In short: a reconfiguration of circulation. In layman’s terms: All one has to do is change the angle of a jet to make the eddies swirl in the opposite direction. Complex correlations help avoid Simpson’s Paradox (a trap for overly-linear thinking that is blind to deterministic-relationship sign-reversals). Cheers.

  65. Bob Tisdale said:
    “And when I use the 121-month running mean in a post, I usually preface it with something along the lines of, it’s the same filter used by the NOAA ESRL for their AMO data.”
    Yes, I find it a bit worrying how commonly these running means are used in view of how badly they behave, and above all that many seem to use them with unquestioning faith that a mean just filters out the noise.
    This page is a quick intro to gaussian filters and compares the frequency response of G and RM filters. http://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm
    The large and repeated side lobes let significant amounts of higher frequencies through that we tend to image we are filtering out using RM. These can sometimes come back to bite.
    To illustrate the point with the el nino3.4 data, here is a comparison of the 121 month RM to a gaussian filter. Though there is “some” similarity there are many peaks that are significantly displaced and periods like the ’80s when the amplitude is very different and the two plots are 180 degrees out of phase !
    http://i56.tinypic.com/24dr8eg.png
    So, which is a better representation of the raw data? Let’s look closer.
    http://i51.tinypic.com/2ypkodf.png
    Looking at 1985, 1989 and 2000 we see that the gaussian is following the raw data and the RM is going the wrong way. It is out of phase. There must be frequency components in the surrounding data that are getting through.
    BTW Met Office , Hadley also use a 21 month RM when presenting their NH/SH and global temperature data.
    Describing running mean as a filter gives the impression that what comes out the other end is somehow cleaner or purer, that some noise has been removed. However, to someone with a signal processing background the word filter says: “frequency response ?”
    It is better to think of filters as distorting the data rather than cleaning it up and ask if the distorted version accurately shows the features you are trying find or is displaying spurious artifacts.
    This may not affect too much your just trying to spot nino/nina periods but I would not chose to use a “filter” that gets both magnitude and phase so badly wrong.
    The implementation of a gaussian filter is no more than a weighted mean. It’s bearly any harder of computationally heavy than plotting a running mean.

  66. P. Solar: Many thanks for the detailed discussion and links on gaussian and RM filters.
    You wrote, “Looking at 1985, 1989 and 2000 we see that the gaussian is following the raw data and the RM is going the wrong way. It is out of phase.”
    But if my interest is to show, for example, that the frequency and amplitude of El Nino events outweigh the frequency of magnitude of La Nina events for specific periods of time, then the gaussian filter is picking up the high-frequency component and it’s out of phase. In other words, it depends on what one is interested in presenting.
    Regards

  67. “But if my interest is to show, for example, that the frequency and amplitude of El Nino events outweigh the frequency of magnitude of La Nina events for specific periods of time, then the gaussian filter is picking up the high-frequency component and it’s out of phase. In other words, it depends on what one is interested in presenting.”
    If the peaks are shifted this will affect the frequency even possibly the number of events you are going to identify. Some may be spurious artifacts due to the filter letting through some frequencies and not others. If you are interested in the magnitude , that magnitude is wrong, even inverted in some cases.
    I appreciate that you are only looking very grossly at larger trends but even that may be affected.
    I don’t understand your comment about gaussian “picking up the high-frequency component. It has no ringing or pass bands in its frequency response (which is it self a gaussian profile). In fact if you look at the plots you link here you will see a lot of detail in some areas that should not have got past a 5 year filter. These are artifacts. As seen in the years I noted this can even lead to a trough being plotted where there is a peak in the data. That would worry me.
    What do you think the gaussian result is “out of phase” with? Certainly not the data.
    I had a quick look at your blog and you seem to be doing a lot work looking for trends and causal links between different data sets. If you are using rm (which you are) you may well be missing some useful correlations or inferring ones that do not exist.
    I would suggest you try re-plotting some of those comparative graphs using G instead of rm . You may find some interesting differences. Maybe some correlation will appear stronger, some lags may (will) change in size or may become leads. It could be very enlightening.
    “In other words, it depends on what one is interested in presenting.”
    Which I am sure M. Mann, Steig et al would agree with entirely.

  68. There are no “bad” “filters” & “good” “filters”, just goofy interpretations. Without good instinct about the effect of integrating across harmonics, sensible interpretation might not be possible. “Frequency response” graphs won’t help with interpretation if one lacks basic conceptual understanding. Say one has a symmetrical (& sinusoidal) valley between 2 peaks. A repeat narrow-band smooth will see the same basic pattern. A single wide boxcar from peak-to-peak is obviously going to put a peak over the middle of the valley. That’s not a problem if the goal is to show the average elevation for a boxcar centered on that point. Does the general public have the skills necessary to sensibly interpret? No. Worse than that: I know career academic statisticians who get tangled in knots thinking about this stuff (because it isn’t necessarily something they’ve ever thought about carefully …certainly not because they are incapable – quite the contrary). There is no substitute for taking whatever amount of time is necessary to build fundamental conceptual understanding from scratch. “Frequency response” plots are not a magical shortcut to enlightenment. A background in signal processing is not necessary for developing conceptual understanding of fundamentals (nor is it a guarantee of natural instinct).

  69. Bob Tisdale says:
    December 9, 2010 at 2:45 pm

    Ian L. McQueen says: “Caveat: Though solar insolation or cloud cover do not cause ENSO events, they may well modulate the strength of these events. High insolation during El Nino strengthens it, low insolation weakens it. Vice versa for la Nina.”
    Do you have a link to a paper or data that supports this?

    Actually Ian was backquoting my comment Bob and sorry for the late response.
    No I don’t have any papers or data to point to. My comment was my opinion/idea that I wanted to contribute.
    However if I inadvertantly just added “noise” I apologise.

Comments are closed.