I was pointed to this graph by an email from WUWT reader Phil Ravenscroft and I’m reposting it here for discussion.
While the correlation looks plausible, it seems almost too good. Since the email tip for this graph did not include the source data files, I was ready to dismiss it.
UPDATE: my first impression was the correct one – see comments
But in doing my own research, I found myself being led back to late John Daly and his references to Theodor Landscheidt in this page. While I don’t put much stock in Landscheidt’s barycentric theories, I’ve never known John Daly to pursue a wild hare. Looking further, in the peer reviewed literature, there is this paper:
Connection between ENSO phenomena and solar and geomagnetic activity (PDF) by M A. Nuzhdina, Astronomical Observatory of Kiev National T. Shevchenko University, Kiev, Ukraine in Natural Hazards and Earth System Sciences (2002) 2: 83–89. European Geophysical Society.
I find this passage interesting:
The analysis of planetary fields of pressure shows that they
are connected both with the 11-year and the 22-year solar cycles
(Wagner, 1971). The atmospheric barometric centers,
Icelandic and Aleut depressions, Pacific, Siberian, Azores
anticyclones in the northern hemisphere, displace close to
the maxima of 11-year solar cycles. The Azores and Icelandic
barometric centers tend to displace on the east close
to the maximum of a solar cycle (Herman and Goldberg,
1978). The clockwise circulation, connected with Azores anticyclone,
causes passat winds in the north-east direction.
The response of barometric formations at the midday regions
of the Earth (namely, recess or filling of cyclones or
strengthening or destruction of anticyclones) depends upon
tbe sign of the magnetic field of the sunspot which is crossing
the central meridian of the Sun (Nuzhdina and Barkova,
1983). Spontaneous phenomena of solar activity (solar
flares) and crossings by the Earth of an IMF sector boundary
are accompanied by changes of atmospheric pressure and
cyclonic activity in some regions Mustel, 1972; Roberts and
Olsen, 1973; Herman and Goldberg, 1978). A low-pressure
region in the gulf of Alaska is more significant, when the
IMF is directed away from the Sun, than towards (Wilcox,
1978).
The authors conclude:
– QB and QA oscillations in ENSO data are coherent with
the same oscillation in Ap-index and Wolf number data. 5.3-year oscillation is coherent in ENSO and Wolf number data.
– In our opinion, cyclic dynamics of ENSO phenomena
are due to solar activity and geomagnetic variations.
It is background long-period variations on which high frequency
oscillations are imposed.
This is an interesting concept and worth further discussion. My goal in posting this is to have our team of WUWT readers take a good hard look at this and see if the SOI – Ap graph has any merit.
I’m traveling today, so have at it. Please, please, keep on topic. Lately people have just been posting random links and OT’s.
Peter Ravenscroft (06:06:35) :
Sorry folks, I blew it!
The graph is too good to be true, as several suspected, but it took me a while to track what I had done wrong. I picked an Excel grapher that does not allow the different graphs to cross.
Peter, IMHO you did not blow it, Microsoft did. Why? They regularly ship user hostile software. Little land mines scattered for you to step on. You were just their latest victim. Yeah, you ought to have understood your tools better before using them, but I spent a couple of decades supporting MS software in companies and I couldn’t keep up with their bug lists, mis-feature lists, user hostile behaviours, etc. So if a guy who was employed full time in I.T. can’t find “issues” as fast as Microsoft can create them, how can the typical consumer?
So don’t feel too bad and don’t let it stop you from continuing your line of enquiry. Just next time think about using a different product for your first cut comparisons 😉
For those MS supporters who think I’m over stating this: In NT Server (yeah, a while ago, but I’m out of the field now so my stories will be old…) there was a ‘memory leak’ such that the server would gradually mark all of it’s memory as in use even when it wasn’t. The end result was that the server would gradually run worse and worse until it hung, frozen, shutting down the whole work group using that server. A major telco where I worked at the time had the “process” that every Friday we were to reboot every single server so that they wouldn’t hang the whole company mid week of week 2. It took MS forever to fix this egregious bug.
In comparison, a Unix server (BSD on a PC) at another company had been running for a couple of years steady when I decided we needed to shut it down because the power supply fan needed replacing and was running way slow. Yes, the hardware wore out before the software needed a reboot. THAT is quality software.
I could list other bugs, misfeatures, and traps in Excel but I won’t. It’s too painful… But just consider the “feature” that bit you: A graph selection that changes the presentation of the data so that the two graphs will not cross and does not tell you this in a pop up when you select it. I’m sure the typical person who picks a graph does not expect it to change the data being graphed so that the graph looks better but does not represent the actual data.
So yeah, you were the captain of this ship and take responsibility for steering into the dock, but someone else sold you a boat with the rudder wired backwards some of the time …
Bob Tisdale (07:28:30) :
Here’s the unsmoothed data with the AP Index lagged 60 months:
http://i44.tinypic.com/rsdm2o.jpg
I spend most days looking at a lot of charts, trying to see the tiny pattern that makes a trade profit or gives tiny heads up before something falls apart. No, it isn’t mathematical, but yes, it works well (though it takes a while to get the brain trained…). To the extent this translates to non-stock non-price data, what I see in this chart is:
There seems to be a rough positive correlation of SOI to AP until AP approaches a peak then it looks like SOI has a breakdown and plunges in a downward spike when AP nears / hits a local high. Tracks 1937-1944, breaks 44-45, 47; tracks 48-52, breaks53; tracks 54-64, breaks 65 and 67; tracks until 72, breaks 73/4 (then SOI seems to ring until a break / dip in 79); tracks to 83, breaks 84; tracks weakly to 87, then breaks 88; tracks a bit low to 98, breaks 99; tracks to 2006 with a spike that recovers to a break in 2008.
This fine structure is similar to what happens with stock prices. They rise for a while, then at a peak have a sudden breakdown, followed by a return to the rising pattern with underlying business cycle activity. It is one of the things that a trader watches for. It is encoded in the wisdom to “buy the dips” and “take profits early”. In other words, a break in a rising trend is a chance to buy in cheaply, not sell; and a rising trend is not to be trusted since it will have a price break at some point (often after a nice smooth run up for a while).
Now maybe I’m just a one trick pony trying to use a hammer to drive screws in, but I’d go looking for some kind of breakdown process. Similar to a high voltage circuit in a TV that has the picture wink out every so often when a corona discharge zaps away the stored charge and the cycle repeats. (Or when folks have significant gains in stocks and start putting trailing stop loss orders behind their positions so that a momentary dip can turn into a cascade of selling…)
And here it is smoothed with a 12-month filter with the AP Index lagged 60 months:
http://i43.tinypic.com/6jplb5.jpg
This will remove the fine structure of a fast break, yet a large strong break can bleed through the average. What I see in this chart is tendency to non-correlation that is an artifact of the large breaks bleeding through the average to cause a spurious non-track over a longer time period that ought to be seen as a breakdown spike. It would be interesting to ‘peak clip’ the breakdown dips and / or use a longer time period average to mask them better and see if the correlation goes up.
FWIW, this is why trend following averaging indicators fail to work in stock trading. A faster cycle breakdown takes out your profits before you can react with your trending indicator. (i.e. the day trader shorts you to death before your moving average or MACD says to sell). Oddly enough, it also tells you how to win: IF you have a stop loss order (to get a fast response) have it close. You want to sell out quick in a down spike. IF you don’t, then use the dip to buy, not to sell. “Double down” the bet in a Grand Martingale when the dip inflects upward (‘buy if touched order”) as long as the longer cycle is in your favor (this only really fails at the longer cycle inflection, but you get several short cycle wins with it for each long cycle inflection that loses, so net the strategy works. Even though in truly random games of chance like roulette it is a losing strategy).
Also, FWIW, several of those breakdown spikes in SOI happen to be in or near years with stock market collapses. Coincidence? …
gary gulrud (07:50:47) : At last, someone points out the obvious and it gets past moderation. Thank you. He has been… petulant in other places and people cave because of his esteemedness.
Erl – Ulric sends regards and I say – love your stuff.
There are three of us in this office who frequent WUWT and one said recently-
“anyone checked out how the total june anomalies look when graphed” he meant extract all the june data from the CET record and graph it. Interesting.
Bob Tisdale (13:52:54) : Forecasters call those “lookback periods” some are numbered in months and some in years. When they work they are amazing.
Terry Ward (12:26:31) :
At last, someone points out the obvious and it gets past moderation.
“TEF is a totally meaningless concept.”
Normally one should not feed the trolls, but perhaps I have been reading the wrong literature. So, find me a couple of peer-reviewed papers discussing the meaning and usefulness of TEF.
“Lassen’s work is critically flawed and has been soundly rejected.”
It should be obvious to the sharp-eyed people here that there are severe flaws in the Lassen et al. papers. For once, one can simply repeat the analysis [without the dubious heavy smoothing] as in http://www.leif.org/research/Cycle%20Length%20Temperature%20Correlation.pdf . The rejection is succinctly discussed here:
EOS, TRANSACTIONS AMERICAN GEOPHYSICAL UNION, VOL. 85, NO. 39, doi:10.1029/2004EO390005, 2004
Pattern of Strange Errors Plagues Solar Activity and Terrestrial Climate Data
Paul E. Damon
University of Arizona, Tucson, Arizona, USA
Peter Laut
Technical University of Denmark, Lyngby, Denmark
Abstract
The last decade has seen a revival of various hypotheses claiming a strong correlation between solar activity and a number of terrestrial climate parameters. Links have been made between cosmic rays and cloud cover, first total cloud cover and then only low clouds, and between solar cycle lengths and northern hemisphere land temperatures. These hypotheses play an important role in the scientific debate as well as in the public debate about the possibility or reality of a man-made global climate change. Analysis of a number of published graphs that have played a major role in these debates and that have been claimed to support solar hypotheses shows that the apparent strong correlations displayed on these graphs have been obtained by incorrect handling of the physical data. The graphs are still widely referred to in the literature, and their misleading character has not yet been generally recognized. Readers are cautioned against drawing any conclusions, based upon these graphs, concerning the possible wisdom or futility of reducing the emissions of man-made greenhouse gases.
E. M. Smith,
Go ahead and bash Microsoft all you want, but you’re off on this one. Peter simply picked the wrong type of graph. No harm no foul. Blaming Microsoft for Peter’s mistake is just poor analysis – something we’re all tired of seeing from the “other side,” so let’s not do it ourselves.
Leif Svalgaard (13:17:54) :
Terry Ward (12:26:31) :
At last, someone points out the obvious and it gets past moderation.
I was not trolling about TEF.
I was trolling about your seeming AGW apologism and your overt derision and covert censorship. The closer I approach retirement the more I rail against blinkered dismissal of opposing views based solely on an unwillingness to break the chains of habit and think outside of self constructed boxes.
Given the power to do so, you always get any reply that disturbs your world view snipped. For someone who is all over Internet, on forums and discussion groups, and unhesitatingly and frequently offers third party rebuttal of a point of view that you refuse to put any effort into other than to offer scorn and to poke fun at, your energy and omnipresence is nothing short of remarkable for someone presumably busy at work. Your focus on the sun is commendable purely because you are a good solar scientist but your quoting from, and using as a shield, 100 year old papers that “prove” cyclomaniacs are mad if they attempt to present a system based on what you deem to be Astrology is as bad as the CO2 crowd claiming that 100 year old science works for them and we should all just fade away.
I have been shown something that you have not considered, that, with readily available tools (which you, no doubt, can access superior versions of and have staff, or interns, to delegate to the task) can be seen by this layman and you, and anyone, could repeat. There are definite alignments of the planets that, somehow (I am not claiming to know how) have cyclical effects upon the sun and, subsequently, upon weather/climate here. The solar system is exactly that, a system. Taking one element apart from that whole is inelegant and smacks of agenda.
Your treatment of “cyclomaniacs” (on CA for instance) appears rude, dogmatic, recidivist and haughty. It does not become you to dismiss without further investigation, by yourself, with a software orrery (The Sky from Software Bisque for example), those planets that cannot have gravitic, tidal or barycentric (or any other “alchemical”) effect on sol yet do, most emphatically, have repeating, cyclical, strong and predictable force of some kind with regard to this planet’s variables.
Just because someone had not the wit, skill or equipment to discern those relationships over a century ago, yet had the ear of their peers and the establishment sufficient to publish findings and just because currently “eminent” scientists are too lazy, too frightened or too arrogant to re-investigate a phenomena does not mean that those relationships are not present.
You could have taken a fresh look when the opportunity arose. Many people were saying similar things yet you chose the paths of ridicule and of asking for remarks to be snipped. Yet it is as plain as day that, viewed from the sun, planetary configurations that repeat in a cycle and can be utilised for prediction, are anathema to you. This does not mean that those theories are wrong. In fact, several people make a very good living from long range weather forecasting based upon those very principles and the farmers, market speculators, supermarket managers and holiday firms (to name a few) who rely upon those private forecasts are no fools and observe high success rates or they would drop them like a hot potato. The astroclimatologists may hold proprietary knowledge close to their chests (the withholding of which from society I do not agree with for many more reasons than the rather obvious and delightful one that would put egg on many faces) because of the arduous extraction of signal from noise that they all had to perform, but let us not demean their investigations without commensurate application.
We all appreciate the time you spend educating us in the ways of the solar sleuth. Your reciprocation would be equally well received if you were to investigate, for example, extreme weather events and then plot the orbital bodies’ (all of them) positions and vectors at those times to see the similarities. I do, however, expect you to skirt this issue as you have previously. Probably with snide remarks such as “rev up your orrery” and “angel wings beating”. Prove me wrong.
re Terry Ward.
Hear! Hear! I too think Leif is a covert (and very patronising) AGW supporter.
” Your reciprocation would be equally well received if you were to investigate, for example, extreme weather events and then plot the orbital bodies’ (all of them) positions and vectors at those times to see the similarities. I do, however, expect you to skirt this issue as you have previously. Probably with snide remarks such as “rev up your orrery” and “angel wings beating”. Prove me wrong.”
What did your last slave die of, TW??
He does have a career and a salary to justify, who do you think Lief is? Jim Hansen?
Pattern of Strange Errors Plagues Solar Activity and Terrestrial Climate Data
This load of manure was taken apart at CA a couple of years back, more unabashed duplicity.
Terry Ward (03:47:05) :
I was trolling about your seeming AGW apologism and your overt derision and covert censorship.[…] Given the power to do so, you always get any reply that disturbs your world view snipped. […]
I have been shown something that you have not considered […]
The astroclimatologists may hold proprietary knowledge close to their chests …
First, what has AGW to do with the Sun? Second, I have the best ‘credentials’ in the anti-AGW realm as I have been permanently banned from tamino’s site because of my opposition to AGW.
The planetary theories can be what they are and people can believe what they want. I do not object to that. Only when a specific physical mechanism is brought forward that is wrong [in the sense that it either violates physical laws or is not energetically viable] do I point out the failing of that mechanism. An example is the Angular Momentum idea. Another is electric currents from the planets, or neutron stars at the center of the Sun. My objection is not censorship in any form, to wit the endless postings on these ideas.
My world view in this regard is simply physical science. And all I present [and represent] is just my own view on the viability of the ideas promoted. Typically, a posting sequence goes like this: poster A presents some ‘perfect’ correlation and claims that mechanism B perfect explains what goes on and if just everybody would accept it, the face of physics would be changed forever. I point out why, in my opinion, the mechanism does not work, or the correlation is not so perfect. A gets upset and rebuts. I patiently explains in more detail why I think the mechanism cannot work. A gets more and more upset, and so it goes, until the moderator decides that this has gone on for long enough.
And the ‘secrets’ of planetary influences that ‘you have been shown’ are not science unless they are discussed openly.
Leif Svalgaard,
I have heard about how you have jumped somewhat to the forefront within the solar field but I had no idea about your attiude toward certain things. Like the planetary effects upon space weather. And I use the latter since it describes their influence better.
So am I to assume that you think it is impossible to forecast certain specific events from well out in time ? As in a forecasted time period of days to a week and from months out or even further ?
And I have done the latter and gone on record within the science communty. I look forward to hearing your response.
Jim Hughes (05:59:40) :
So am I to assume that you think it is impossible to forecast certain specific events from well out in time ?
There are two kinds of forecasts:
1) general, statistical forecast. E.g. I’ll predict that next summer will be warmer than last winter outside of the tropics. I’ll predict that space weather at the next solar maximum will be stormier than at this solar minimum.
2) specific, particular forecasts. E.g. that solar cycle 24 maximum will 75, or that a solar flare will occur on May 11, 21:34 UT, 2012.
Some orthogonal to this distinction is the forecast based on physical theory or on statistical correlation.
1) The prediction of SC24 max at Rmax=75 is based on physical theory coupled with the observation that the polar fields are as weak as they are. This is a specific prediction based on current data supported by sound theory.
2) Since statistically weak cycles occur in groups, one can make a statistical prediction that SC25 will be weak too.
Statistical prediction could turn out wrong without invalidating the correlation [it was only statistical, after all]. Physical predictions cannot afford to be wrong, as that will invalidate the theory [or at least show it is incomplete and needs amendment].
On time scales:
Solar cycle predictions can be made on the time scale of one cycle for physical forecasting, but no longer. Since solar activity often has a lifetime exceeding solar rotation period, predicting that activity will recur in 27 days is pretty, but not after 10×27 days. Since the same active region often gives rise to several flares, predicting a heightened flare probability for a few days after the first flare is also a reasonable forecast.
Leif Svalgaard (09:27:10) :
There are quite a few recent cycles that happened quite comfortably on there own in the brief history of our sunspot knowledge, SC12, SC14, SC16 and the infamous SC20. Your logic of SC25 being a low one is scant of any real understanding of what drives the Sun, and is a probability case at best. This is not good enough.
There are lots of cases of more than 2 low cycles in a row in the last 400 years, would you like to predict SC26?, not a chance I suspect….
Geoff Sharp (06:04:26) :
would you like to predict SC26?, not a chance I suspect….
Statistically it would be low too, but with less probability, of course. I don’t have a cultist’s certainty here.
Leif Svalgaard (16:41:03) :
I don’t have a cultist’s certainty here.
Resorting to ad hominem will not help your cause. If you don’t know…. you don’t know.
Leif, Thank you for your response but I still did not get a direct feel for what you believe is possible , forecast wise. As in what is possible to forecast from way out. Example… I forecasted that a Cycle 24 group would show up in the SH on December 10th, or be present quite a while back.
And only one Cycle 24 SH group had been seen prior to this as you might know. And a Cycle 24 group did emerge in the SH on December 11th. I could also give you other similar examples like for late 2006 which was made well out in advance.
And I’ve also been talking about a June increase for the past several weeks with some people within the meteorological community because some of them believe in the space weather forcing connection to the ENSO. And I have said that this will be the largest level of activity since March 2008.
So I want to know is a forecast like this good enough to some extent or do you want to hear specifics like flare strength, sunspot location, and total etc….
And please keep the criteria for my own forecasts on the same playing field level as someone like yourself. When it comes to specific forecast details. But I am also unaware if you make these types. Thanks in advance.
Geoff Sharp (06:26:05) :
Resorting to ad hominem will not help your cause. If you don’t know…. you don’t know.
I think you are the one with a cause. And you don’t know either. Except you claim you know. I take a dim view of such claims.
Jim Hughes (07:24:03) :
I don’t know how long beforehand you made that forecast. If it more than 6 months I would either be impressed or dismiss it as a fluke. If less, that might be possible in rare cases [with not too much other activity – imagine there were 2 new spots every day] as there are precursors to spots: recurrence [spots occur in the same place], intensification of magnetic field, etc. As an example look at the first Figure in http://leif.org/research/Most%20Recent%20IMF,%20SW,%20and%20Solar%20Data.pdf showing the interplanetary magnetic field [blue line marked B]. Note that in the rotation prior to all major flares there was a spike in B. I actually predicted ][a month ahead] the famous Halloween storm in 2003, and the big flares in 2005 and 2006.
Forecasting with too many false positives is useless. If I predict the “Big One” in San Francisco every week for years on end, the population soon tires of the weekly wholesale evacuation of the City.
An interesting paper about solar / weather coupling is at:
http://www.lavoisier.com.au/articles/greenhouse-science/solar-cycles/IanwilsonForum2008.pdf
It connects AMO PDO, SOLAR cycles and the length of day variations.
Not much causality shown, but interesting correlations.
Merrick (13:31:22) :
E. M. Smith,
Go ahead and bash Microsoft all you want, but you’re off on this one.
Thank you I will. But I’m not “off”. I said it was “user hostile” and it is. There are trivial ways it could be made user friendly and they are not done. The Microsoft pattern of behaviour would predict they will leave it hostile for a long time to come. That is just poor human factors design.
Peter simply picked the wrong type of graph. No harm no foul. Blaming Microsoft for Peter’s mistake is just poor analysis –
IFF I had asserted that Microsoft drew the wrong graph, you would be right, but my complaint is about the ease with which MS lets you step on land mines. I’ve spent far too much of my life supporting software (from MANY makers) to accept a bad user interface as “no problem”. It is a problem. It is harm and it is foul. In physical products you get sued if someone chops of fingers because your product snapped shut unexpectedly and there was no warning. In MS land it’s called normal.
something we’re all tired of seeing from the “other side,” so let’s not do it ourselves.
I am a software professional. I’ve written production code for over 20 years. I’ve supported software from most major makers on everything from PC class to supercomputers. I’ve managed software development groups including a compiler tool chain and a user oriented product that got 4 software patents (and that was both the groups writing the software and the software QA group). I think I’m qualified to render a professional opinion of the quality of a user interface.
Now maybe I’m spoiled, since I spend most of my time on one of the best platforms in the business from a UI point of view, but then again, maybe measuring against the best available is exactly how it ought to be done… So no, this is not a ‘poor analysis’ it is a straight forward observation of the limitations of a product and it’s failure to “measure up” to what the competition provides in a specific aspect: It is not user friendly. (And I stated exactly what MS could have done to make it friendly. – a simple “are you sure you want this bahaviour?” prompt. Basically, built in prompting documentation of the often unexpected behaviour.)
So yes, Peter is responsible for picking the wrong graph, but no, Microsoft is not free of blame. They made this user interface such that at least one other poster stated that it is the usual error folks made. “That happens a lot” is one of the key hallmarks of a design fault and it ought to be fixed as quickly as possible. I know, I’ve managed groups that did exactly that kind of repair work. When folks regularly put the car in reverse instead of forward, we add a reverse lockout… for a reason. And product liability law (for everything other than software, it would seem) has clearly stated that the bad UI design is the fault of the manufacturer.