Guest Post by Willis Eschenbach
I came across a curious graph and claim today in a peer-reviewed scientific paper. Here’s the graph relating sunspots and the change in sea level:
And here is the claim about the graph:
Sea level change and solar activity
A stronger effect related to solar cycles is seen in Fig. 2, where the yearly averaged sunspot numbers are plotted together with the yearly change in coastal sea level (Holgate, 2007). The sea level rates are calculated from nine distributed tidal gauges with long records, which were compared with a larger set of data from 177 stations available in the last part of the century. In most of the century the sea level varied in phase with the solar activity, with the Sun leading the ocean, but in the beginning of the century they were in opposite phases, and during SC17 and 19 the sea level increased before the solar activity.
Let me see if I have this straight. At the start of the record, sunspots and sea level moved in opposite directions. Then for most of the time they were in phase. In both those cases, sunspots were leading sea level, suggesting the possibility that sunspots might affect sea level … except in opposite directions at different times. And in addition, in about 20% of the data, the sea level moved first, followed by the sunspots, suggesting the possibility that at times, the sea level might affect the number of sunspots …
Now, when I see a claim like that, after I get done laughing, I look around for some numerical measure of how similar the two series actually are. This is usually the “R2” (R squared) value, which varies from zero (no relationship) to 1 (they always move proportionately). Accompanying this R2 measure there is usually a “p-value”. The p-value measures how likely it is that we’re just seeing random variations. In other words, the p-value is the odds that the outcome has occurred by chance. A p-value of 0.05, for example, means that the odds are one in twenty that it’s a random occurrence.
So … what did the author of the paper put forwards as the R2 and p-value for this relationship?
Sad to relate, that part of the analysis seems to have slipped his mind. He doesn’t give us any guess as to how correlated the two series are, or whether we’re just looking at a random relationship.
So I thought, well, I’ll just get his data and measure the relationship myself. However, despite the journal’s policy requiring public archiving of the data necessary for replication, as is too common these days there was no public data, no code, and not even a Supplementary Online Information.
However, years of messing around with recalcitrant climate scientists has shown me that digitizing data is both fast and easy, so I simply digitized the graph of the data so I could analyze it. It’s quite accurate when done carefully.
And what did I find? Well, the R2 between sunspots and sea level is a mere 0.13, very little relationship. And even worse, the p-value of the relationship is 0.08 … sorry, no cigar. There is no statistically significant relationship between the two. In part this is because both datasets are so highly auto-correlated (~0.8 for both), and in part it’s because … well, it’s because as near as we can tell, sunspots [or whatever sunspots are a proxy for] don’t affect the sea level.
My conclusions from this, in no particular order, are:
• If this is the author’s “stronger effect related to solar cycles”, I’m not gonna worry about his weaker effect.
• This is not science in any sense of the word. There is no data. There is no code. There is no mathematical analysis of any kind, just bald assertions of a “stronger” relationship.
• Seems to me the idea that sunspots rule sea level would be pretty much scuttled by sunspot cycles 17 and 19 where the sea level moves first and sunspots follow … as well as by the phase reversal in the early data. At a minimum, you’d have to explain those large anomalies to make the case for a relationship. However, the author makes no effort to do so.
• The reviewers, as is far too often the case these days, were asleep at the switch. This study needs serious revision and buttressing to meet even the most minimal scientific standards.
• The editor bears responsibility as well, because the study is not replicable without the data as used, and the editor has not required the author to archive the data.
So … why am I bothering with a case of pseudo-science that is so easy to refute?
Because it is one of the papers in the Special Issue of the Copernicus journal, Pattern Recognition in Physics … and by no means the worst of the lot. There has been much disturbance in the farce lately regarding the journal being shut down, with many people saying that it was closed for political reasons. And perhaps that is the case.
However, if I ran Copernicus, I would have shut the journal down myself, but not for political reasons. I’d have closed it as soon as possible, for both scientific and business reasons.
I’d have shut it for scientific reasons because as we see in this example, peer-review was absent, the editorial actions were laughable, the authors reviewed each others papers, and the result was lots of handwaving and very little science.
And I’d have shut it for business reasons because Copernicus, as a publisher of scientific journals, cannot afford to become known as a place where reviewers don’t review and editors don’t edit. It would make them the laughing stock of the journal world, and being the butt of that kind of joke is something that no journal publisher can survive.
To me, it’s a huge tragedy, for two reasons. One is that I and other skeptical researchers get tarred with the same brush. The media commentary never says “a bunch of fringe pseudo-scientists” brought the journal down. No, it’s “climate skeptics” who get the blame, with no distinctions made despite the fact that we’ve falsified some of the claims of the Special Issue authors here on WUWT.
The other reason it’s a tragedy is that they were offered an unparalleled opportunity, the control of special issue of a reputable journal. I would give much to have the chance that they had. And they simply threw that away with nepotistic reviewing, inept editorship, wildly overblown claims, and a wholesale lack of science.
It’s a tragedy because you can be sure that if I, or many other skeptical researchers, got the chance to shape such a special issue, we wouldn’t give the publisher any reason to be unhappy with the quality of the peer-review, the strength of the editorship, or the scientific quality of the papers. The Copernicus folks might not like the conclusions, but they would be well researched, cited, and supported, with all data and code made public.
Ah, well … sic transit gloria monday, it’s already tuesday, and the struggle continues …
w.
PS—Based on … well, I’m not exactly sure what he’s basing it on, but the author says in the abstract:
The recent global warming may be interpreted as a rising branch of a millennium cycle, identified in ice cores and sediments and also recorded in history. This cycle peaks in the second half of this century, and then a 500 yr cooling trend will start.
Glad that’s settled. I was concerned about the next half millennium … you see what I mean about the absence of science in the Special Edition.
PPS—The usual request. I can defend my own words. I can’t defend your interpretation of my words. If you disagree with something I or anyone has written, please quote the exact words that you object to, and then tell us your objections. It prevents a host of misunderstandings, and it makes it clear just what you think is wrong, and why.
“Well, the R2 between sunspots and sea level is a mere 0.13, very little relationship. And even worse, the p-value of the relationship is 0.08 … sorry, no cigar. ”
On what years are you getting those values because from the paper they have a bunch of different cycles pointed out. I’m assuming your getting those values from the whole data set which both you and the paper point out starts with the data in question as an opposing force. Thus pointing out again through these values makes no sense since the paper clearly states as such. I only briefly skimmed the paper since its late but I can I can already see some problems with this whole line here assuming your using the whole data set.
Thanks Willis,
I know you are talking about much more than this one paper, and I see that the correlations are poor, but seeing both of those lines having about the same frequency just makes me wonder what is going on! Why would the frequency of the two signals be about the same? What causes the signal in the sea level? Could there be a loose coupling between the two, like something poking at a pendulum? Why would you expect them to be perfectly in lock-step? Perhaps the sun is a factor, but there are multiple other cyclical drivers of sea level change? The sea level looks like the path of a drunk with several people (one visible, others unknown) trying to push him in the right direction. OK, maybe too much coffee…
Willis,
“And I’d have shut it for business reasons because Copernicus, as a publisher of scientific journals, cannot afford to become known as a place where reviewers don’t review and editors don’t edit. It would make them the laughing stock of the journal world, and being the butt of that kind of joke is something that no journal publisher can survive.”
You mean like Nature and the rest of the Believers Climate papers??
HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA
OK, that is one paper. Gonna do the rest for them??
Willis,
The next paragraph says:
The author appears to have used Fig 2 to illustrate the de-coupling between solar cativity and sea levels; specifically noted the phase inconsistency and then reasoned that oceanic thermal mass is the reason for phase shifting and a reduction in response for “weak” cycles.
I can’t see how you got the idea from the paper that the author was trying to indicate a simple relationship between solar activity and sea level. Admittedly, I haven’t read the paper in as much depth as you must have before draughting your article.
By the way Willis, by picking a weak paper to pick on you leave the implication that they are ALL weak. is this your intent or are you going to do the rest of the papers??
It’s called spurrious relationship/correlation geeses and babies etc////
Sir James Jeans of the Royal Society posted a similar chart for Sir Richard Gregory in his book “Through Space and Time”. The chart shows the water line of Lake Victoria perfectly correlation to Sunspot Activity.
I was allowed by the publisher to repost that work in my papers along with Atlantic Ocean hurricane seasons Accumulated Cyclone Energy. You can do the same with Earthquake data.
Maybe with for different correlations along with temperatures on our sister planets maybe the EPA will toss the Hockey Stick and Puck back to Ice and quit their false science.
My work is posted at sunspotshurricanesandglaiers.com. Top section.
Sincerely,
Paul Pierett
I would like to thank professor Jan-Erik Solheim, a specialist in astrophysics, for an interesting article.
Please read the article with an open mind.
Regards
Agust
kuhnkat says:
January 21, 2014 at 11:08 pm
By the way Willis, by picking a weak paper to pick on you leave the implication that they are ALL weak. is this your intent or are you going to do the rest of the papers??
##########################################
Why should willis do what reviewers should have done.
In the first 3 papers I read ( this was one) I found similar claims of “relationship’ or correlation, but no numbers.
You CANNOT replicate a find of “good correlation’ unless the author actually tells you what he found.
After reading 3 papers I gave up.
Ok, Team WUWT, united really only by shared insomnia, go!
So Willis, Mosh, if all, or almost all of these papers similar in quality, would a better name for PRiP be CitC, (Castles in the Clouds)?
Maybe this could be a paper in the Journal?
Okay! And YOU GO, CHARLES THE MODERATOR! #(:))
btw: no insomnia — just lack of self-discipline to get to bed.
(hope you don’t mind this totally OT post to just say, “Hi!”)
(yawn)…. bowl of cereal time…. then….. zzzzzzzzzzzz (after I kick my German Shepherd off the bed where he is now snoring away, heh)
LOLOLOL — it’s fun hanging out on WUWT after midnight!
I think you have completely misconstrued the argument into a strawman. Sunspots are an effect, not a cause. No one says ‘Sunspots caused this anomoly’. They are a just a proxy for sun’s magnetic activity and nothing else. And whatever magnetic event that might be affecting earth’s climate could easily precede the change in sunspots. It is absurd to try to disprove the paper based on a claim the authors weren’t even making.
The Journal is about pattern recognition.
A pattern is being discussed in the paper.
The conclusion to be derived from any pattern and the question as to whether or not there is a real pattern are obviously open for reasonable discussion.
In this case the pattern appears to hold for a time but to shift in and out of phase.
Many things can cause real patterns to do that and the proper way forward is to investigate whether the pattern may or may not be real in the light of all the potentially confounding influences.
Seems a reasonable scientific endeavour to me.
Note that the author actually draws attention to the problems in the analysis as a precursor to further investigation.
The author makes some attempt to suggest a possible explanation in the millennial solar cycle and oceanic thermal inertia.
That is a more scientifically valid approach than most of AGW theory which appears to try and hide or adjust inconvenient data.
I think Willis is being unfair here.
Janice Moore says:
January 22, 2014 at 12:42 am
btw: no insomnia — just lack of self-discipline to get to bed.
Similar here, Janice. I could easily keep my thoughts here for the rest of my life.
I raised a G Shep Akita mix for 18 years and 3 generations. I still see them in my dreams at times. They were beautiful creatures.
Willis,s argument about taking care to maintain the proper level of integrity in published research makes sense to me.
The graph does have interesting features. Could it be that the main story this graph is telling is that there is another modifier interacting with these systems?
No, Willis is not being unfair. Willis is pointing out that if you’re going to do peer-reviewed scientific analysis, then you’d better make sure that your peers are competent enough to critique the analysis.
If Willis can spot the obvious flaws in a paper with five seconds of inspection and a quick calculation, then its clear that the journal (and therefore the publisher) is heading for a pasting that is thoroughly deserved.
Pattern recognition is at the basis of human intelligence, but it is also the greatest threat to human progress if it is not constrained by proper analysis.
Willis,
Notwithstanding your gripes about the overall paper and the journal review process, If someone placed the curves in front of me, I;d be very interested i knowing why they are related. They are related without a doubt. So why?
Is the sea level data repeatable? The sunspot data looks right.
I’d say there is something there.
Agree; this comes across as that preliminary pre-hypothesis stage called “speculation”. It’s not trying to claim to do more than point out something curious, suggestive. Demanding it meet inappropriate standards is churlish.
Willis,
“The p-value measures how likely it is that we’re just seeing random variations. In other words, the p-value is the odds that the outcome has occurred by chance. A p-value of 0.05, for example, means that the odds are one in twenty that it’s a random occurrence.”
This is not correct. The pvalue is the probability of observing the given result or one more extreme, given that the null hypothesis is true. If you don’t understand the difference between what I have written and what you have said, you shouldn’t be attempting statistics. Period.
“And what did I find? Well, the R2 between sunspots and sea level is a mere 0.13, very little relationship. And even worse, the p-value of the relationship is 0.08 … sorry, no cigar. There is no statistically significant relationship between the two. In part this is because both datasets are so highly auto-correlated (~0.8 for both), and in part it’s because … well, it’s because as near as we can tell, sunspots don’t affect the sea level.”
So let me get this straight- you find very little relationship between the two, and yet you say that the two datasets are highly auto-correlated!!! What do you think autocorrelated means Willis??? I mean look at the two lines Willis- they are practically on top of one another!!! Maybe you didnt find any relationship between the two because you know squat about statistics?
This is why it is an utter waste of time reading blogs like these written by poorly trained pseudo-scientists. The peer review process is there so that crap like this doesnt see the light of day.
What an embarassment.
Steve Mosher: “Why should willis do what reviewers should have done.”
What? You mean READ the paper before trying to diss it? Don’t be silly.
Willis just picks out one graph and ignores the rest of the paper and then goes on to draw his own preconceived conclusions that have little “correlation” with the contents of the paper.
The rest of the paper shows a much deeper examination of the spectral content including a strong 9.1 year cycle that Scafetta, BEST and others have already documented.
Now if you superimpose a 9.1 year and a rather variable 10-11 year cycle you will get exactly the kind of phase shifts and decorrelation that Willis’ sharp eye spots in figure 2. if you only look at either one in isolation.
This is precisely the point I made in an article I submitted to Talkshop last year just before getting banned for daring to question with His Majesty.
My copy here:
http://climategrog.wordpress.com/2013/03/01/61/
Willis, if your are going to dismiss a paper at least have the good manners to read it first.
well, it’s because as near as we can tell, sunspots don’t affect the sea level.
I don’t offer any judgement on the worthiness of this paper, but this remark jumped out at me. Surely you are not suggesting that the authors think sunspots affect sea level? Maybe it ought to have been more explicit in the paper, I don’t know I haven’t read it, but the first thing I thought of looking at the graph you displayed is that they are using sunspots as an indicator of solar activity the consequence of which may affect temperatures which may affect sea level. Sunspots have been pretty reliably measured which is why I presume this graph was made.
My first thought to this was I didn’t think sea levels would be affected on these time scales because of thermal inertia, but maybe it has something to do with the top layer or something. But sea levels are notoriously difficult to measure reliably and I wondered about the level of error – i suspect they would be so large that any sort of wiggle would fit within them.
I think your skepticism is well-merited – the danger of “pattern recognition” is that the human mind contrives to find patterns where none exist, and this looks like a prime candidate. Yet if you are going to kick this one down the hall, at least don’t mischaracterise what they are saying. Or DID they say that sunspots cause sea level change explicitly? Because that would be nuts…
“if you’re going to do peer-reviewed scientific analysis, then you’d better make sure that your peers are competent enough to critique the analysis.”
And if the analysis throws up new areas for investigation such that no peers are sufficiently competent ?
This paper does not purport to ‘prove’ anything. The criticisms are indeed churlish.
John A : “If Willis can spot the obvious flaws in a paper with five seconds of inspection and a quick calculation, then its clear that the journal (and therefore the publisher) is heading for a pasting that is thoroughly deserved.
Pattern recognition is at the basis of human intelligence, but it is also the greatest threat to human progress if it is not constrained by proper analysis.”
So rather then read the paper and find out that it does contain a detailed “proper analysis” , you just pile on, on the basis of Willis’ inconvenienced comments, which do indeed seem to be based on ” five seconds of inspection “.
Mindlessly repeating what you read in a blog post , without checking the facts, is “the greatest threat to human progress”.
Now go away, read the paper then come back and tell what you think.
kuhnkat says:
January 21, 2014 at 11:08 pm
Am I going to do them all? No, my stomach is not that strong.
Are they ALL weak? I don’t know. I’ve only looked at four of them, and they were all depressingly bad.
w.
Paul Westhaver says:
January 22, 2014 at 2:05 am
No, they are not “related without a doubt”. What part of an R^2 of 0.13 and no statistically significant correlation seems unclear?
However, you illustrate a problem, one that is in the title of the journal. Humans are pattern recognition machines … to the point where we often see a pattern where none exists. That’s why we have math … and it’s why a paper which makes no attempt to do the math is handwaving rather than science.
w.
Brian H says:
January 22, 2014 at 2:14 am
Since when is requiring a calculation of correlation for a claimed relationship “churlish”? That’s nonsense, it’s a standard scientific requirement to make such a calculation before publication of such a claim.
w.
Hi Willis
First of all I am recognised as a ‘sceptic pseudo-scientist’, although I give primate to my engineering degree to the higher science degree and often say ‘I am an engineer not a scientist’.
Of the ‘oscillating’ things going in and out of phase:
few years ago I looked at the AMO (9yr) and SSN (11yr), a perfectly normal thing:
http://www.vukcevic.talktalk.net/AMO-SSN.htm
Hey, ‘pattern recognition’ not Copernicus but ‘vuk’ style had me occasionally speculate: is there anything more to it, and speculation lead to a highly speculative outcome as registered here , where everything fell neatly into phase.
Sometimes in the remotely distant future we might exchange my speculation and your knowledge and experience of the amazing Solomon Islands.
Agnostic says:
January 22, 2014 at 2:35 am
Sorry for the lack of clarity. The sunspots are on the sun. The ocean is on the earth. Obviously they think that something for which sunspots are a marker is affecting the sea level, which I shortened to sunspots are affecting the sea level.
However, their own chart makes that solar effect on sea level very unlikely, no matter what terms you use to describe it. Sometimes the sun moves first, sometimes the sea moves first, sometimes the sun and sea level move in parallel, sometimes in opposition. The correlation is bad, R^2 = 0.13, p=0.08. In other words, whatever energies sunspots might be a proxy for is not affecting the sea level.
w.
Whilst sunspots are an easy thing to count sea level is not. In fact sea level can change in different ocean basins in different ways so we need to know how this metric was arrived at.
While I agree with Greg Goodmans points to Willis I’ll just say that like Willis, he wasn’t banned from the talkshop for his science.
Goodman teaches Eschenbach manners – more popcorn please.
Siberian_Husky says:
This is why it is an utter waste of time reading blogs like these written by poorly trained pseudo-scientists. The peer review process is there so that crap like this doesnt see the light of day. What an embarassment.
Who are you to judge? You are just trolling, anonymously. Post your name, and readers will see if you have the credentials to pass judgement like that.
You would love it if you could censor articles like this. That’s what you wrote here, isn’t it? FYI, this site does the job that your pal reviewed ‘science’ shirks.
I’m a huge Willis fan, but on this issue I’m a bit – shall we say – sceptical.
The first problem is obvious: by simply eyeballing the graph you can see a striking correlation. Of course, the eye can be easily fooled. One test would be to create a set of similar but random graphs and to estimate what proportion seem to have the same correlation.
The sea level graph has 10 positive peaks. The sun spots very accurately match 5 of the peaks, particularly in term of phase (date of the peak). Several other peaks match fairly well. Strangely, the two worst phase matches are at the start and end of the graph. Coincidence?
There is a dramatic phase mismatch right at the start of the graph. There are two possible explanations for these mismatches:
1. Sunspot data should be accurate (easy to measure) but sea level data is much harder. My guess is that, if Holgate had used a different (but still good quality set), his graph would have been similar overall, but in detail somewhat different. Remember that Holgate’s data set was quite small. A different data set might well change the phase matches.
2. The thing (if it exists e.g. solar wind) that affected the sea level is probably complex. And the relationship between that thing and sunspots is also probably complex.
I think R2 tests may be fairly misleading in some cases. Any test has to do what the human eyeball is very good at: noticing correlations between specific features rather than an overall trend.
As I said, some kind of Monte Carlo test could possibly prove whether the apparent correlation is real or just a trick of random chance: create a set of similar randomly generated graphs and ask this question: what proportion of randomly generated graphs appear to have the same correlation?
Ironically, true believers would kill to be able to show a graph with similar correlation between CO2 and global temperature (obviously, with CO2 leading and not lagging the temperature).
Finally, there is an abundance of evidence that solar activity has a dramatic effect on river flows and lake levels. If solar activity had literally a zero effect on sea levels, that would be quite surprising.
Chris
tallbloke says:
“While I agree with Greg Goodmans points to Willis I’ll just say that like Willis, he wasn’t banned from the talkshop for his science”.
No. I was banned because you are unable to accept criticism and have a tantrum if anyone disagrees with you on your blog. You control your blog space like a petulant child and don’t apply your own blog rules.
Rule (1) There are no rules.
Rule (2) See rule (1)
Rule (3) See rule (2)
http://climategrog.wordpress.com/2013/03/05/talkshop-immoderation/
I would have thought you would have the sense to keep schtum about that outside of a space where you have admin rights. You end up looking about as objective as Grant “Tamino” Forster and Real Climate crew.
@johnmarshall
Whilst sunspots are an easy thing to count sea level is not. In fact sea level can change in different ocean basins in different ways so we need to know how this metric was arrived at.
Agreed – this is the reason I am skeptical of drawing a link between sunspots as a metric for solar activity and sea level rise. I don’t think it is necessary to go as far as an R^2 – I’d have questioned the sea level data. And R^2 of harmonically related trends don’t automatically discount a relationship. If you were to describe π in binary you wouldn’t get a statistically significant result but it doesn’t mean that it wasn’t meaningful.
I think the reasoning behind the graph is reasonable, but I doubt there is sufficient accuracy in the sea level data to draw much in the way of conclusion from it.
I don’t think the paper is particularly strong , it is mainly recapitulation of other work (mainly Scafetta it would seem), however, I have not seen this particular tide series before.
Some of it seems of dubious value, eg:
“The GISP2 may have a timing error of decades and/or
show temperatures out of phase with the global temperature
variation. In Fig. 9 we compare the simulation determined
from the GISP2 data with the HadCRUT4 global tempera-
ture series, and find a good fit if we introduce a shift of 85 yr,
which means the response in the ice core as shown in Fig. 8
is delayed 85 yr compared with the instrumental temperature
record. ”
One half wiggle looks similar to another one if we shift it by an arbitrary 85 years. Hmm.
Comparing frequency spectra in fig. 3a and 3b:
“The dominance of a 22 yr period compared with a 10–12 yr
period can be explained by GCR variations. The 22 yr Hale
period is the Sun’s magnetic period, and represents a polarity
change in the two hemispheres of the Sun. This is observed in
the GCR variations as shown in Fig. 5. ”
This is more like it. What is measured in SSN is a “rectified” version of solar variations.
Climate is very complex and it is not possible to just overlay two time series or do trivial linear regressions. No understanding will be gained by such trivial analysis, nor can the presence of a signal be refuted over simplified correlation tests and statistical significance.
This is the main problem with current climate science. There is a resounding need to apply the wealth of existing knowledge in systems analysis to climate system.
It seems these papers suffered from the rather incestuous circle of authors, editors and reviewers and could have been strengthened by applying the publishers rules. There is plenty of expertise in relevant techniques that can be called on, even if its application to climate is lacking.
@Vukcevic:
Nice graph. Nine years is one of the lunar cycyles. I would look into lunar tidal cycles and orbital resonance periods with the gas giant planets for the “connection”. Both for your work and for the interesting graph in the posting.
@Willis:
Your analysis is statistically sound for two matched signals, but will fail on two resonant signals of common origin but different harmonics… A 3:2 frequncy ratio will give shifting phase, yet is a relationship. It speaks to something deeper to find.
Is the article trash? Depends on the authors’ claims in the text, not just the stats on the graph. Taking your word for it that strong correlation was claimed without the math: if so, that is very weak. Ought to have shown the math basis for the claim. IMHO, and also shown the correlation stats for harmonic relations that do match, if any.
But in any case, to look at tides and NOT look at orbital periods of the moon and sun is a big weakness. There are 3, 9, and 18 year lunar orbital cycles, and more, and an 11 year Jovian jiggle. Ignoring them in a lunar solar cycle paper is silly.
Is this just coincidence?
–
>NASA Finds Sun-Climate Connection in Old Nile RecordsIs solar variability reflected in the Nile River?<
Alexander Ruzmaikin, Joan Feynman, and Yuk L. Yung
JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 111, D21114, doi:10.1029/2006JD007462, 2006
http://trs-new.jpl.nasa.gov/dspace/bitstream/2014/39770/1/06-1256.pdf
http://onlinelibrary.wiley.com/doi/10.1029/2006JD007462/abstract
http://trs-new.jpl.nasa.gov/dspace/handle/2014/40231
–
By the way, Joan Feynman is Richard's Feynman sister.
http://en.wikipedia.org/wiki/Joan_Feynman
NASA Finds Sun-Climate Connection in Old Nile Records
http://www.jpl.nasa.gov/news/news.php?feature=1319
Hi Chiefio, this look at lunar cycles in Arctic ice data may interest you.
http://climategrog.wordpress.com/?attachment_id=756
It’s also a good example of the detailed analysis needed to detect signals in spectral data.
Many things in climate lead to one effect modulating another, this leads to splitting of peaks into triplets where the central value may be nearly non existent. Failure to understand how this shows up in spectral data will lead the false conclusion that a particular frequency is not present.
I’d also been surprised by the lack of a clear 29.5 day signal in this data until recently though there was a broad spread in that part of the spectrum there was nothing relating to the usual figure of 29.53 days as synodic lunar period.
I recently discovered (thanks to a link of cheifio’s site) that there is quite a spread in length of the visible lunar cycle. It turned out that the extreme values are clearly present whereas the average is not.
None of this is simple and trivial analysis is not sufficient to either detect or refute the presence of natural cycles.
Similar processing finds a notable anomalistic month signal (distance of moon) . Equally looking for the signal directly you would conclude there is nothing at 27.5545d , however, the combined energy of the triplet makes it clearly significant against the background noise.
http://climategrog.wordpress.com/?attachment_id=757
I also have to say hats off to NOAA for the quality of this data. It really says something for the signal to noise ratio when you can pull out this kind of detail.
Well done again, sir! It never ceases to fascinate me to think how easy it is to fool oneself.
@Siberian_Husky
You should do a little research before you imply that someone is a fool. Especially when your accusation exposes your glaring lack of understanding of simple math terms. Autocorrelation means that a signal correlates to itself (auto = self, correlation = the state of being correlated) . The two datasets are indeed highly autocorrelated since they are not only cyclical, but also quite periodic. Here is the Wikipedia article regarding autocorrelation.
The Merriam-Webster definition of autocorrelation is:
In conclusion, it would appear that autocorrelation does not mean what you seem to think it means.
A recurring problem with “open review” is when the reviewers reach beyond their own grasp (and refuse to admit it). See figure 1 — when there is coupling, linear correlation is for political distortion artists:
http://www.uvm.edu/~cdanfort/csc-reading-group/sugihara-causality-science-2012.pdf
http://climategrog.wordpress.com/?attachment_id=755
The “9.1” year peak found by N Scafetta (as referenced in the Solheim paper) and in the recent BEST land temp study is also found in cross-correlation of N. Pacific and N. Atlantic SST.
Closer inspection reveals that it is likely to be 9.05 , the result of interference between lunar declination by lunar perigee (tidal force) .
The super-position of the two close cycles results in 9.05 year cycle modulated by 356 years. Only the 9.05 is readily detectable in available records due to limited data length. This kind of analysis can reveal longer periods.
Data disclosure and personal statement on peer review by Professor Jan-Erik Solheim:
In my paper “The sunspot cycle length – modulated by planets?” the section 2. Data and methods, contain links to the data:
http://www.ngdc.noaa.gov/stp/space-weather/solar-data/solar-indices/sunspot-numbers/cycle-data/table_cycle-dates_maximum-minimum.txt.
(here I made contact with the person responsible – who promised to keep this address alive)
and to the program package I use for analysis:
http://www.astro.univie.ac.at/dsn/dsn/Period04/.
This is a thrusted period search code used for variable stars – based on Discrete Fourier Analysis
For my second paper “Signals from the planets, via the Sun to the Earth” the same period search code is used
Regarding peer review:
I have done most of my research in the field of close binary stars and pulsating white dwarfs. Our communities are quite small, so we prefer to be anonymous in our referee work. I think this is the best practice.
Normally there is only one referee in Astronomy and Astrophysics publications. Even if I had my last publication in that field in 2010, I still occationally referee for Monthly Notices of the Royal Astronomical Society and The Astrophysical Journal.
The last years I have had a number of referee tasks for Climate research journals.
Regards
Jan-Erik
[snip – you are welcome to resumbit without the ad homs – mod]
To the Moderator, Which ad homs are you talking about?
I am resubmitting with some editing.
Willis criticizes without reading a paper first.
The same graph discussed above by Willis was discussed with Anthony’s great approval here:
http://wattsupwiththat.com/2009/04/15/the-oceans-as-a-calorimeter/
and here
http://wattsupwiththat.com/2009/04/07/archibald-on-sea-level-rise-and-solar-cycles/
The graph is essentially taken from Shaviv (2008):
Shaviv, N. J.: Using the oceans as a calorimeter to quantify the solar radiative forcing,
J. Geophys. Res., 113, A11101, doi:1029/2007JA012989, 2008.
which has been properly referenced by Solheim and where a lot of details are present. Solheim does not need to repeat the details given the fact that those are in the referenced work.
The uncertainty noted by some above is also discussed in Solheim’s paper
http://www.pattern-recogn-phys.net/1/177/2013/prp-1-177-2013.pdf
and it is due to the fact that the decadal climatic cycle may be due to a soli-lunar oscillation at about 9.1 year and to the 10-12 year solar cycle. This is clearly show in numerous figures of the paper such as figure 7 which is taken from one of my papers, which are these
Scafetta, N.: Solar and Planetary Oscillation control on climate change hind-cast, forecast and an comparison with the CMIP5 GCMs, Energ. Environ., 42, 455–496, 2013a.
Scafetta, N.: Discussion on climate oscillations: CMIP5 general circulation models versus a semi-empirical harmonic model based on astronomical cycles, Earth-Sci. Rev., 126, 321–357, 2013b.
Willis, unfortunately, does not seem to me to know how to read a scientific paper.
Willis, sounds like you are arguing the “a trend is not a prediction” story. The article is pointing out a trend, from which one can reasonably conclude that somewhere in the background there might be at least a co-determinant.
Kicking a dog when its down doesn’t make you a great hunter.
Nicola Scafetta says:
January 22, 2014 at 7:43 am
[snip – you are welcome to resumbit without the ad homs – mod]
Yes Nicola, behave yourself on his Nibs thread. Here’s an example of the sort of thing you can’t say:
“Copernicus, as a publisher of scientific journals, cannot afford to become known as a place where reviewers don’t review and editors don’t edit”
REPLY: That’s not ad ad hom, it isn’t directed at a specific person. You really don’t have much integrity to stand on here Mr. Tattersall, since you don’t allow Willis to comment at your own blog. And then there’s that other incident where you violated my trust. – Anthony
Anthony, by the way,
have you read Giovanni de Gregori letter?
http://tallbloke.wordpress.com/2014/01/22/giovanni-p-gregori-scientific-freedom-paradigms-and-dogma/
“A p-value of 0.05, for example, means that the odds are one in twenty that it’s a random occurrence.”
no, that is the “p-value fallacy”, the p-value is the probability of observing a result as extreme as that observed IF it ocurred due to random chance. This is not the same thing as the probability that it ocurred DUE to random chance.
REPLY: That’s not ad ad hom, it isn’t directed at a specific person. You really don’t have much integrity to stand on her Mr. Tattersall, since you don’t allow Willis to comment at your own blog. And then there’s that other incident where you violated my trust. – Anthony
That violation of trust was over Willis’ ad hom attacks too, remember?
Here’s Niklas Morner’s statement on the way we conducted peer review:
(2) “the editors selected the referees on a nepotistic basis”
Nepotism is to favor friends and relatives without respects to qualifications. We did the opposite; the reviewer chosen were all specialists on the topics in question.
It is true that they primarily were chosen among the authors of the special issue with some additional from outside. This does not mean “pal-reviewing”, but serious colleague reviewing. Most members of the author-team only new each other superficially or as authors. It is common practice when printing proceedings or collective volumes to seek the reviewers within the group, not in order to make the reviewing process less serious, but because those persons are the true experts within the field.
And almost always they do a tremendously good job to improve the papers in constructive ways. So also in our case: our reviewing was simply excellent, which I am sure all persons involved would happily testify. This includes strong points and forces for relevant changes and updating.
And what we achieved was a wonderful collection of papers that together make a very strong impact of elevating an old hypothesis into a firm theory saying that the solar variability is, indeed, driven by the planetary beat.
Amen to that Niklas, you did 100 hours a week for several months to make this happen, and I’m with you all the way. The 580+ peer reviewed papers you have to your name in the geophysics and oceanology fields enabled you to get some superb outside reviewers such as the very eminent Physicist Prof. Giovanni P. Gregori, who has written a brilliant letter to Martin Rasmussen. Anthony should read it.
http://tallbloke.wordpress.com/2014/01/22/giovanni-p-gregori-scientific-freedom-paradigms-and-dogma/
REPLY: So because you hate what Willis has to say, along with the maths to back it up (you’ve presented none), it was OK to violate my trust? Interesting. – Anthony
Dear Anthony,
a famous physicist wrote me after having read your recent posts. He said this about you
******
As for Anthony Watts, this reminds me of the old advice: when you circle the wagons to fend off an attack by wild Indians, direct your fire outward, not inward.
******
Don’t you think that would be better for you to close this post by Willis (it has been fully disproved by me above), and open a new post where you reproduce Giovanni de Gregori letter?
You may also make two posts by coping and past my articles on notrikzone:
(1) http://notrickszone.com/2014/01/21/lynch-mob-science-scafetta-accuses-publisher-of-skipping-trial-conducting-academic-lynching/
(2) http://notrickszone.com/2014/01/19/scientists-react-sharply-to-copernicus-publishing-censorship-of-alternative-scientific-explanations-do-you-realize-what-you-have-done/
Should add, it is better to perform a statistical test and not describe it completely accurately than not to perform the test at all! It is a common error even made by scientists on a regular basis.
Stephen Wilde says:
January 22, 2014 at 1:34 am
The Journal is about pattern recognition.
A pattern is being discussed in the paper.
=========================================
That is not what is meant by Pattern Recognition in science, to get a good idea of what it ought to be about, see the content of the journal “Pattern Recognition” http://www.journals.elsevier.com/pattern-recognition/
REPLY: So the Copernicus “Pattern Recognition in Physics” was nothing more than a ripoff of the Elsevier journal, right down to the orange cover?
– Anthony
Anthony, I think that may be another spurious correlation! ;o)
It is a pity really as PRIP was a really good idea for a journal, there are lots of good applications for pattern recognition in phsyics and the sciences, if it was the sort of pattern recognition methods published in Patter Recognition (which is a very good journal).
@ Dikran Like you, I think the idea of such a journal was a good one, though the titles and covers are a bit too close for copyright comfort. Unfortunately, the people that ran the special edition bollixed the opportunity handed to them in the most self destructive way possible. It was a classic own-goal. As I pointed out previously Copernicus made equally bad errors. They should have spotted and fixed the problem before it exploded. – Anthony
Siberian_Husky says:
January 22, 2014 at 2:29 am
Ummm … I think “autocorrelated” means correlated to a lagged version of the data itself, but it appears you don’t have a clue what it means. Medice, cura te ipsum.
w.
Chris Wright says:
January 22, 2014 at 4:03 am
Dear God, would all of you folks making this or a similar claim please do the freakin’ math!
The human eyeball is famous for finding correlation where there is none at all. We invented statistics in part to get measurements of correlation, and you’re all “No, don’t bother me with all that nasty number stuff, I’m just going to squint at the graph and make my decision”.
DO YOUR HOMEWORK, DO THE MATH!
w.
Yes, I agree with much of that. The cover is probably just a coincidence and I don’t think the name is to big a deal either, it is quite common to see a journal titled X and then another International Journal of X and then a Journal of Applied X etc. It is difficult to find a good name for a journal after a while. Journals that have the most obvious short name for the topic are ususally good ones as it is an indication that they got there first and have survived for a while!
Also these days we mostly access the journals electronically and rarely see the cover, I’d forgotten PRs was even orange!
E.M.Smith says:
January 22, 2014 at 4:59 am
…..
Hi E.M
Thanks. The graph is based on the interaction of two magnetic fields. The Earth’s field apparently is not immune to tidal forces acting on the liquid core where it is generated. Andy Jackson, who is a world recognised authority on the mater, has publicly acknowledged (in a speech to some august science organisation) that they have found frequencies that are coincidental with certain E-M orbital numbers.
There is nothing gained, other than disappointment, concerning this affair. An extraordinary opportunity has been fumbled. Hindsight and finger pointing after the fact… brings no relief.
Personally, I always try to fail safe: If anything must be hidden, or secret… can that thing be good? The answer to that question, has kept me mostly out of trouble. GK
Anthony
Thats not the first time Rog has violated trust. Recall gavin’s letter and Lisbon.
Wilde
‘This paper does not purport to ‘prove’ anything.”
then it’s not even wrong.
Novels and poems dont purport to prove anything either.
Agust Bjarnason says:
January 22, 2014 at 5:11 am
The authors (not NASA but the authors) find cycles in the Nile high- and low-water records at NEAR 88 and 200 years. Here is their description of the significance of their finding:
Since the Nile is where Hurst-Kolmogorov dependence was discovered, and the Nile is justly famous for being the home of “strongly correlated fractional noise”, and their results are only significant at the one-sigma level when tested against that noise, this means that none of their results are statistically significant as the term is used in climate science.
In climate science, results need to be significant at the two-sigma level. It’s a weak standard, it means one result in twenty will come up by chance, it means that if you look at a dozen datasets you have a 50/50 chance of getting a “significant” result by chance, but two sigma is the standard … and their results are only significant at one sigma, which means not significant in any sense.
In addition, take a look at their Figure 3, right panel. The “peak around 260 years in the low water level” they discuss actually peaks at 303 years, and the “near 88 year periodicity in both records” they discuss at length actually peaks at 102 years in the low-water record. Since when is 102 years a “near 88 year periodicity”??
I always get nervous when I see this kind of fudging, claiming “near” for anything that vaguely resembles some astronomical cycle or other … overall, I’d give the paper low marks.
Definitely an interesting read, though, thanks.
w.
It is interesting that the two cycles appear to have similar frequencies…but a simple relation is agreed to not fit due to phase shifts. Think of an electrical analog — You can have a sinusoidal voltage source driving current in a resistor, and the measured current would be in phase. You could parallel that resistor with a capacitor, and as you varied the value of the capacitor, you would phase shift the current w.r.t. the drive. Pretty simple differential equations. Clearly, if there is a correlation & causation by solar forcing, than a complete model has to look into what other effects might be varying the energy storage. PDO? Something else? This is an intriguing pattern, but there is a lot to figure out here.
vukcevic says:
January 22, 2014 at 9:52 am
Thanks. The graph is based on the interaction of two magnetic fields.
No, just poorly done curve-fitting.
Nicola Scafetta says:
January 22, 2014 at 8:04 am
Thanks, Nicola, it’s always interesting to hear from you.
However, I just did a quick search of both the first WUWT post and the Shaviv paper that you reference … neither one of them contains the graph under discussion.
But its not just that … neither one of them even mentions sunspots once.
Something similar to the graph appears in the Archibald post you also reference in your comment. However, that post also says nothing about sunspots. And rather than tracing its derivation to the Shaviv paper (which doesn’t mention sunspots), Archibald says:
To sum up, none of the three references you give show the graph I discuss above or even mentions sunspots … and the one of the three above that is similar, the Archibald post, has no real provenance for the data (it appears to be smoothed sunspot data … but which sunspot data and what smoothing?)
Sadly, like the author under discussion, Archibald doesn’t provide us with either the R^2 value or the p-value for the claimed correlation. I can tell you right now, the p-value will be worse than I show above, because of the smoothing that has been applied to the sunspot data.
Short version? Your claim, that the graph in the head post is from the Shaviv document, is contradicted by the facts. Shaviv doesn’t mention sunspots.
Nor does Shaviv matter, because nothing in the provenance of the graph can overcome the lack of statistical significance of the claimed correlation in this graph.
Nor can all of your bluster rectify the fact that no peer-reviewer asked for that most elementary of calculations, the R^2 and the p-value.
Regards,
w.
PS—Your claim that the Shaviv paper was “discussed with Anthony’s great approval here” is an egregious misrepresentation. Of the entire post, Anthony only wrote the introductory paragraph, which says:
Since that was Anthony’s entire contribution, and he made no further comment on the paper in the comments section, this claim of yours about Anthony’s “great approval” is as false as your other claims … and you accuse me of not reading the paper under discussion?
Thanks Willis.
Doug Proctor says:
January 22, 2014 at 8:13 am
I don’t understand this. There is no trend in the ∆ sea level data, nor in the sunspot data. There is a claimed relationship where sometimes one leads the other, sometimes it follows the other, sometimes they move in parallel, and sometimes they move in opposition … perhaps that impresses some people as a prediction method. Me, not so much.
True. And pulling a random folk aphorism out of your fundamental orifice doesn’t make it a scientific objection to what I’ve written.
w.
I am not interested in reading the paper, however one can see from just looking at the graph that there is great correlation and that makes it fascinating. Sure it could just be a coincidence (which I assume is what you are essentially saying it is), but that seems extremely unlikely. There is ‘probably’ some reason for the close correlation.
It’s a similar situation with Milankovitch cycles. There is a great correlation. So great that people assume there is ‘something there’, they just aren’t sure exactly how the small-in-theory perturbation of the cycles translates into a huge difference in climate.
I won’t dispute your criticism of paper or journal, but I would like to point out your consistent closed-mindedness. There is an obvious correlation. Instead of being curious about why that is, you just choose to dismiss it, never to be thought about again as it doesn’t fit within your existing mental model. That’s poor science. Be curious! Being a criticizing curmudgeon is easy … anyone can do that.
Thank you. Another good presentation.
I would not automatically disparage a model fit with an R^2 = 0.15 and p = 0.08. Granted, it is not very strong evidence against a null hypothesis, so I wouldn’t believe the result either, but for a field in which nearly all relationships have small p values it is probably reportable (unless it has been selected out of a large number of results, in which case the observed significance level is really greater than the reported p value.) On the other hand, it is more probable than not that the reported association is closer to R^2 = 0.15 than R^2 = 0.
What’s curious, as you noted, is how quickly the curves changed from out of phase to in phase.
tallbloke says:
January 22, 2014 at 8:56 am
Thanks, Roger. I fear that peer review that does not require the mathematical calculation and reporting of the significance and R^2 value of a claimed relationship is not “serious colleague reviewing”, it’s a joke. The same is true of not requiring the “out of sample” testing of the Salvador model, plus not requiring at least some mention of the effect of using 20 tuned parameters in said model that Dyson warned about …
Look, you and Nicola and Gregori can puff and blow all you want about the high quality of the peer review, but the facts don’t bear you out. The peer review was horrendous, and you didn’t follow the requirements for reviewers or for the editors as well.
Show me how I can replicate Scafetta’s work without access to his data and code, for example … it’s written down as part of the Editors job to make sure that can happen. We’ve been fighting with Michael Mann and his ilk for years, fighting for the normal scientific transparency, fighting with Science magazine to enforce their own policies regarding data and code archiving … and you fools come along and you publish without including links to your data and code? What on earth were you thinking, Roger, that no one would care? That we wouldn’t read the papers critically?
Pathetic.
You guys signed up to play in the peer-reviewed swimming pool. Then you not only ignored the posted rules that you had agreed to follow, you smashed them to bits. Now you want to complain because the lifeguard threw you out of the pool? … color me unimpressed.
w.
Matthew R Marler says:
January 22, 2014 at 11:15 am
Thanks, Matthew. I’m skeptical because I’ve seen this same thing over and over in a wide variety of natural datasets. You find some relationship, it even might be strong … then it fades out, and some other relationship comes to the fore. Or it stays there, but suddenly changes phase. Or it’s there, but sometimes one leads the other, and sometimes one trails the other.
Causal relationships don’t do any of those things. When you walk out into the sun, you get warm. It doesn’t fade away and get replaced by warming from the stars. Nor does it change phase, so that next time you walk out into the sun it makes you cold. Nor does one sometimes lag and sometimes lead the other—I don’t get warm a few seconds before walking outside in some cases, and a few seconds after walking outside in other cases.
As I’ve said elsewhere, I spent a lot of time looking at, examining, and doing curve fitting on a whole host of natural datasets. The more I’ve done, the more skeptical I’ve become of this kind of claimed causal correlation, the kind that reverses phase and sometimes leads and sometimes follows and sometimes plays the Cheshire Cat …
w.
Willis: “it’s a joke. The same is true of not requiring the “out of sample” testing of the Salvador model,”
I’ve already pointed out that this claim is incorrect. You ignore that I point it out and repeat it.
There is much to criticise in this paper and the whole review question. Repeating inaccurate claims and not reading the papers before trying to dismiss them is not part of it.
… color me unimpressed.
I am not a statistician, but it seems like R-Squared is the wrong measure for determining significance of periodic signals. Imagine you have a signal
y(t) = a Cos(w t) + b Cos(w2 t)
And you have a theory about some phenomenon that affects this signal has an output
o(t) = a(t) Cos(w t)
Now if b>a you do an R-squared on this you are going to get a low R. Low significance. But clearly this downplays the accuracy of o(t) in ‘partly’ explaining y(t).
As an experiment I would challenge you to do an R-squared of Milankovitch cycles versus temperature. I’m pretty sure the result will be terrible. Yet you wouldn’t simple go on to dismiss Milankovitch as irrelevant and coincidental would you? No, because the correlation is too high over too long a period and suggests cause and effect.
Matt: What’s curious, as you noted, is how quickly the curves changed from out of phase to in phase.
Willis: Causal relationships don’t do any of those things.
They do _exactly_ that kind of thing when there is an interference of two or more relatively close periodicities. I explained that in detail above and as usual you ignore what does not suit you.
The paper goes into much detail looking at what other periods are present and highlights the 9.1 year periodicity which would combine to produce something like the effects you noted. I explained that in detail above and as usual you ignore what does not suit you.
whether R^2 of 2 (or whatever) is significant needs to be assessed in relation to the number of data points in question and the kind of data, something which I don’t recall you ever having calculated despite your insistence of people calculating R^2 and publishing it , you seem to have some unstated idea of what is a “good” R2 value.
Calculating the R2 value and publishing it without any criterion to judge whether it is significant does not tell us much except that you think you know more than you do know.
I suspect the offset near 1990 would match up much better if the sea level data were corrected for the effect of the Mt. Pinatubo eruption.
tommoriarty says: “I suspect the offset near 1990 would match up much better if the sea level data were corrected for the effect of the Mt. Pinatubo eruption.”
I suspect we’d do a lot better if we _stopped_ trying to “correct” data . See last graph in this post:
http://climategrog.wordpress.com/2013/03/01/61/
.. and here:
http://judithcurry.com/2012/03/15/on-the-adjustments-to-the-hadsst3-data-set-2/
[i]Causal relationships don’t do any of those things. When you walk out into the sun, you get warm. It doesn’t fade away and get replaced by warming from the stars. Nor does it change phase, so that next time you walk out into the sun it makes you cold. Nor does one sometimes lag and sometimes lead the other—I don’t get warm a few seconds before walking outside in some cases, and a few seconds after walking outside in other cases.[/i]
Please quote the line in the paper where they say sunspots cause changes in sea level. Again, they aren’t saying there is a causal relationship. The sunspots and the sea level are both effects. It is perfectly plausible for sunspots and sea level to lag behind a common root cause at varying times. This is a terrible failure of logic.
There are no error bars in the graph of sea level change. The sea level itself is difficult to measure, and its changes doubly so. I would like to see the uncertainties in that graph.
Greg Goodman says:
January 22, 2014 at 11:44 am
Great, Greg. I’m sure, then, that you can compare for me the R2 and p-value of the out-of-sample forecast with the R2 and p-value of the in-sample forecast.
“Testing”, whether in or out of sample, requires measurement and comparison. Near as I can tell, they have done neither. But you say they have, and perhaps you are right … so where are the results of the out of sample tests?
w.
One option seldom addressed in climate science is the possibility of synchronization. The idea that over time a system can be perturbed into synchronous behavior even with very weak coupling. Many chaotic systems are ‘accepting’ of synchronizing with a periodic driver even if the coupling is very small. These would seem to be a fascinating area to look into and I think should be considered more frequently. Sun activity and climate seems a ripe area for synchronization to occur without the activity having to be at direct forcing levels.
What’s the R^2 of the rotation of the Earth with temperature of Edmonton. Probably not great if you don’t also take into account very important periodicity of Earth’s tilt (seasons) and random noise of ‘weather’ on top of it 😉
Greg Goodman says:
January 22, 2014 at 11:57 am
Thanks, Greg. Please give me a real world example of this from climate data, if you think it exists. You know, something like where the sun warms the earth for thirty years, then cools it for ten years, that kind of thing. Or take evaporation. Show me a dataset where for four decades, evaporation increases with wind speed … and then for three decades evaporation decreases with wind speed.
Next, show me a causal relationship where for several decades, the putative effect lags the claimed cause, and then for several decades the effect precedes the cause. You know, like where for half a century the tides trail in the wake of the movements of the moon and sun, and then for the next half century they precede the movements of the moon and sun. That one should be interesting.
As Fourier showed, and as you correctly state above, you can create anything by combining wave forms. Whether that particular situation exists in causal relationships here in the real world of cause and effect is another question. In particular, it’s rare for the effect to precede the cause …
w.
Greg Goodman said “I suspect we’d do a lot better if we _stopped_ trying to “correct” data.”
When data can be properly corrected it should be corrected.
The first link you provided is about smoothing techniques, but I did not suggest smoothing the data. However there are many occasions when smoothing is appropriate and useful.
The second link you provided concerns a specific data set – totally different from the sea level data that this post considers.
I think there is fairly broad agreement among alarmists and realists that the Mount Pinatubo eruption in 1991 reduced the sea level rise rate for several years. So why not correct for it in this context?
Consider…
Significant decadal-scale impact of volcanic eruptions on sea level and ocean heat content (Church, et. al., Nature, 2005)
http://www.nature.com/nature/journal/v438/n7064/full/nature04237.html
“For the Mt Pinatubo eruption, we estimate a reduction in ocean heat content of about 3 1022 J and a global sea-level fall of about 5 mm…
To better understand the processes involved, we focus on the Mt Pinatubo eruption (June 1991) because of its stronger radiative forcing and the better observations available. The primary driver of the fall in sea-surface temperature, GMSL and GOHC in the PCM is the rapid reduction in net solar flux at the ocean surface (Fig. 3a) of up to 6 W m-2 in late 1991. By early 1994, the net solar flux had virtually recovered to pre-eruption values”
Tom
To understand what is observed it is necessary to understand the mechanisms. Mass balance and temperature changes do not explain the recent ocean level changes. The paleo record shows unexplained large changes in ocean level that cannot be explained. Something not melting ice sheet or change in temperature is causing the ocean level to change.
The explanation as to why the ocean level changes following the periodicity of the solar cycle with a change in lead and lag is the same reason the boost satellites get from the close earth orbit maneuver has changed with time. We know gravity cannot change. There is a missing variable. Working with the laws of physics what variable could affect satellite orbits and ocean level.
(Ocean level is now falling based on raw satellite data. The raw data is adjusted to create the rise in ocean level.)
Mass and volume contributions to twentieth-century
global sea level rise
The rate of twentieth-century global sea level rise and its causes are the subjects of intense controversy1–7. Most direct estimates from tide gauges give 1.5–2.0 mm/yr, whereas indirect estimates based on the two processes responsible for global sea level rise, namely mass and volume change, fall far below this range. Estimates of the volume increase due to ocean warming give a rate of about 0.5mmyr21 (ref. 8) and the rate due to mass increase, primarily from the melting of continental ice, is thought to be even smaller. Therefore, either the tide gauge estimates are too high, as has been suggested recently6, or one (or both) of the mass and volume estimates is too low.
ftp://falcon.grdl.noaa.gov/pub/bob/2004nature.pdf
tommoriarty says: The first link you provided is about smoothing techniques
No, that article is not about “smoothing techniques”. It uses frequency filters to look at inter-annual and decadal scale variation in individual ocean basins and points out exactly the mistake Willis is trying make here in misreading the ‘phase crisis’ problem.
” See last graph in this post:” refers you to the part which is relevant to my comment about doing better if we stop “correcting” the data. It shows how Hadley processing is corrupting the lunar signal that is a strong component in many ocean basins and is the same one Scafetta and BEST (land) data show. It is also found in HadCRUT, presumably due to land portion since the “Had” part has removed , or more likely corrupted it into something else.
With many regarding HadSST as the “gold standard” of climate records, it hardly surprising natural cycle signals are not being detected.
All good clean fun. Of course, the ultimate proof of a hypothesis like this “There is a predictive correlation between sunspot numbers and the rate of SLR” is easily tested, since the graph above inexplicably ends at the year 2000. We have the additional advantage that in the subsequent 14 years, the sun has been at its lowest ebb in 100 years, and we have multiple sources for measuring SLR including the old tidal gauge network and the newer satellite-based measurements.
If the hypothesis is correct, then, we should have very low levels of SLR for the indefinite future, as predictions of sunspot levels are pretty low indeed.
But that raises many questions. One is “which sunspot numbers were used”, since Our Friend Lief suggests that sunspot numbers reported in the early 20th century do not match those reported in the late 20th century. This is an ongoing problem, I understand, but it is apparently an important issue if early numbers are all smaller by an (in)consistent scale factor as has been suggested. Something to clarify.
The bigger issue I have, however, comes from the SLR data. Say what? That’s not the SLR data. This is:
http://en.wikipedia.org/wiki/File:Trends_in_global_average_absolute_sea_level,_1870-2008_%28US_EPA%29.png
Note that both tide gauge data and satellite data (from two sources) are presented. Now, the data presented in the figure above is SUPPOSED to be the first derivative of this curve — its slope. And it’s not.
I count approximately thirty years in the actual record from 1900 to 2000 where SLR was negative. There are at most fifteen such years visible in the plot above. The clustering of negative years is also completely different — they are nearly uniformly distributed in the actual SLR data, and are somewhat antibunched, with a negative year more likely than not to be followed by a (possibly strongly) positive year. The bunching evident in the graph above is difficult for me, at least, to discern.
So I have to question: Not only which sunspot data, corrected or uncorrected, but where the hell did the SLR data come from? Because it certainly didn’t come from the graph I link above. In the graph above, only 4 decades in the entire century had years with negative SLR. In the actual data, there isn’t a single decade without multiple years of negative SLR.
If you plotted sunspot numbers against the actual coarse grained first derivative of SLR, I don’t think there would be any correlation at all.
rgb
tommoriarty says: “Consider…
Significant decadal-scale impact of volcanic eruptions on sea level and ocean heat ”
To support this claimed “decadal-scale impact” you quote a paper saying flux was back to normal by 1994. That’s less than three years.
The big problem is false attribution. There are temporally coincident cooling trends that are already under way _before_ the eruptions of both Mt P and El Chichon. Alarmists confound the two and make exaggerated claims of “decadal-scale impact” of volcanoes that allow them to jack up the CO2 sensitivity. This worked until we had a long period with virtually not volcanism. Then we saw the warming vs CO2 connection broke down.
Willis Eschenbach says:
January 22, 2014 at 3:01 am
Paul Westhaver says:
January 22, 2014 at 2:05 am
Willis,
Notwithstanding your gripes about the overall paper and the journal review process, If someone placed the curves in front of me, I;d be very interested i knowing why they are related. They are related without a doubt. So why?
No, they are not “related without a doubt”. What part of an R^2 of 0.13 and no statistically significant correlation seems unclear?
——————————————————–
Sorry Willis,
an r value is not necessarily an appropriate measure here. Considering its limitations is easy:
Two straight lines have an r value of 1, but the information content of that result would be close to zero.
However, if you have 2 identical curves with multipole ups and downs and thus a high information content, and shift the phase of one curve into different directions for its first and last half, you may end up with an r value of 0, though everyone would “see” that they are “related” or in this example even coherent.
That is why Shaviv can say that some correlations between solar variables and temperature are the best between any two variables in climate science.
And because of curves like this
http://kaltesonne.de/wp-content/uploads/2012/05/bond-et-al-2001.gif
I think, the existence of a solar amplifier is likely (though the r-value is “only” r=0.45)
Good evening, Willis. Would you be willing to publish your data from the digitized graph? I’m afraid my skills in that area do not match up to yours.
I ask because I am curious what the correlation would have looked like if one excluded the single out-of-phase period (which could perhaps be edge-effect in the data?) or what the correlation might be if one hypothesizes that the apparent lagging in cycles 17 and 19 were artifacts of measurement error. Such statistical tinkering after the fact can’t prove anything of course but it might lead to more interesting questions.
Manfred,
Exactly and good example. Correlation tells us if a particular factor is important. R^2 tells us if we how well we have put all the factors together into a predictive model. We can’t look at R^2 of 0.13 and conclude that sunspots are uncorrelated. That’s wrong. We can only conclude that a predictive model using only sunspots is incomplete. We have to do a correlation to determine if it is ‘correlated’ and clearly it is.
Who you gonna believe Willis, or your lying eyes? 😉
RGB, firstly you link to global average SL , that is not what the paper is using,
Secondly I recall a paper from Jevrajeva 2002 (?) that did a careful reconstruction just form tide gauges that had very clear 60 cycles. It may have been plotting 2nd diff IIRC. The official GMSL now includes GAIA [sic] adjustments and inv. barometer and you don’t to look at non-barmon data any more.
Also there were HUGE “corrections” to Jason data. In 2011 the GMSL record roughly followed UAH TLT, then something bad happened to it.
http://climategrog.wordpress.com/?attachment_id=524
Yet another climate parameter gets taken to the cleaners. I regard what Colorado our now presenting as having no analytic value. It has gone through political correction training.
Jevrejeva, S., J. C. Moore, A. Grinsted, and P. L. Woodworth (2008),
Geophys. Res. Lett., 35, L08715, doi:10.1029/2008GL033611.
Can’t find the graph but paper is worth a look.
Ian Schumacher:
At January 22, 2014 at 1:52 pm you ask
I don’t “believe” either. I accept the indication of the r^2 statistic, and that indication is that the paper is junk.
Richard
Greg Goodman said “No, that article is not about “smoothing techniques”. It uses frequency filters to look at inter-annual and decadal scale variation in individual ocean basins”
Yes, you are right. I mistakenly went to the home page of “Climategrog” instead of the sub-page that you specified. Sorry about that.
Greg Goodman also said “See last graph in this post:” refers you to the part which is relevant to my comment about doing better if we stop “correcting” the data. It shows how Hadley processing is corrupting the lunar signal that is a strong component in many ocean basins and is the same one Scafetta and BEST (land) data show. It is also found in HadCRUT, presumably due to land portion since the “Had” part has removed , or more likely corrupted it into something else. With many regarding HadSST as the “gold standard” of climate records, it hardly surprising natural cycle signals are not being detected”
This is only marginally relavent to the Holgate data.
It seems that you are implying a maxim that there are no appropriate corrections when you say “I suspect we’d do a lot better if we _stopped_ trying to “correct” data.” Did you mean the no data should ever be “corrected” (whatever that correction means) or are you referring to a specific reason that the Holgate data should not be corrected for the effect of Pinatubo in the early ’90s?
Willis, if you’d been asked to be one of the reviewers for the Copernicus papers, would you have said yes ?
lsvalgaard says:
January 22, 2014 at 10:34 am
Dr. S.
Curve fitting is my speciality, it’s great fun.
Here is something you might like to amuse yourself when you got some spare time:
http://www.vukcevic.talktalk.net/OldLadysHalo.htm
Any reason why (polarised) Aurora spectrum would have stronger correlation with the Earth’s core field than with the solar magnetic cycle that drives it?
Polarised?
Yes, following the NASA’s guidance on the mater: Solar coronal mass ejections CMEs in the even-numbered solar cycles tend to hit Earth with a leading edge that is magnetized north. Such CMEs open a breach and load the magnetosphere with plasma starting a geomagnetic storm.
Polarisation:
even numbered cycles Auroras = + (positive)
odd numbered cycles Auroras = – (negative)
Notice tiny 5 year ENSO signal in the Aurora’s spectrum, if Mr. Tisdale might like to know about, the explanation is waiting.
.
Mike: “I ask because I am curious what the correlation would have looked like if one excluded the single out-of-phase period ”
I don’t think it’s necessary to question the data or think it’s just one odd-ball cycle. There are other factors at play in climate (as is detailed in the paper !!) .
It maybe possible to see something from a cross-correlation of the data shown but just annual points is not going to be very informative.
The point is that this kind of slipping in and out of phase is _exactly_ what you get when you have two close frequencies ( like 9.1 and 11 , for ex.) As Ian says , all the low correlation value tells is that saying SSN is the primary and sole cause of MSL or temp change is not correct.
richardscourtney,
R^2 tests the predictive value of a model. Sunspots are not a model. They are ‘maybe’ a single factor in a model containing many factors. Correlation tells us if something is a likely factor and how important a factor it probably is. Sunspots are highly correlated and are therefore probably an important factor, but not the complete story for predictive purposes.
Do you disagree? Here are some examples.
Smoking is correlated with lung cancer. Smoking is not predictive though. Using smoking as the only factor for lung cancer would give us a poor R^2 value. Smoking is highly correlated with lung cancer though. This tells us that while not the only factor, it’s an important one.
The temperature in Edmonton is correlated with rotation of the Earth (whether the Sun is shining or not), however it rotation is not the only factor. Tilt is also important. If we build a model of temperatures that uses ‘only’ rotation, we will have a poor R^2 value. It doesn’t do a very good job of predicting absolute temperatures in Edmonton as daytime/nighttime temperatures only vary a small amount compared to seasonal temperature changes. However it has really high correlation. That tells us that Earth’s rotation is probably an important factor, just not the only factor and that we need to take other things into account.
I’m not a ‘sunspot guy’ and have no position on it, but when I see sunspot numbers correlation highly with something (correlation, not R^2) then I acknowledge that there is almost definitely something there.
Tom: “Did you mean the no data should ever be “corrected” (whatever that correction means) or are you referring to a specific reason that the Holgate data should not be corrected for the effect of Pinatubo in the early ’90s?”
I mean there is far too much speculative “bais correction” going on. If you make an uncertain “correction” to the data you ADD to the uncertainty, you don’t reduce it. In the case of volanism we do not know accurately how much radiative change is produced and more importantly we don’t the climate response to that change.
Much of the time it would be more appropriate to recognise that there are uncertainties and live with them. The current state of play is that we spend most of the effort analysing the result of corrections as much as the data.
Most these data sets seem to be controlled by groups with a “message” who are less than objective about how they alter the data.
What currently gets called “global mean sea level” is some phantom hovering slightly above the waves and getting more so each year.
It’s getting damn hard to find any true climate data to analyse.
All these people saying the correlation is visibly obvious – the first thing that stood out to me when looking at the graphs was the LACK of correlation. In some spots, the peaks coincide, in others they’re opposite. To me, the lack of correlation is quite clear. Are we looking at different graphs?
Willis is a little too smug for my liking. If it helps his ego.
What does WUWT fear? Turned into a mob.
Mike Rossander says:
January 22, 2014 at 1:50 pm
………….
I don’t know much about the SL measurements and have a view that are most likely within the error margin and hence taken on annual bases irrelevant.
However cycle 17 is a particularly important one, it shows what happens to the Earth’s magnetic field when the solar intra-cycle oscillations are close to the Earth’s orbital period.
http://www.vukcevic.talktalk.net/SC17.htm
TonyG says:
January 22, 2014 at 2:45 pm
All these people saying the correlation is visibly obvious – the first thing that stood out to me when looking at the graphs was the LACK of correlation. In some spots, the peaks coincide, in others they’re opposite. To me, the lack of correlation is quite clear. Are we looking at different graphs?
######################
ah ya,
worse than that they used 9 tidal guages? which 9? what happens of you pick a different 9 than the original author Holgate.
worse than that the author might have stole the idea, inlcuding the bits about GCRs
http://climateaudit.org/2007/02/11/holgate-on-sea-level
Ian Schumacher:
At January 22, 2014 at 2:40 pm you ask me
I strongly disagree.
The coefficient of determination (r^2) does NOT test “the predictive value of a model” except within the limits of a model which is correlation between two variables.
When two data sets are correlated then the r^2 indicates the proportion of the variance of one variable that is predictable from the other variable. Hence, it is a measure of the certainty of a prediction of one variable from the other as indicated by their correlation.
For example, if r^2 is 0.850, then 85% of the total variation in y can be explained by the linear relationship between x and y (as described by the regression equation). The other 15% of the total variation in y remains unexplained.
Putting that in plain language, if the r^2 is low then confidence that there is a useful correlation is low.
In his article above, Willis says
In other words, almost all the variation between the two parameters is NOT explained by a correlation between them.
Of course, that does not disprove a possibility that they are not each related to a third parameter, but the paper does not suggest any such third parameter. All one can say is that the correlation between the two considered parameters is so poor that the correlation is NOT a model which enables one of the two parameters to be usefully predicted from the other.
Richard
@Steven Mosher
January 22, 2014 at 3:02 pm
This was Steve McIntyres comment:
“…Doesn’t it look like there’s something like an 11-year cycle in this? Remind you of anything? I know that it’s a bit of a mug’s game trying to identify solar cycles, but here’s a plot of sun spot numbers in the same period. The maxima and minima of the solar cycles seem to match the fluctuations in sea level rise rather uncannily. While the resemblance is impressionistic (I don’t have a digital version of Holgate’s series), offhand, I can’t think of any two climate series with better decadal matching…”
http://climateaudit.org/2007/02/11/holgate-on-sea-level/
It’s late, past my bed time, I’ve forgotten but I did a graph on sea level ‘correlation’ while ago, so here it is:
http://www.vukcevic.talktalk.net/SeaLevel.htm
yes Manfred, trust nobodies eye’s. That’s why the experts on the thread call for the data to do the calculation. Note too that nobody dug down to check the data.
Now, If I selected 9 stations out of 177 and proved to you that UHI wasnt real.. wouldnt the FIRST thing youd do is ask for the data?
If I told mcintyre that 9 tree rings out of 177 gave me a flat MWP.. what do you think he would do.
Its simple. Before you let your eye’s decieve you. Get the data. Check the data. Then get the code. Check the code. Otherwise even the best eyes are lead astray
To DirkH “What does WUWT fear? Turned into a mob.” And the earlier commenter who made the analogy to circling wagons and shooting arrows outwards not inwards. For me this should not be a matter of taking one ‘side’ or the other – or of finding evidence to support the paradigms of one’s own ‘side’. Sceptism is for individuals – not ‘sides’ or teams (or any grouping that could refer to themselves as ‘we’ or ‘us’). So don’t expect me to join-up with some great ’cause’.
Above all, I want to see good science – of the Feynmann “bending-over backwards to disprove one’s own theory” type. Not the grasping at any old straw to support one’s own theory type of junk science that has become all too prevalent (in lots of different subjects). The sort of junk science that is trotted-out in support of the CAGW agenda is a pet-hate of mine – which is why I lurk at blogs like this one.
Unfortunately, the papers Willis and others are criticising fall far short of Feynmann’s ideals – to put in mildly – Willis has been too kind in my opinion.
For those interested: a really quick power spectrum from cross-correl, Jevrejeva d/dt(MSL) and daily sunspot area. No bells whistles or other decoration. 😉
http://i40.tinypic.com/206j7th.png
By far the largest peak at 5.405 years. Ubiquitous 9.03 present but small
20.27 will be rather uncertain
It’s only annual resolution CC but peak correlation of .335 at one year lag. Guessing by form, actual peak near 0.5 years.
5.4= 10.8 / 2 looks like the main solar linkage.
Interestingly the is damn near the period I extracted from Arctic sea ice about a year ago that got Tamino in such a panic he spent a week trying to diss my efforts before getting is a sulk and slamming the door shut.
http://climategrog.wordpress.com/2013/03/11/open-mind-or-cowardly-bigot/
It is also quite close to the 5.8 semi cosine that I found as a repetitive pattern this year when evaluating decadal variation in Arctic Sea Ice.
http://climategrog.wordpress.com/2013/09/16/on-identifying-inter-decadal-variation-in-nh-sea-ice/
@Steve Mosher,
eyes may deceive you, but also an r-value or similarly Persons’ correlation coefficient if you trust them without considering their limitations.
These are just widely used measures of correlation, but not the only ones.
A single phase shift in one data set can turn an r value from one to zero. And there may have been phase shifts in above data set. Or if you have 2 identical curves of shorts pulses, a small jitter may bring the r value from one to zero. And there may have been dating issues working that way.
For such curves with multiple ups and downs (like above or Bond et al 2001 etc), I think a better measure of correlation would be how often ups and downs in one curve occur in the other as well, plus compare similarity of the shapes of each 11 year cycle, what essantially is trying to quantifiy what your eyes tell you.
I did a very interesting correlation in regards to PRP,
Why was Roger chosen as an “editor” and “reviewer” [with no relevant qualifications] and what do all the authors of the “Special Edition” have in common? It’s quite special…
They are all favorite discussion topics at Tallbloke’s Talkshop (ordered by word frequency);
1. Wilson (2,380)
2. Scafetta (1,180)
3. Salvador (854)
4. Jelbring (812)
5. Morner (222)
6. Charvatova (213)
7. Solheim (59)
Oops. Too late at night to be doing this sort of crap. Slip up in collating data missed out several years and screwed up the CC calcs.
Here’s a corrected SPD. Peak at 10.4 as expected max max CC at 0.68 years is relatively low at about 0.18
Estimate signif at 0.138 with 127 pts but way past bedtime, so take with a pinch.
http://i44.tinypic.com/sffddu.png
Hey Pops, you should have done a control for significance against red noise. 😉
https://www.google.com/search?q=Solheim+site:tallbloke.wordpress.com#q=Red+noise+site%3Atallbloke.wordpress.com (118)
Even I score 135 and I’m banned !!
You do have a point though, the review process does look rather incestuous.
Here’s the cross_correlation fn before spectral analysis. 79 year period corresponds to the phase crisis in 1920s.
http://tinypic.com/view.php?pic=2hojthe&s=5#.UuB3_aJw1uA
Now quick calculation of what would mix with 10.45 to cause phase lag of one year in 79 …..
9.23 years.
So that’s why there’s a low corr coeff. Like I’ve been saying for some time it seems you can’t refute the presence of a periodicity just by trivial regression analysis.
This was a quick hack so that 9.2 could be N Scafetta’s 9.1 , my 9.05 or maybe 186/2 = 9.3
In view of other evidence I suspect it’s 9.05
Willis,
I’m a fan. Your scientific and personal essays have been educational and lightened my days.
Unfortunately, your latest work was the proverbial straw on a load from many, many essays from many sources. But . . .
. . . I must, at long last, say something about an endemic statistical solecism. (I apologize for wasting band width if someone beat me to it — I didn’t read all the comments.)
Hypothesis testing is a concept most undergraduates and a surprising fraction of Ph.D.s in fields that rely on non-experimental data find . . . perplexing. I have no idea how many times, in thirty some years, I said something like the following to one, a few, or a room full of (usually) undergraduate students (pedantry alert) :
The null hypothesis that a human’s body weight and donut consumption (say) are unrelated (null: r = 0, alternate: r 0) might be tested by comparing the:
a) calculated and critical values of a correlation coefficient, r, given the sample size and desired confidence interval (0.95 by convention in fields that depend on non-experimental data) or
b) maximum acceptable risk to a decision maker of rejecting a true null hypothesis (alpha, chosen carefully (?) in advance, 0.05 by convention in fields . . . . The sum of the confidence interval and alpha is 1.0. Always.) and the risk given the sample (the glorious p-value).
The null hypothesis can be rejected if the:
a) calculated correlation coefficient (r-hat, a caret over lower case r for estimated) is not smaller than the correctly chosen critical value (sometimes r*) or
b) p (a calculated value) is not larger than alpha (the maximum acceptable risk of rejecting a true null hypothesis, a carefully (?) chosen value).
Both methods yield the same result. Always.
in regression, of which Fourier analysis is an example, (capital) R-square reports the ratio of explained to total variation in an estimated equation, ignoring the effect (for example) of serial correlation in time series data and the inclusion of explanatory (independent) variables with obscure theoretical connections to the response (dependent) variable, both of which inflate r-square.
Iff (if and only if) there is exactly one explanatory variable, r = sqrt(R-square).
(pedantry off)
You report p = 0.08, so, if the author tested an hypothesis whose alpha was the conventional 0.05 with a test statistic (not R-square), the author, you, and your readers are unable to reject the hypothesis that there is no relationship between the dependent and set of independent variable.
I know of no hypothesis test based on R-square. One might test the null hypothesis that none of the estimated coefficients are “significant” with the F-test and related p-value, but reading a sentence that combines an R-square and a p-value makes my head . . . never mind.
Thank you for the time, energy, diligence, . . . your work shows. I look forward to reading many more reports of your always interesting, always cogent, analyses and anything else you may post. All the best!
Willis, will you please post the digitized data you took? Thanks
The other reason it’s a tragedy is that they were offered an unparalleled opportunity, the control of special issue of a reputable journal. I would give much to have the chance that they had. And they simply threw that away with nepotistic reviewing, inept editorship, wildly overblown claims, and a wholesale lack of science.
I think you have overreacted.
What is the alternative hypothesis to H0: R^2 = 0? How about H1: R^2 >= 0.10? For something that is influenced by many different agents, R^2 = 0.10 is too large to ignore, even though it is sometimes (in some behavioral research) customarily called “small”. ( Most of the risk factors for high blood pressure, such as obstructive sleep apnea, have R^2 <= 0.10, but they are important in a large population.) Then the probability of obtaining R^2 = 0.15 is at least 0.32, making the outcome that was reported 4 times as likely given H1 than H0. Say for the sake of argument that you were willing to place prior odds on H1 at 1:4, 0.2 probability that H1 is true, 0.8 probability that H0 is true? Then by Bayes' rule the posterior odds would be 1:1, or H0 and H1 equally likely. Even though p = 0.08 is not strong evidence, it does support H1 more than H0.
(That's a sketch. A full development would need some more detail, including explicit examination of conditional independence.)
Null hypothesis testing with conventional levels of statistical significance is not the only important method of statistical inference.
The full paper is worth reading. It's too bad that the authors and editors have not made the exact data and code used available publicly, but that does not make the paper a waste of reading time. If you an I ran the scientific publishing industry then data and code would have to be presented as a condition of review, not merely of publication, but we don’t, and in the mean time we have to work with people who don’t agree with us; and that includes readers on this forum, most likely.
vukcevic says:
January 22, 2014 at 2:25 pm
Curve fitting is my speciality, it’s great fun.
But it ain’t science, and should not be peddled as such.
Ah, nuts. The italics are supposed to end before “I think you have overreacted.”
and the word “review” is to be italicized in “as a condition of review“.
[Fixed. -w.]
Willis,
I don’t believe that your categorical dismissal of a pattern is appropriate.
I see a pattern. It is there. My mathematics skills limit me in that language but my pattern recognition machinery is very convincing. I suppose I could work on it but it has been a while since I did such comparisons.
I suggest that the data sets are better related than scatter plots.
So I say why is my pattern recognition machinery telling me that there is a pattern? I see groupings within the set. so dealing with the whole set without accommodation localized features seems like a half-hearted attempt at being a critic.
I suggest “seemingly unrelated regression analysis” as a subject matter.
Keep an open mind.
JCasey
I see similar frequencies as well with locally varying phase and amplitude.
A periodic function and many of its harmonics will also fail the R and P tests.
I would normalize the data sets against a common aggregated periodic function. Then do a test. I bet that will yield a close correlation. I wish I had time to do it. Maybe I will later.
One last comment.
The underlying relationship will be revealed by looking at the outlier sets. That is where all the information resides. I as an act of discipline, assume that the two sets would otherwise be superimposed. (This is where I differ from Willis) Examining the regions of greatest error, to me, are the areas of interest.
Hey Willis,
getting anywhere with the reproduction of the 350 yr SSN record with the randomly chosen orbital periods I gave you yet? Yo said you could do it easily with all the extra ‘free parameters’. Y’know, wiggling trunks and all that? Don’t forget R.J. Salvador’s model also reproduces the Maunder minimum well, Svalgaards previously stated acid test.
Talk is cheap. A little less condescension and a little more action baby.
REPLY: Are you unable to read comments? He’s posted it right here: http://wattsupwiththat.com/2014/01/22/riding-a-mathemagical-solarcycle/#comment-1545760
– Anthony
REPLY: So because you hate what Willis has to say, along with the maths to back it up (you’ve presented none), it was OK to violate my trust? Interesting. – Anthony
I violated your trust by alerting the PhD’s to the ad hom and bad math attack about to be launched on them by Willis, and you sacked me from the voluntary 30 hour a week moderation job I was doing for you. It was a merciful release from a situation supporting behaviour I could no longer stomach.
Willis’ maths left a lot to be desired on that occasion Anthony, and you refused the PhD’s the right of reply here where the attack on them was posted. So they asked me to post it at the talkshop where they could get a fair crack of the whip.
http://tallbloke.wordpress.com/2012/02/09/nikolov-zeller-reply-eschenbach/
Can you or Willis find anything wrong with their maths? I doubt it.
REPLY:Well OK since you’ve brought it out into the open, here’s my say. As you know, your breach of trust caused a very insulting and nasty personal attack to be sent to me from the N-Z camp, and even though the article had not been published, they made ludicrous demands to go along with the insults. And so, you rationalized using privileged information, obtained by the trust I placed in you by giving you administrative access to my blog, using that to launch a rebuttal. I said then after they way they emotionally and unprofessionally exploded over something not yet published that you could have them and their reply. You were a lot more contrite then when I explained why I could no longer trust you, and you understood why and agreed with my decision to terminate your access. I find it telling that you would use my trust that way. It is a sad commentary on your ethics.
You were also quite thankful when I went to bat for you in your times of need.
It is unfortunate, the situations you have created for yourself through your own actions. which further isolate you. Best of luck in the future. – Anthony
Anthony says:
I said then after they way they emotionally and unprofessionally exploded over something not yet published that you could have them and their reply.
Willis had already infuriated them with the avalanche of ad hom attacks and misrepresentations of their work in his previous posts. They demonstrated to you that his maths was bunk and that posting this next attack was ill advised. But you posted it anyway and denied them a right of reply.
You were a lot more contrite then when I explained why I could no longer trust you, and you understood why and agreed with my decision to terminate your access. I find it telling that you would use my trust that way. It is a sad commentary on your ethics.
I knew we’d no longer be able to work together after you condoned WIllis’ behaviour, and that was a matter for regret. I was certainly sorry we’d arrived at the parting of the ways. I have no regrets over my ethical choice to put correct maths and decency in debate before loyalty to a blog which permitted the personal and scientific abuses Willis indulged in, and your condoning them.
You were also quite thankful when I went to bat for you in your times of need.
I was. But that innings was before the events we parted company over. I maintained decorum about those events so far as was possible since then. But the unwarranted attack you and Willis are now making on the integrity of the members of our research group is the last straw for me. We’ll keep going with our productive research though.
Best of luck in the future. – Anthony
I believe in making my own luck, but thanks for the sentiment, and the same to you.
Rog
There are two fundamental problems here.
1) Just how flexible the concept of ethics has become, and
2) Whether or not integrity has anything at all to do with the concept of ethics anymore.
The solution(s) might very well lie closer to the end of the next ice age. Perhaps more propitiously, the one after that. Assuming we actually manage to get to either one, or both………
“An examination of the fossil record indicates that the key junctures in hominin evolution reported nowadays at 2.6, 1.8 and 1 Ma coincide with 400 kyr eccentricity maxima, which suggests that periods with enhanced speciation and extinction events coincided with periods of maximum climate variability on high moisture levels.”
state Trauth et al in “Trends, rhythms and events in Plio-Pleistocene African climate”, Quaternary Science Reviews 28 (2009) 399–411 http://www.manfredmudelsee.com/publ/pdf/Trends-rhythms-and-events-in-Plio-Pleistocene-African-climate.pdf
We are at an eccentricity minima right now. It will most likely be 200kyrs (two more post-MPT100kyr glacial/interglacial cycles) before we are at the next eccentricity maxima, e.g our next potential hardware braincase-capacity upgrade.
Yet, to this very day, try as we might, chimps STILL do not post their data or their code……go figure
Enjoyed this Willis, Anthony, et al etc. Stay thirsty my friends, and remember, it’s only as bad as it hits your wallet, ye old universal gaia language.
Strange doth it seem, methinks, how much scrutiny be given the trees at the expense of the forest. Especially given the half-precessional-cycle+ age of the Holocene.
Apologies to all. The last sentence of my last post was actually a partial, still being edited, response to a comment on a sports blog (another tab that was up). I really do not know how it ended up at the end of the response intended here. I did not paste anything before clicking “Post Comment” here. Yet here is the last sentence from a longer comment on some other blog on something else altogether. How a sentence not actually complete from another tab got appended here I really do not understand.
Again, my apologies.
Obviously a reboot, at a minimum, is in order……followed by some serious system scanning for bots and malware.
[Fixed. -w.]
Willis Eschenbach says:
January 22, 2014 at 9:36 am
“Dear God, would all of you folks making this or a similar claim please do the freakin’ math! ….”
Willis, did you read the rest of my post? The next sentence is:
” Of course, the eye can be easily fooled.”
I simply observed that there does appear to be a striking correlation, but nowhere did I state that the correlation is real or proven. That’s why I suggested some kind of Monte Carlo analysis to give an indication as to how frequently a similar correlation could be produced by chance.
I made some reasonable criticism and suggested more analysis is needed. Isn’t this what science is supposed to be about?
Best regards,
Chris
ROFLMAO! What did you actually prevent? It took N&Z two weeks to write a rebuttal anyway. Was a critique by Willis (which he does all the time) worth abusing your moderating privileges? Most authors respond in the comments and a debate ensues. I have never seen such a hysterical reaction before …wait, maybe there is a trend here.
I had no idea anyone took the Nikolov & Zeller stuff so seriously. I initially looked to see if it was published (still not) and moved on, or was that being saved for the second “special” edition of PRP? Fits my PRP special edition correlation to the Talkshop theory ,
Nikolov (2,200)
Who seriously reads these comments by the author and does not step back for a minute,
It reads like WUWT is being punked.
Paul Westhaver:
To ensure that I am not addressing it out of context, I am copying all your post addressed to Willis which is at January 22, 2014 at 8:56 pm.
Taking your final point first, it is an old adage that
One needs to keep an open mind, but not so open that your brains fall out.
The assessed paper suggests
So, it is claimed there is an effect of “yearly change in coastal sea level” which is RELATED to “solar cycles“ as indicated by “sunspot numbers”.
Willis is assessing the claimed effect.
There are two possibilities which are
(a) The “sunspot numbers” and the “yearly change in coastal sea level” each varies with similar periodicity BUT their variations ARE NOT RELATED.
Or
(b) The “sunspot numbers” and the “yearly change in coastal sea level” each varies with similar periodicity BECAUSE their variations ARE RELATED.
Willis has provided cogent evidence of (a); i.e.
(a) The “sunspot numbers” and the “yearly change in coastal sea level” each varies with similar periodicity BUT their variations ARE NOT RELATED.
His evidence includes
Lack of coherence indicated by varying synchronicity over the assessed period.
Lack of correlation indicated by low R^2 and p values.
Lack of causal mechanism to induce a relationship.
Possible cherry picking of sea level data.
and Leif Svaalgard adds possible cherry picking of sunspot data.
Simply,
1.
the paper provides no evidence for existence of the relationship claimed by the paper,
2.
analysis of the data used in the paper indicates that the relationship claimed by the paper does not exist
3.
and the paper uses dubious data for each of the parameters which the paper claims are related.
However, you say you “see a pattern”. Yes, you do. The “pattern” is that the “sunspot numbers” and the “yearly change in coastal sea level” each varies with similar periodicity. But so what? Nobody disputes that those two parameters varies.
Willis has demonstrated that the variations of those two parameters are NOT related although the paper claims they are related.
I hope this has clarified the matter.
Richard
My comment got censored at the Talkshop,
“Good thing you alerted them in time, otherwise we would of had to wait longer than two weeks for the rebuttal – an eternity when dealing with Blog science. Thanks to your swift action, crisis averted!”
I don’t know why?
By Team PRP defending the indefensible they are creating a skeptic turkey shoot. They are full of bullet holes and bleeding out, while we have a band of zealots going around telling everyone they have not been shot. So instead of admitting they should not have stepped in front of a loaded gun, they keep telling us it is just a flesh wound!
lsvalgaard says:
January 22, 2014 at 8:43 pm
But it ain’t science, and should not be peddled as such.
Oh yeah, or is it just because you happen to find implications somewhat distressing to the ‘sun is impotent’ idea.
This specific curve fitting is based on two well known sets of data:
– Sunspot count from the world’s standard (SIDC)
– Geomagnetic data from the four world’s top scientists in the field (Jackson, Bloxham, Gire and LeMouel).
Both sets of data is used by NASA, NOAA and Institut de Physique du Globe de Paris among others.
I am sure you actualy delighted by this finding that climate and solar magnetic cycle have strong and direct relationship, but as a true scientist choose to remain sceptical until my results are verified by the academic establishments.
Italian professor Giovanni Gregori announced: “I am presently smoothing an 8 volume set (8,000 pages, completed) on the electromagnetic coupling between solar wind and Earth. It has been a hard 10-year job.”
I assume you have already reserved your set of copies.
Richard Courtney,
You’ve made my point exactly. Thank-you,
Willis’ analysis ignores normalization of the OBVIOUS periodic behavior.
What I asserted, as did JCasey etc, is that both sea level rise and sunspot number share a periodic influence. Not that one drives the other. Rather that they are both influenced by something shared. Gosh knows what that is.
The data shows that and despite Willis’ narrow application of statistics on the broad set the pattern remains.
It is contingent on us to discover what the connection is.
I find it remarkable that something so obviously a pattern has to be so viciously dismissed with such lame application of stat analysis. Seems to me there is a politic at work here that is outside my experience.
So be it.
There is a pattern, I call it like I see it. The weakness then is not in my eyes but in Willis’ inability to extract the periodic behavior and THEN apply the regression analysis. Maybe he can’t. I can’t at the moment. But it is there.
I suggest a good frequency analysis be done. My eye says there is a fundamental, and maybe a low F modulation and a linear superposition. That is what I would filter out, if I had the time.
George in Pennsylvania,
Do you think you are up to a Fourier type analysis? Seems to me the data set to a bit small for that but I bet you’ll get something. As for the donuts and my mass, something tells me there is also a connection.
To Willis:
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1545217
There Willis says: “However, I just did a quick search of both the first WUWT post and the Shaviv paper that you reference … neither one of them contains the graph under discussion. But its not just that … neither one of them even mentions sunspots once.”
Willis further proved the case that he does not know how to read a scientific work, he simply jumps around and he is quite confused on basic scientific concepts.
This is the figure of Solheim that Willis reports in his post:
http://wattsupwiththat.files.wordpress.com/2014/01/sea-level-change-and-sunspots.jpg
Here sea level changes and sunspot number oscillations are compared.
Here is the figure of Shaviv reported in WUTW post that I linked above:
http://www.sciencebits.com/files/pictures/research/calorimeter/calorimeter2.gif
Here sea level changes and a solar irradiance reconstruction, which Willis does not know is made mostly with the sunspot number record and presents the same cycles, are compared.
Here is the figure of Archibald:
http://wattsupwiththat.files.wordpress.com/2009/04/sea-level-rate-of-change-and-solar-cycles-510.jpg?w=640
where sea level changes and solar cycles are compared.
Those who understand a minimum of these things realize that the three figures are exactly the same thing because the issue was to compare sea level changes and solar cycles. If the solar cycles are expressed in sunspot number or in total solar irradiance is the same, evidently. But Willis did not get it, and Anthony doesn’t it either!
vukcevic says:
January 23, 2014 at 5:22 am
This specific curve fitting is based on two well known sets of data
And that is where the pseudo-science comes in. You have not described in detail how the two sets of data are combined and by what process they should be combined.
lsvalgaard says:
January 23, 2014 at 8:11 am
vukcevic says:
January 23, 2014 at 5:22 am
This specific curve fitting is based on two well known sets of data
And that is where the pseudo-science comes in. You have not described in detail how the two sets of data are combined and by what process they should be combined.
……………….
Aaahh, memory is sometimes like a hazy sunset…
See email Dec 10 2012
from vukcevic to leif@….
Alternatively, you can derive same waveform directly from your aurora data, since aurora contains both solar and Earth’s magnetic oscillations.
vukcevic says:
January 23, 2014 at 9:04 am
Aaahh, memory is sometimes like a hazy sunset…
See email Dec 10 2012 from vukcevic to leif@….
That is how I can state it is pseudo-science
Alternatively, you can derive same waveform directly from your aurora data, since aurora contains both solar and Earth’s magnetic oscillations.
No, they don’t. Only solar.
Dr.S.- No, they don’t.
Vuk – Oh yes they do, see my comment with the ‘OldLadysHelo’ link
vukcevic says:
January 23, 2014 at 9:20 am
Vuk – Oh yes they do, see my comment with the ‘OldLadysHelo’ link
already here you disqualify yourself from serious consideration.
You can take that from this world-famous expert on solar-terrestrial physics and aurorae.
Paul Westhaver:
Your post at January 23, 2014 at 6:16 am
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1546284
either misunderstands or willfully misrepresents my post at January 23, 2014 at 2:49 am
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1546134
There is NO RELATIONSHIP to investigate. I explained this.
You are being deceived by appearances. I explained this, too.
The two variables each have similar periodicity but there is NOT a causal link between them.
Please read what I wrote. I have provided a link for you to jump to it from this post.
Richard
Nicola Scafetta says:
January 23, 2014 at 6:39 am
Thanks for that, Nicola. I had said that the current author didn’t do the math to establish whether or not his model was a good representation of reality.
Your claim was that his graph about sunspots could be traced to a paper by Nir Shaviv which doesn’t say a word about sunspots. Your further claim was that the lack of a mathematical analysis in the Solheim paper was fine, because all of the needed analysis was in the original Shaviv paper, viz:
How does that work? You’ll have to connect the dots for me on that one, Nicola. Do you simply transfer the R^2 and the p-value from one graph to the other, despite the fact that neither of the two things that Shaviv is comparing are the things that Solheim is comparing?
Shaviv shows the correlation of TSI with the Douglas sea level record, while Solheim shows the correlation of sunspots with the Holgate sea level record. The Douglas sea level record is from 9 tide gauges, and starts in 1900 and ends in 2002. The Holgate record is from 24 tide gauges, and starts in 1909 and ends in 2000 … are you seriously saying that we use Shaviv’s figures regarding the correlation of TSI and the Douglas 9-gauge sea level data, pretend that they are actually the figures for the sunspot and Holgate 24-gauge sea level relationship, and call that science???
Yeah, I suppose you are.
However, if you think the answer is “Yes”, if you think we can use Shaviv’s results as a proxy for those of Solheim, you haven’t looked at the two sea level change datasets … here they are:
Yeah, that’s impressive … the R^2 on the data sets is only 0.2. And you think that the mathematical results from one can simply be substituted for the results of the other? Really?
Nicola, I have to say, that is easily the most convoluted, childish, devil-may-care attempt I’ve ever seen to explain away the fact that someone didn’t do a basic necessary mathematical analysis to back up his claims. He didn’t do the math, Nicola, the reviewers let it slide, the Editor didn’t care … and now we get this farrago of excuses from you, and a claim that because someone once did something kinda sorta similar, that we can use their numbers? This is your idea of science?
And you wonder why your special issue got shut down …
w.
Richard,
With respect, you may well have intended to flame my post or refute it but you, in fact, confirmed what I said.
I have no control over what you say. You said what you said.
You are in no position to say that ther is no relationship when there is clearly one recognizeable by an 8 year old child. ( I asked one who was nearby what they saw)
It seems that even the least inform among us see that there is a common behavior in the thwo plots and that they look alike.
It is contingent on science (scientists)to investigate the reason we think that is so. To dismiss an obvious relationship as a trick of the eye or brain is shoddy incurious science.
I see what I see. I see a relationship to something.
Your state agreement OR refutations are no substitute investigation and analysis.
I see a pattern.
Plain and simple.
rgbatduke says:
January 22, 2014 at 1:32 pm
As if …
Following Nicola Scafetta’s suggestion above, I looked at the two sea level datasets used by Nir Shaviv and by Solheim, viz:
Now, following your excellent suggestion, here’s the further comparison of Solheim sea level data with the Church and White dataset you referenced, which is available here:
Oh, my goodness! You put your finger on that one, Robert. This is hilarious. When we use the C&W data, which is somewhat of an industry standard, that’s what we get.
And since this is a science site, the R2 value of the relationship between the Church and White sea level data, and the data used by Solheim, is 0.004 … and the p-value is 0.40.
Finally, this makes it perfectly clear that as you suggest, Robert, “If you plotted sunspot numbers against the actual coarse grained first derivative of SLR, I don’t think there would be any correlation at all.”
Nicola and Tallbloke, are you following this? Your comments would be … well, likely inflammatory but interesting. In any case, any paper making the claims Solheim makes should have dealt with this glaring problem.
w.
Here is the true relationship between sunspots (I’ve used the NASA values) and sea level changes (Church and White values):
The R2 between the two is 0.009, and the p-value is 0.26 …
In other words, the Solheim paper discussed in the head post got the fate it deserved when the journal was shut down …
w.
Willis Eschenbach says:
January 23, 2014 at 10:54 am
Here is the true relationship between sunspots (I’ve used the NASA values) and sea level changes (Church and White values)
It looks like the solheim values are just a smoothed version of C&W, perhaps with a 5-year window. What happens if you smooth C&W and compare with Solheim?
Mike Rossander says:
January 22, 2014 at 1:50 pm
Mike, usually I would do that, but in this case to emphasize my point about poor science in the Special Issue, I’m gonna say no, and refer you to the author. Why should I do the author’s work for him?
In this debate, many people including the authors think that not publishing data is some minor thing. Lots of folks, including Jo Nova, tell me that I should just email the author. So I regret to say, I’m gonna just pass their excellent advice on to you …
w.
Ian Schumacher says:
January 22, 2014 at 1:52 pm
Looking at the graphs above, I’d say you should believe me … false correlations are a dime a dozen in climate science.
In addition, you say:
Since the R^2 value is the square of the correlation, your statement has no meaning.
w.
Greg Goodman says:
January 22, 2014 at 1:56 pm
Ummm … I think that’s his point …
w.
One last comment (I shouldn’t even take the time as I’ve got deadlines rapidly approaching). The paper at the top only uses 9 specific coastal sites that appear to be correlated with sunspots (although there are many more sites to choose from, IIRC, tide data is available from most the major harbors around the world on hundred year timescales is it not?) This exposes the paper to the criticism of cherrypicking/data dredging. In the figure caption they indicate that they only included nine of 177 that were available. Really? Why THESE nine? Could it be because when these nine were averaged, it looked like there was a correlation, where when all 177 were averaged, there was nothing?
So sorry, Willis, your R^2 and in particular your p-values are incorrectly computed. You have to use:
http://en.wikipedia.org/wiki/Bonferroni_inequalities
(or a Bonferroni correction) to assess the p-value, lest you fall into the trap of believing that green jelly beans cause cancer. Choosing 9 out of 177, I can confidently state that one must adjust even the p-values you obtain to nothingness. If one looks at all possible groupings of the tide gauge data and chooses a finite set that happens to exhibit a (comparatively poor) correlation, you have to take into account all of the rejected permutations that had even less. Otherwise we’ll be able to prove that ESP is real, that prayer works, pretty much anything we like. In fact, this is sort of like what the warmists do when they point to a bad storm and go “look, must be due to global warming”.
So its worse that we thought. So to speak.
rgb
Greg Goodman says:
January 22, 2014 at 1:56 pm
Greg Goodman says:
January 22, 2014 at 2:03 pm
Here are the Jevrejeva annual sea level changes plotted against sunspots …
You sure that you want to use that dataset? …
w.
J Martin says:
January 22, 2014 at 2:19 pm
Good question. Probably yes, just to watch the chips fly. However, I’d have insisted that my name and reviews be posted as part of the deal.
However, I fear that the editors and authors were not looking for my kind of reviews … hard as it may be to believe, I’ve been known to be blunt and direct in the past …
w.
Mike Rossander says:
January 22, 2014 at 1:50 pm
You ask, if we selectively weed out and ignore the data that doesn’t agree with our hypothesis, will the correlation get better and make our hypothesis look stronger?
I don’t think you’ve considered all the ramifications of what you are asking, Mike.
w.
Ian Schumacher says:
January 22, 2014 at 2:40 pm
First, no, sunspots are not “highly correlated” with sea level rise. That’s an artifact of poor data selection. See above et seq.
Second, while there are good examples of what you are talking about, cigarette smoking isn’t one. The R^2 between smoking and lung cancer is actually quite high. See here for an in-depth analysis.
w.
Willis Eschenbach says:
January 23, 2014 at 11:23 am
You ask, if we selectively weed out and ignore the data that doesn’t agree with our hypothesis, will the correlation get better and make our hypothesis look stronger?
some people actually do this. They call it Bayesian estimation. Here is an example:
“The regression slope was found to be somewhat different if Least Median Squares (LMS) or Bayesian Least Squares (BLS) were used [Lockwood et al., 2006a, and references therein], but the procedures converged on very similar regression lines if outliers were progressively removed. Hence we here use OLS but the largest outliers were removed until the regression converged on a stable line” from ‘Reconstruction of Geomagnetic Activity and Near-Earth Interplanetary Conditions over the Past 167 Years: 1. A New Geomagnetic Data Composite, M. Lockwood et al. [Ann. Geophys. 2013]’.
Paul Westhaver says:
January 22, 2014 at 8:56 pm
Yes, I know your pattern recognition is very convincing. It is a common problem with all humans—we often see patterns that are not there, very convincing patterns.
That’s why we invented math, Paul, in part so we didn’t have to depend on our pattern recognition skills.
w.
Paul Westhaver:
I am replying to your post addressed to me at January 23, 2014 at 10:15 am. For convenience of others, this is a link to it
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1546575
It begins by saying
Well, everybody knows what is to follow when a message begins saying, With respect”, and your post does not disappoint.
There is some confirmation of your view in my post at at January 23, 2014 at 2:49 am
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1546134
I said there
So, I said, “you say you “see a pattern”. Yes, you do.”
That is the ONLY confirmation of what you said that I have provided.
But the apparent “pattern” which you see is spurious for the reasons I explained.
You have persistently ignored my explanation and now say
Rubbish!
I wrote
Your “position” is that you ignore all that.
And you have the gall to say to me
I summarised the “investigation and analysis” which you refuse to consider because it refutes what you want to be true.
And you assert that I should ignore the results of that “investigation and analysis” because it does not agree with the view of “an 8 year old child”!!!
Richard
lsvalgaard says:
January 23, 2014 at 10:59 am
Not all that good … R2 = 0.12.
Remember that Solheim is using just 9 tidal stations …
w.
Dr. S.
Here it is Aurora and aurora data alone
http://www.vukcevic.talktalk.net/OldLadysHalo.htm
vukcevic says:
January 23, 2014 at 1:39 pm
Here it is Aurora and aurora data alone
You claimed that aurorae and ‘magnetic’ oscillations of the Earth itself were related. Show that. Aurorae and solar wind magnetic field are causally related but that is not what you claim. Pseudo-science piled on higher just makes it worse.
Willis, thanks for correcting my error.
What I am claiming it is that geomagnetic storms are modulated by small changes in the Earth’s magnetic field (generated by differential rotation within the liquid core).
You have to be patient, another university professor and a prominent scientist will have the first sight of entire endeavour.
You ascertain it is just a ‘pile of pseudoscience’, I guess you are being polite. For most of my practical engineering carrier,I dealt with data and signals, so science or pseudoscience aspect doesn’t bother me a single iota, but for the benefit of your serenity let it stay as pseudoscience. It is my bedtime; see you at another thread at another time. Keep well.
I hope I’m not being provocative, but your statements about correlation make no sense to me.
What does an autocorrelation function of ~0.8 actually mean?
How does this effect the “R^2” value?
I realise that I am being incredibly stupid but I can quite easily envisage signals that has an “aurtocorrelation” of 0.999 and an R^2 of 0.000. Alternatively I can envisage a signal that has an ” autocorrelation of 0.000 (ideally) yet can have an autocorellation of 1.0.
I realise that I cannot match your brilliance, but I would be very interested in your explanation.
P.S.: I see you have discovered “Mathematica”.. God help us all! I would point out that this is a symbolic language for mathematical manipulation – it is not a substitute for understanding (or hard work)
vukcevic says:
January 23, 2014 at 3:20 pm
What I am claiming it is that geomagnetic storms are modulated by small changes in the Earth’s magnetic field (generated by differential rotation within the liquid core).
You used to claim just the opposite: that the field in the core is modulated by geomagnetic storms, i.e. causality going the other way.
Now, on very long time scales [centuries] the intensity of geomagnetic storms is modulated by the strength of the geomagnetic dipole, as the latter determines the stand-off distance of the solar wind above the surface of the Earth and hence determines the size of the magnetosphere. The tilt of the Earth’s dipole against the solar wind direction introduces daily and seasonal variations [up to 30%] of the size of magnetic activity [and thus also of geomagnetic storms] as the solar wind sees a varying strength of the Earth’s magnetic field at the ‘sub-solar’ point as the Earth wobbles. This effect accounts for the well-known semiannual and UT-variation of geomagnetic activity http://www.leif.org/research/Semiannual-Comment.pdf but I don’t think this is what you have in mind as those variations average out over a year.
RC Saumarez says:
January 23, 2014 at 3:43 pm
What does an autocorrelation function of ~0.8 actually mean?
How does this affect the “R^2″ value?
A high autocorrelation means two adjacent data points are almost the same [and so are not independent] so that when you think you have 1000 data points you may only have [say] 50 points. this affects the p-value making you believe that the correlation is more significant than it really is. A good example is the sunspot number: in a solar cycle there are 4000 days and thus 4000 daily values of the sunspot number, but since a high sunspot number on a given day is always followed by a high number the next day, the number of independent data points for a whole cycle is only about 25, not 4000.
@Lsvalgaard
Why not consider the correlation between a sine wave and cosine wave?
Alternatively, why not consider the corellation between a random sequence and a time shifted random sequence?
RC Saumarez says:
January 23, 2014 at 4:26 pm
Why not consider the correlation between a sine wave and cosine wave?
Alternatively, why not consider the corellation between a random sequence and a time shifted random sequence?
Yes, why not? Demonstrate your eloquence by elaborating on this as you feel the need for…
RC Saumarez says:
January 23, 2014 at 3:43 pm
It means that if the dataset is duplicated, and then lagged by 1 timestep, the correlation of the dataset with the lagged version of itself is 0.8.
I have no idea about your brilliance, you’re a better judge than I. Setting that aside, autocorrelation doesn’t affect the R2 value at all. What it affects is the p-value. This is because autcorrelated datasets (also called “red noise”) tend to “wander” much more than “white noise” datasets. As a result, if we are trying say to determine if a trend is a random occurrence (p-value), autocorrelated datasets are much more likely to contain random trends, and to move in cycle-like swings. Note that these are not real cycles … but they can certainly resemble them.
To put it another way, if we see a big swing in what appears to be a white noise dataset, there’s a good chance that there is some underlying phenomenon at work. A similar swing in a highly autocorrelated dataset, on the other hand, could much more easily be just random chance.
“Discovered”?? Listen, you snide little twit. I’ve owned and driven a copy of Mathematica since Steven Wolfram came out with Version 1.0 in 1988, so I’ve used it for a quarter century now. Over that time I’ve created programs in Mathematica to provide the code to run the plasma cutters that cut out all the steel plates for the construction of four 40 metre ships, along with dozens of other real-world, “failure is not an option” type of practical tasks, plus a variety of more theoretical uses. I can program Mathematica in three languages, while you were likely unaware that there is more than one way to program it. And yes, Virginia, I also know that it also does symbolic manipulation, and I’m good at that as well. Come back when you can say the same.
I don’t advise you try your tongue on me, RC, or you’ll get a tongue-lashing. I’m willing to move on past this insult of yours, but if you’d like to continue the conversation …
w.
I want to say a few more words about “obvious” patterns. Someone commented above:
Someone else said yes, there is a pattern, it’s so clear that an eight-year old can see it.
Humans have succeeded as well as we have in part because we are superb at recognizing patterns, and learning from them. It’s one of our more amazing skills. The downside of this is that we often ascribe patterns where none actually exist. By that I mean the pattern may exist, but it’s just there by chance.
As examples of this, I offer the constellations in the stars, and the faces in the clouds.
We see both patterns … and they are definitely there, even an eight-year old can see them.
But they have no meaning. They are not a clue to an underlying reality, as many patterns are.
To deal with this conundrum, to assist us in determining which patterns are indications of something deeper and which patterns are just faces in the clouds, we invented a kind of math called statistics.
For example. Suppose I pull out a coin, flip it four times, and each time it comes up heads.
Does that mean the coin is weighted to produce heads? In other words, does that pattern have meaning, or is it random?
With math, we go OK, one chance in two of getting heads if the coin is random.
One chance in four of flipping two heads in a row.
One chance in eight of three heads.
One chance in sixteen of four heads.
So we know that IF the coin is random, there is one chance in sixteen that we’ll get four heads in any given run of four flips.
So … is the result, four heads in a row on the very first try, significant or not? That’s only half a question. The other half is the level of significance we want to see before saying yep, it’s real. For climate science the standard is very easy to achieve, one chance in twenty.
As a result, four heads in a row would NOT be considered statistically significant, because with a fair random coin, we would expect that outcome once in sixteen tries.
However, if we flip the coin again, and get heads, the odds of that are one in thirty-two. And that WOULD be considered a statistically significant result in climate science. Like I said, it’s a low threshold.
However, the sea level – sunspot correlation above, DESPITE THE APPARENT PATTERN, couldn’t even get over that low bar.
There is another curiosity that bears emphasis. If the chances are one in 32 of five heads in a row … if we do say a dozen runs of five flips each, what are the chances of five heads in a row coming up in one of the runs?
To do this, we multiply the odds of it NOT happening (31/32) times itself 12 times … and subtract that from 1. That gives us 1-(31/32)^12 = 32% chance of finding a five-head run in twelve tries. In fact, there’s a simple rule for finding a relationship that is individually statistically significant at the one in twenty level—just keep looking.
As Robert Brown said above, this can be a huge unrecognized problem. If a researcher hypothesizes a relationship, and looks at one dataset to see if it is real, the appropriate level of significance in climate science is one in twenty.
But if the researcher doesn’t find it, and looks at say five other datasets where the effect might be noticed … at that point, the odds of finding a “1 in 20” occurrence are down to one in four … and so now, in the six datasets, the researcher needs to find a signal with odds such that (( x-1 ) / x )^6 = 0.95, which solves to odds of about one in a hundred …
This comes into play, for example, if the researchers have free choice among the datasets to be selected for analysis. For example, the two sea level datasets used by Shaviv and by the author of the study in question are made from twelve and nine tidal station records respectively … out of a few thousand possibilities.
Finally, many natural datasets are like the surface of the ocean. There are a variety of waves that appear, and have strength for a while … but then they fade out and some other wave of a different height and period takes over … then that wave slowly both lengthens and decays, and there is a period of calm, which we notice and name the “Maunder Minimum”, and then another new and different wave starts building up …
My general point is, natural datasets are like trailer-trash ex-husbands. My best advice is, DON’T TRUST THOSE SUCKERS, THEY ARE LYING TO US JUST LIKE LAST TIME! It is treacherously easy to chase such will-o-the-wisps forever. It is ludicrously simple to drop a trend and a sixty-year cycle onto the HadCRUT temperature data and declare there is a relationship, even an eight year old can see the pattern … but just like a celebrity marriage, that lovely relationship likely won’t last.
So in response to those folks who insist that the pattern is there before their eyes, I cannot dispute that any more than I can dispute a face in the clouds.
However, only the math can let us know whether it is significant … and even there, we find lots of hidden pitfalls.
My best to all, this continues to be a most interesting discussion.
w.
PS—having now looked at the Shaviv dataset, and the Church and White dataset, it’s clear that the sea level data won’t bear the weight of the conclusion that sunspots correlate with sea level. I also took a quick look at the CERES data, no relationship there either. So the math seems to have been right, it was either a spurious or a very weak correlation.
Come on Willis, I think you ignore other’s input. If you would do your maths with CO2 concentration (instead of sun spots) versus sea level, what r value would you get ? Would you also conclude the correlation is very weak or spurious ? Would you compute r with the noisy unsmoothed data set ? If you would have looked at above plot a month ago, before that mess, would you really have computed an r value without further effort of extracting the “pattern” ?
Leif, do you have a link to whatever might be the longest observationally based and count-corrected sunspot dataset?
Thanks,
w.
Don’t know if anybody is still reading here, perhaps everything has been said, except perhaps, that Shaviv computed an r value of 0.54 giving a p=10E-4 (even without further pattern matching / signal extracting effort).
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.2162&rep=rep1&type=pdf
Otherwise I would totally agree with Paul Westhaver / Ian Schumacher.
I comment again, because I would love to see an audit happen where the meat of the papers is (or should be), which is the planetary connection. And not correlations which have been commented numerous times at WUWT. (And in my view support the existence of a solar amplifier or quoting Steve McIntyre first impression of above plot again “…offhand, I can’t think of any two climate series with better decadal matching…”)
And the planetary connection here may be really interesting to discuss, as it is claimed as follows in the paper:
“The coastal sea level variation cannot be explained as due
to expansion/contraction of the oceans due to heating/cooling
during a solar cycle as proposed by Shaviv (2008) simply because,
near the shore, the thermal expansion becomes zero
since the expansion is proportional to the depth (Mörner,
2013b). The good correlation and nearly in-phase response
between solar activity and sea level indicates that this is a direct
mechanical response – and not a thermal response that
needs time to heat up and cool, and therefore shows delayed
response.”
So they disagree with Shaviv (!) and say the expansion is not a thermal response but mechanical.
I would be interested in an expert comment on this part “….simply because, near the shore, the thermal expansion becomes zero since the expansion is proportional to the depth (Mörner, 2013b).”
The Moerner paper is paywalled, so I could not check the reference, but in my amateur’s view, I would have thought that expansion occurs instantly and that such expansion levels out quickly. The volume would expand in x,y,z and x,y would push water into the coastal regions immediately, while the z expansion may take longer as it is leveled by gravitational force.
Manfred says:
January 23, 2014 at 7:11 pm
~ 0.00.
Sure. However, it’s a very different problem because the increase in CO2 is so highly autocorrelated, and is also essentially a straight line. As a result, establishing significance between CO2 and anything is highly difficult … hence the unending search for a CO2 “fingerprint” that to date hasn’t appeared.
Definitely. If your math is done correctly, it shouldn’t matter. You gain R^2 by smoothing, but you lose significance because of high autocorrelation. If it’s not significant unsmoothed, odds are good that it’s not significant smoothed. Following the guidance of the estimable William Briggs, I make a practice of never smoothing data used to calculate significance.
The other problem is that smoothing can actually introduce bogus, spurious correlations. I wrote a post on that some years ago, hang on … OK, six years ago, here you are.
It’s not the first time I’ve seen variations on that theme, and I wasn’t any more impressed the first time than I am now.
More to the point, I’ve looked at too many natural datasets to be impressed by that kind of thing. Heck, one reason I wrote about this was that I looked at it and estimated by eye that there was no statistically significant relationship … which turned out to be the case.
w.
Willis Eschenbach says:
January 23, 2014 at 7:12 pm
Leif, do you have a link to whatever might be the longest observationally based and count-corrected sunspot dataset?
We are still working on hammering out the ‘final’ series, but it will be close to the series you get if you use the Official SIDC numbers and then increase all values before 1947 by 20%.
Our deliberations are detailed here http://www.leif.org/research/CEAB-Cliver-et-al-2013.pdf
Manfred says:
January 23, 2014 at 7:32 pm
Don’t believe everything you read. I just digitized Shaviv’s data, and compared it to the Lean TSI data from here … R2 = 0.09, p-value 0.09 … not sure why the difference. Perhaps it’s the use of a different TSI set … which just reveals the weakness of the method.
Spreadsheet here …
w.
Hi Richard.
A bit snarky?
Listen, an 8 year old child looked at the plot at my request and I asked the child what he saw. He said about the plots: “they are doing the same thing”.
When an 8 year old sees something and he tells you that the emperor has no clothes, you might want to listen. Maybe the naked emperor is a fool and his sycophants don’t have the courage to tell him.
I call them like I see them.
I see a pattern of some sort and no amount of bullying and browbeating from you will change that.
You are quite a tyrant want-to-be.
lsvalgaard says:
January 23, 2014 at 3:49 pm
………….
Yep, it appears to be both ways; a feedback positive or negative depending on the phase at the time of re-encounter at the pole. Perhaps you should revive the Svalgaard –Mansurov effect for the energetic solar events, apparently the ordinary SW has not sufficient energy.
Next step would be to apply the same process to the Ap index to see if it reproduces the signal, idealy data from a single polar region station, e.g. Thule if you got a long enough set.
Will keep you informed.
As many have mentioned, the apparent correlation between the two graphs is quite dramatic. This is what Steve McIntyre said in 2007, when this graph was discussed at CA:
“The maxima and minima of the solar cycles seem to match the fluctuations in sea level rise rather uncannily. While the resemblance is impressionistic (I don’t have a digital version of Holgate’s series), offhand, I can’t think of any two climate series with better decadal matching. I think that this resemblance is pretty obvious.”
http://climateaudit.org/2007/02/11/holgate-on-sea-level/
If Holgate’s data is good, then the SLR shows a very distinct periodicy with a period close to 11 years. I’m not a statistician, but I would be fairly confident that the probability of this being purely random is vanishingly small. If it’s not random then there has to be a cause, pretty well by definition.
In my original post I specifically noted that the eye can be deceived. But in many cases the human eye can be far more true than vast constructs based on dodgy computer models – there have been many examples of this at WUWT, and some of them probably from Willis! In one example – a paper used incredibly complex computer model constructs to demonstrate a claimed link between the UK 2000 floodings and climate change – a simple eyeballing of the actual data clearly showed there was no overall trend.
Shaviv’s paper has been mentioned:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.2162&rep=rep1&type=pdf
Shaviv uses different data but the result is very similar. He gets an R2 correlation of greater than 0.5, which is much more believable. Hopefully he had the original data and did not need to scan the graphs. His explanation of the effect, if I understand correctly, is that the solar cycle effects ocean surface temperatures (the top 10 meters) and this in turn effects SLR through expansion.
Is there a real and provable relationship between solar activity (in this case, sunspots) and SLR?
I don’t know. But Holgate’s data clearly shows that there’s more work to be done. That’s what makes science so fascinating.
And, yes, I agree wholeheartedly with Willis that the only thing that matters in the final analysis is mathematics, evidence and proof.
I did mention that doing a Monte Carlo analysis would be very useful, it would strongly indicate the probability that the apparent correlation was caused by chance. Any takers?
Chris
lsvalgaard says:
January 23, 2014 at 3:49 pm
………..
Hi again
I did a quick run on the Ap data; resulting graph is now added to the one from the aurora data
http://www.vukcevic.talktalk.net/OldLadysHalo.htm
It confirms previous findings, significance could be important if a physical process can be identified correctly (i.e. if indeed there is one !).
I am surprised that no one has done it before since Ap was accurately measured since (was it ?) 1930s.
Some further observations from the Old Lady’s Halo (Old Lady = Terra, Halo = Aurora) graph
1880 -1925 there is reasonable agreement pulse for pulse for both down and up short term trends.
1925 – 1945 it is a mess, I suspect due to strongest geomagnetic jerk in recorded data:
http://www.geomag.bgs.ac.uk/images/image018.jpg
peaking in 1925
1945 – 2010 again there is reasonable agreement pulse for pulse for both down and up short term trends, albeit amplitudes of pulses in the temperature domain are considerably less prominent.
Since I assume that Ap measurements have maintained same standard, although decline in the strength of of the Hudson Bay magnetic pole has not been fully compensated by the slow rise in the strength of the Central Siberia one. The temperature side of the equation there are number of factors (from CO2 to data processing methods) which could explain rising discrepancies.
R2 for any of the above three time periods is negligible, but these type
Just to add: 3 year moving average is used in the temperature domain.
Paul Westhaver:
Hi Paul:
re your post at January 23, 2014 at 10:13 pm.
All “snark” was yours.
I took the trouble of explaining the issues to you twice and your response was – and is – that my explanation must be wrong because an 8 year old child sees the pattern you can see.
I twice agreed that you did see a pattern but I explained that the pattern is meaningless. Willis also explained that to you.
But you persisted – and still persist – in claiming my understanding and explanation should be ignored because an 8 year old child can see a relationship. That is an insult. My offence at the insult does not indicate I am “quite a tyrant want-to-be”.
It only indicates that I am offended by gratuitous insults and respond appropriately.
Richard
Chris Wright:
Many things oscillate naturally and some could have a periodicity close to ~11 years by chance. Hence, observation of something having similar periodicity to the solar cycle periodicity does merit investigation of whether that similarity is or is not a chance coincidence.
At January 24, 2014 at 2:47 am you say
Why would it be “vanishingly small” when many things vary with different frequencies? These two have similar frequencies.
Of importance is WHY the SLR varies as it does.
J E Solheim published a paper which suggested that the SLR variation is related to the solar cycle. In other words, the solar cycle causes the SLR variation. And, as you say, their similar periodicities does imply that the SLR is solar driven.
But the analysis by Willis strongly suggests that the similarity of the SLR oscillation is purely by chance. I summarise his findings above at
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1546134
Willis’ analysis certainly indicates no direct causal effect although there could be an indirect effect. Such indirect effect would result from the solar cycle determining the periodicity of ‘something else’ which – in turn – affects the periodicity of SLR. But nobody has suggested what such a ‘something else’ could be.
So, at present, Willis’ analysis indicates that there is no relationship between the solar cycle and SLR variation. Hence, the hypothesis of a direct causal relationship between the solar cycle and SLR variation is falsified.
This is a useful finding because it frees people to investigate whatever is the true cause of the SLR variation.
Richard
‘richardscourtney’ says:
‘J E Solheim published a paper which suggested that the SLR variation is related to the solar cycle. In other words, the solar cycle causes the SLR variation. And, as you say, their similar periodicities does imply that the SLR is solar driven.’
NO! Must read: [..] In other words, the solar cycle AND the SLR variation MIGHT have A COMMON CAUSE. full stop
Whatever that cause might be – tidal forces of the planetary system and/or even electromagnetism … etc. – the effect is verifiable in the record. At earth: as SLR. At the sun as SSN.
The COMMON CAUSE has yet to be discovered.
Jan Stunnenberg says:
January 24, 2014 at 9:20 am
…………..
There are number of articles positing that the geomagnetic storms affect the global atmospheric pressure, there is also something called Svalgaard-Mansurov effect ( which I think is real but its discoverer think it is not).
Atmospheric pressure changes will affect the rate of sea level change.
If you compare geomagnetic disturbances to the sunspot cycle than there is considerable discrepancy between the two.
http://www.esa-spaceweather.net/spweather/workshops/proceedings_w1/POSTER4/figure_01.gif
So Perhaps Willis could look at the geomagnetic data, but my concern is that changes of few mm are well within margin of error.
vukcevic says:
January 24, 2014 at 10:02 am
There are number of articles positing that the geomagnetic storms affect the global atmospheric pressure, there is also something called Svalgaard-Mansurov effect ( which I think is real but its discoverer think it is not).
You are confusing the Svalgaard-Mansurov effect which is very real with the Mansurov-effect which is not.
If you compare geomagnetic disturbances to the sunspot cycle then there is considerable discrepancy between the two.
It is wrong to call it a ‘discrepancy’ as the difference itself is related to the solar cycle and is due to the prevalence of coronal holes on the declining branch of the cycle. All this is well-understood.
OK, bos, I stand corrected on the difference.
Svalgaard-Mansurov effect is related to the polar cap magnetic deflections, associated with ionospheric currents flow resulting from the release of magnetic tension on newly open magnetic field lines.
The Mansurov effect refers to North–south asymmetry of geomagnetic and events with effect on the troposphere.
It would resolve many problems if Mansurov happen to be correct.
Burns et al 2007 do think Mansurov effect is real
The Mansurov effect, which for the Southern Hemisphere consists of a positive association between the By component (east-west) of the interplanetary magnetic field (IMF) and the ground-level pressure for stations poleward of ∼80° magnetic latitude, is confirmed for Vostok (78.5°S, 106.9°E; magnetic latitude 83.6°S) using modern data. The magnitude of the association is small (0.19 hP per nT; 1.2% common covariance) but statistically significant (at the 96.1% level). A more substantial association exists, with a slight delay (2–3 days) and a cumulative influence, between the
………….
We confirm a previously reported Sun-weather linkage (the Mansurov effect), provide evidence that the mechanism operates via the atmospheric electric circuit and present data supporting an inferred and more substantial surface pressure response to changes in the global atmospheric circuit.
http://onlinelibrary.wiley.com/doi/10.1029/2006JD007246/abstract
If there are geomagneticaly induced currents at altitude of 10-15km, how could one be certain that the atmospheric events are unaffected.
http://www.vukcevic.talktalk.net/GEC.jpg
vukcevic says:
January 24, 2014 at 11:25 am
If there are geomagneticaly induced currents at altitude of 10-15km, how could one be certain that the atmospheric events are unaffected.
Since there are not, this is a non-issue.
Richard,
You had not contributed anything to the rational explanation as to why a perceived pattern is not real. Only ad hominem to me.
Your lack of a an attempt at any possible reasonable hypothesis as to the basis of why a pattern is commonly perceived yet is absent in W.E’s analysis or yours, is the reason I am unconvinced by his and your assertions.
Why don’t you comment on whether my assertion that a simple Fourier type adjustment of each data set might reveal an error?
I argue that Willis really hasn’t done the data set justice. He simply ran a comparison and tossed it out, out of hand. Seems to me that based on the Sunspot number approximation, where he got ok mapping, then he ought to be able to do the same with these 2 data sets. Then compare and resolve the adjustments.
How about that Richard? Follow me?
Paul,
A PERCEIVED pattern is a matter of what you see coupled with the human brain’s tendency to recognize patters, WHETHER OR NOT THEY REALLY EXIST.
For example – the constellations. The stars have no pattern. But we PERCEIVE a pattern, regardless.
The degree of matching of two patterns is NOT something that is visually determined. It is done mathematically. That is to prevent perception from biasing the results. Perception is SUBJECTIVE, and differs from person to person. For example, I DO NOT see the correlation that everyone else appears to see. But that only MY perception. What matters more is the mathematical correlation. The mathematical calculation of the correlation is OBJECTIVE, and it has been done. According to the math, there is NO correlation.
What you’re doing is no different than people who see Jesus’ face on their breakfast toast, or dragons in the clouds.
lsvalgaard says:
Since there are not, this is a non-issue.
………
Andreas Baumgaertner from UCAR thinks there are : Page 11
http://sisko.colorado.edu/FESD/MeetPresentFiles/Jul1_2013/Baumgaertner.pdf
Willis,
You sound pretty confident that there is no common root to the two data sets.
I don’t agree with you.
I don’t think you tested the data sets creatively enough to expose the pattern error.
It is the ERROR between the two data sets that reveals the key.
Professionally I do this all the time. I just am too busy to tackle this set just now. Signal analysis is my area of interest.
I would force that data onto a strong correlation by adjusting the amplitudes, phases and linear trends etc.then examining the error pattern of a series of best fits.
All of the information is hidden in the outliers and the errors.
I think you prejudiced your analysis by categorizing the apparent agreement as a perceptual aberration, corrupting your objectivity. Also, you assumed that there is cause-effect relationship, which would mask an intermediate.
Why don’t you come back to it later after some reflection. You might have a Eureka moment?
PW
vukcevic says:
January 24, 2014 at 12:37 pm
Andreas Baumgaertner from UCAR thinks there are : Page 11
You have to learn to accept what I teach you. The induced currents [and the curved arrow] refer to currents induced 400 km below ground, not at 10-15 km altitude.
Tony G,
Perception of something like a face is hard-wired in the brain, apparently. I am not a neuro psychology expert. I will not dismiss that people are good pattern finders. We are, as a general rule. Galileo notice a pattern in a oscillating chandelier and from that he derived a rule for the period of the pendulum. Good thing his wasn’t distracted by the cat-calls of his detractors. History is replete with people who are mocked because they recognize a patterns before others.
My favorite example is Georges Henri LeMaitre.
Rather than suggesting that the there is a cause and effect relationship between sun-spot number and sea level change I suggest that they share a common root of some sort.
Maybe ocean evaporation rates vary or terrestrial drought vary. We know that cloud formation rates vary based on cosmic ray production which is related to the sunspot cycle. Maybe something else. Clearly the earth is affected by sunlight in many ways.
According W.E.s 1st attempt at a correlation he see no relationship. Fine.
WE is one guy. The Job may be for someone else who is less interested is dismissing the data sets and more curious at picking beneath the paint, so to speak. As always, I vote in favor of curiosity and against people who claim “settled science”.
Tony G. you should abide by Roger Bacon’s (Opus Maius 1267) 4 obstructions to discovering the truth which are:
1) the example of weak and unreliable authority;
2) continuance of custom,
3) regard to the opinion of the unlearned, and
4) concealing one’s own ignorance, together with the exhibition of apparent wisdom.
Quatuor vero sunt maxima comprehendendæ veritatis offendicula, quæ omnem quemcumque sapientem impediunt, et vix aliquem permittunt ad verum titulum sapientiæ pervenire: videlicet fragilis et indignæ auctoritatis exemplum, consuetudinis diuturnitas, vulgi sensus imperiti, et propriæ ignorantiæ occultatio cum ostentatione sapientiæ apparentis.” [Roger Bacon. (1267). Opus Maius.
Bravado is the first betrayer of #4.
Again I see something and I can’t say what it is. But I see something.
Say what you want. I see it.
PW
TonyG says to To Paul Westerhagen:
‘According to the math, there is NO correlation.’
Wrong, it’s only Willis’Math that did so.
Paul Westerhagen seems clever enough to me, to be able to ‘feel’ uncomfortable with that kind of ‘math’.
Usoskin & Korte:
We conclude that changes of the regional tropospheric ionization at midlatitudes are defined by both geomagnetic changes and solar activity, and none of the two processes can be neglected.
The color purple doesn’t exist. Yet we see it still.
You cannot find the color purple anywhere on the visible light spectrum.
Our brains adequately decipher red, ted to yellow to blue very well and yield us a good approximation of the spectrum even though we only have blue red and green receptors.
Purple is a color invented by our brains to resolve the difference in intensity by non-adjacent spectrum receptors in the absence of an intermediate.
So, even though purple is not real, the ratio of the two colors that make purple is real and quantifiable. So purple is real…? but as a difference calculator.
Maybe the data sets are like that. Willis never checked for something like that.
Sorry Paul Westerhagen.
I’ve just submitted my comment as you submitted yours.
However your’s tells it all.
vukcevic says:
January 24, 2014 at 1:49 pm
Usoskin & Korte:
We conclude that changes of the regional tropospheric ionization at midlatitudes are defined by both geomagnetic changes and solar activity, and none of the two processes can be neglected.
As you have a tendency to misunderstand things, you should provide a link so we can see what you misunderstood. Now, I can guess where your confusion comes from: The main sources of electric fields and currents in the Global Electric Circuit are thunderstorms in the troposphere and the dynamo situated in the ionosphere and magnetosphere produced by tides generated in there and tides propagating upward from the lower atmosphere. Again, learn from what I say, rather than digging your holes ever deeper.
vukcevic,
I speculate that what happens on the earth may be the result of solar activity, Radiation, solar wind, solar farts, etc… all that stuff, seen and unseen and yet-to-be-discovered.
Is there a good study of cloud cover (anywhere/everywhere) vs sunspot number (or a proxy)?
I suspect that there is a transfer function of some kind between solar activity and terrestrial variables, like sea level change etc. Maybe the TF is an aggregate of all of these. Why not?
PW
Willis : “Here are the Jevrejeva annual sea level changes plotted against sunspots …”
Where did you get that data from Willis ?? Did you digitise fig 3 from the paper by any chance. If you did you should read the caption: it is the SSA 30year window analysis, which they say is similar to 30y running average. That may explain the lack of any decadal detail !
Data should be available from PMSL but it’s not responding my end right now.
http://www.psmsl.org/products/reconstructions/gslGRL2008.txt
Suggest KNMI
http://climexp.knmi.nl/getindices.cgi?WMO=PSMSLData/gsl_ann&STATION=global_sea_level&TYPE=i&id=someone@somewhere&NPERYEAR=1
A high autocorrelation means two adjacent data points are almost the same [and so are not independent] so that when you think you have 1000 data points you may only have [say] 50 points. this affects the p-value making you believe that the correlation is more significant than it really is. A good example is the sunspot number: in a solar cycle there are 4000 days and thus 4000 daily values of the sunspot number, but since a high sunspot number on a given day is always followed by a high number the next day, the number of independent data points for a whole cycle is only about 25, not 4000.
C centered on 1986, one point at an anomaly of 
C centered on 2005. The probable error (accounting for the scaled autocorrelation and hence sample independence on a smaller scale WITHIN these intervals pretty much completely overlaps and suggests a mean warming of perhaps 0.2/C over 18 years, or roughly 1.1 C/century if it made the slightest bit of sense to extrapolate two data points in any graph ever built!
Excellent summary. I wrote a whole Phys. Rev. paper on this once upon a time, elaborating (among other things) how one can actually determine the autocorrelation time (or number of samples per “independent” sample) by looking at the scaling of the variance of the data. It is easy to make egregious claims for the precision of an entirely spurious experimental result if one has thousands of samples but the scaling of the variance indicates that you really only have a hundred.
I was actually doing it the other way around — I was doing numerical simulations using (and comparing) several Markov Chain processes — Metropolis, Heat Bath, a cluster/metropolis method and a mixed cluster/heat bath method. Metropolis was cheap and generated enormous numbers of samples, but the samples had an enormous autocorrelation as one might expect. Heat bath did better — it isn’t accept/reject and so every spin in the model was changed in every step. Cluster methods alone did break up spatial correlations quickly but — surprise! They were accept-reject AND preserved large blocks of spin bonds in unchanged states per move, and actually slowed down the decay of energy autocorrelation (accept/reject methods more or less guarantee that SOME bonds in a lattice are not thermalized for many, many sweeps). Best of all was a mix of cluster steps to break up egregious spatial correlations at a variety of length scales and heat bath sweeps to facilitate the ergodic progression of the bond energies. All revealed by comparing how the standard deviation of the sample compared to what one would expect from a knowledge of the system variance and the number of samples in question.
This is one reason I’m very skeptical about climate science conclusions in general. There isn’t one autocorrelation time or important time scale in climate science, there are dozens (at least) — there is probably a continuum of them to where Laplace transforms are more appropriate to speak of than an single exponential decay process. And it isn’t clear that these times themselves are stationary; it may not even be that the usual concept of an “autocorrelation time” HOLDS for a climate system, even in an approximate sense.
That’s why I read papers on Hurst-Kolmogorov statistics with great interest. Many climate variables appear to exhibit an H-K pattern of autocorrelation — stretches of order decades of approximately uniform variation followed by a discrete jump to a new stretch of order decades. “The pause” is just such a stretch in the GASTA , where the 1997/1998 Super-ENSO is coincident with the preceding jump, and where quite a few of these intervals are visible e.g. here:
http://www.woodfortrees.org/plot/hadcrut4gl/from:1940/to:2014
or even more clearly here:
http://www.woodfortrees.org/plot/uah/from:1977/to:2014
or here:
http://www.woodfortrees.org/plot/rss/from:1977/to:2014
In the satellite era in the last two of these curves, note well that one does quite well imagining that the climate was locally stable from 1980 to 1998 give or take a year or two — call it twenty years. Then it jumped by around 0.3 C over a very short period — two or three years, with large oscillations attending the jump — and has been stable since for anywhere from 13 to 17 years depending on just where you want to put the jump and whether you want to discretize it or extend it over a 2-4 year period that includes the initial very sharp rise, the overcorrection, and the smaller scale bounces that could just as easily belong to the new equilibrium.
Similar jumps are clearly visible in SST graphs generated by Bob Tisdale. Similar jumps are clearly visible in rainfall patterns (one of the places where they were first observed and analyzed by hydrologists like Koutsoyiannis).
Guestimating (looking at longer timeseries and allowing for both poorer data and various thumbs on the scales) 15 to 25 years is one highly non-exponential autocorrelation time — the Earth seems to like to spend 1-2 decades going only slowly up or slowly down interspersed with intervals where it goes up or down comparatively rapidly. Much of even this cannot really be trusted — IMO global climate data itself has a clear trend of decreasing reliability as one goes back into the past (that no one ever gets to see as no one prints the curves with even guestimated error bars, but the estimated errors in things like HADCRUT4 now are pretty large, on the close order of the 30 year supposed warming “signal” of a few tenths of a degree).
If one reduced the last 36-odd years of satellite data — the most reliable data source out there for an unbiased global temperature anomaly — to the two independent “climate” samples it probably represents, and decorated each of those points with an error estimate of 0.1 to 0.2 C (based solely on the variance around the sample means of the observational data taken in two 18 year chunks) nobody would dare to say anything about a trend in the climate. One point at an anomaly of
The problem here is that we really do not know the various climate autocorrelation times (plural) and so we do not know how much “significance” to assign to any apparent linear trend in the highly correlated data with its own internal dynamics that is essentially noisy but stable for apparent decade-plus intervals.
To conclude, I hate to see Bayesian reasoning bad-mouthed, so I’ll just point out that the correct application of Bayesian reasoning in problems like this is along the lines of the stuff Willis was discussing, to automagically correct one’s initial prior estimates of probability until they are in asymptotic agreement with the observational data.
For example, one might examine as he did flipping a two-sided coin. You might begin with a strong bias, an experience-based belief that two-sided coins are likely to have a probability 0.5 for heads, 1 – 0.5 for tails, and 0 for landing perfectly on edge, making them perfect Bernoulli Trial objects with a nice binomial distribution of outcomes.
Bayesian analysis in principle gives one a way to systematically and nearly smoothly correct this belief on the basis of data as one starts to actually flip the coin. Initially one’s prior for heads might be 0.5, but after observing 78 flips in 100 flips, one would/should have a posterior probability that is much higher than 0.5. This is the flip side of ordinary p-values. The Binomial distribution might allow one to compute the probability of getting 78 heads out of 100 flips of an unbiased coin — the usual definition of the p-value — but it does not tell you what your best estimate for the probability of heads is given the data.
The problem with Bayesian reasoning is that it is often misapplied or used as a means of legitimizing an improbable estimate. By weighting your prior beliefs highly, you can essentially demand a lot of evidence — probably far too much evidence — before you start bending your posterior estimate much. There are right and wrong ways to do it, but in the hands of a clever person Bayesian analysis can conceal a kind of statistical lie that resolves to “the result of my analysis shows that the probability of the event is p, very near where I expected/wanted/needed it to be”. Taleb reviews a very similar case in The Black Swan where a Scientist refuses to alter his prior estimate for the probability of a tail in the face of the observational evidence of 100 straight heads because he knows that 2^100 events can happen (one in 10^30 tries, give or take a bit) where Joe the Cab Driver immediately recognizes this as “a mugs game” — the coin is very, very probably fixed so that p(heads) is nearly unity, at least in the hands of the clever person flipping the coin.
This is very similar to the problems with concluding either correlation or causality from the data. Suppose I flipped a truly unbiased coin several thousand times. Over several thousand flips, it is very likely to see sequences of heads and tails at least 8 or 9 long. If somebody wishing to demonstrate that the coin is biased were allowed to pick samples out of this data and concentrate their analysis on some subset, all they have to do is point to one of these long sequences (which have low probability of happening in any given sequence of coin flips even as they will occur quite reliably if one honestly samples the unbiased coin) and say — LOOK — the coin produced ten highly correlated flips in a raw! The coin must be biased! Our brains make us especially susceptible to this — all of the mundane flips sequences are boring; even though the probability of getting HTTHTHHTHH. is exactly equal to the probability of getting HHHHHHHHHH, the latter is exciting and we notice it while forgetting all of the equally unlikely but less strikingly patterned sequences. Picking particular sequences of data that favor some desired conclusion is called “cherrypicking” and is a cardinal sin of science. The tendency to discover favorable sequences and cherrypick them is called “confirmation bias” and is a even worse sin as the latter can happen by accident but confirmation bias involves deliberate action e.g. presenting the average of 9 tidal stations that we know appear to have some sort of correlation but deliberately concealing — often even from ourselves — the fact that 9 stations picked at random from the full set of stations would exhibit no such visible correlation or worse, that the unbiased average of all of the data exhibits no such correlation.
Climate science (on both sides) research in much of the humanities and soft sciences are rife with confirmation bias and cherrypicking and overreaching conclusions drawn from inadequate or incorrect statistical analysis. A lovely example of the latter is the infamous hockey stick. An example of the former is D’Arrigo’s infamous testimony: “if you want to make cherry pies, you have to be willing to pick cherries”. Whether or not the misuse of statistics is deliberate or accidental, much of published climate science is statistically incompetent. A properly cautious approach would simply refuse to draw conclusions about trends, causes or effects until far, far more reliable data is collected and analyzed by somebody other than a collection of world-saving zealots — on either side. Let me be clear — we do not have sufficient evidence to reject the hypothesis of catastrophic warming by the end of the century (human caused or otherwise) any more than we have sufficient evidence to accept it. At the moment, the evidence in favor of the hypothesis is weak, but because of the uncertainties in things like autocorrelation, the range of so-called “natural” variation, the effect of a variety of processes that we know have timescales on the order of several decades (e.g. the multidecadal oscillations, the observed multidecadal variability of the sun, multidecadal oceanic turnover processes) it is far from sufficient to positively assert that CO_2-linked global warming will not, in fact, eventually cause real catastrophic e.g. sea level rise. There just isn’t any evidence that it is doing so yet.
For fans of proper Bayesian analysis, there is a delightful article here:
http://marginalrevolution.com/marginalrevolution/2005/09/why_most_publis.html
that analyzes a surprising but plausible claim: Most published research findings are false! This is probably not true in all fields (I’d like to think it isn’t true in physics, for example, because we police the field at roughly the level where any further stringency would eliminate the fringe from which new revolutions rarely but significantly spring) but in fields like medicine it is probably (and tragically) true!
It is interesting to see where climate science falls in the schema given at the bottom of this article.
1) I’d have to say climate science is very high in what he calls background noise, the number of hypotheses tested. Examples of this are frequent on WUWT — when every biologist has to add the magic words “to look for evidence of the impact of global warming” to a grant proposal to study pinkfooted bungie-jumpers, when every population study, every medical study, every study of coral reefs or the migratory patterns of beetles invokes the phrase on the presumption that such evidence exists (that has to be presented sufficiently compellingly to help influence a grant officer into funding the work when otherwise there is nothing particularly special to look for sheer Bayesian analysis dictates that a significant fraction of such studies will find something to report at the level of marginal statistical significance.
2) Many results supporting a supposed coming catastrophe are reported for tiny sample populations, or highly constrained and limited environments. Last week we heard about the supposed extinction of a subpopulation of a butterfly in a part of a single meadow subject to uncontrolled and confounding environmental alterations in addition to any supposedly discriminable change in climate. Again, many of these studies of marginal subpopulations or specific locations can easily turn out to be at best statistical accidents — happening to test green jelly beans first and never bothering to report that jelly beans in general have a null result, and even green jelly beans have a null result unless they were tested on one particular Sunday.
3) Small effects are to be distrusted. Wow. Talk about defining an entire science! The entire predicted catastrophe is less than a one percent effect on the absolute temperature scale. All observed warming to date in the sixty-odd years where it could be attributed to human-produced CO_2 is on the order of two-tenths of one percent of the absolute temperature (some fraction of which is almost certainly natural) and there has been no warming at all for the last quarter of that era precisely where one would expect it to be the strongest.
4) Multiple types of evidence are desirable, but are not forthcoming. The GCMs that are the sole basis for predictions of warming fail to predict global warming, they fail to predict tropospheric warming, they fail to predict rainfall, they predict storm intensification that has not happened, they do not predict the correct patterns of what warming there has been, and they utterly failed to predict what has happened with e.g. Antarctic sea ice, the Greenland ice pack, sea level rise, sea surface temperatures, El Nino, the PDO, and well, pretty much anything. It isn’t clear that there is anything that GCM’s have gotten right!
The problem there is that when multiple biological studies find supposed evidence for negative sequellae attributable to GW that supposedly took place over the last 15 years, that is not evidence for CAGW when in fact no statistically discernible warming whatsoever took place over the last fifteen years. So a huge body of what people consider to be “evidence” is in fact evidence — against the hypothesis, and for the presence of an enormous degree of confirmation bias in the literature given that people are finding effects for a cause that in fact has not budged since many of the researchers were in middle school.
5) This also is a problem here. Much of the literature can be discounted when they find evidence of global warming in e.g. the biosphere when no discernible global warming occurred or is occurring since the 1997-1998 ENSO, and only 0.2 C of warming occurred then. Changing “global warming” to “climate change” is intended to hide this since “climate change” cannot be falsified given that the climate is always changing, but does not repair the science. Then there is the deification of Mann by the IPCC and the consequent corruption of the mainstream science. Post-Mann climate science has been all about a literature controlled by the authors of a few individual papers, some of which have been shown to be terrible examples of statistics. Briffa, Jones and others published many papers clearly showing the MWP before Mann “erased” it. Climategate clearly revealed the naked backstabbing, gatekeeping, and behind the scenes pressure to suppress the competing voices that ordinarily keep science honest. It has taken fifteen years plus where the sky stubbornly refused to fall to gradually free the editors and referees to consider papers that don’t conform to a narrowly politicized conclusion as evidenced by the presence of one or more catch phrases about global warming even where it ends up being irrelevant (the mirror image of the grantseeking process alluded to above). One shudders to think about how long it will take to de-politicize the granting agencies themselves.
6) There are damn few papers published that test other people’s theories in climate science, at least as far as I’ve seen. If there were, would not the GCMs largely have been rejected at this point? WUWT has indeed covered at least one paper that tested GCMs directly against one another for a toy problem, where they failed miserably, but only a handful of researchers dare to question the party line, often at the expense of being called names by their peers and being subjected to a brutal and withering peer review process in journals where the editors are routinely subjected to pressure from a small cadre of climate scientists. Indeed, a properly skeptical test of the fundamental theoretical basis for the prediction of CAGW would be most welcome, as those predictions are busy not coming true and everybody knows it but nobody will say so in the literature!
7) Don’t reject papers that fail to reject the null hypothesis. Boy, Climate Science in a nutshell, at least if the null hypothesis is that human CO_2 has had a negligible effect on the climate, one impossible to disentangle so far from natural climate variation and impossible to directly observe even with sophisticated instrumentation and hence discernible at best by dubious analyses that attempt to discriminate recent warming as being due to CO_2 from previous but identical warming that was supposedly natural, e.g. comparing the first and second half of the 20th century.
If a non-expert in the field cannot tell the difference between GASTA in the first half of the 20th century (when most climate scientists agree that CO_2 levels had not yet begun to change at a significant rate from pre-industrial levels) and the second half of the 20th century (where all or almost all of the warming is openly attributed to CO_2 in IPCC ARs) then one has little basis for rejecting the null hypothesis.
rgb
I posted this a couple of days back but it seems to have disappears 😕
Anyway I’ve cleaned the graphs up on posted as description of the derivation.
Cross-correlation:
http://climategrog.wordpress.com/?attachment_id=760
Power spectrum:
http://climategrog.wordpress.com/?attachment_id=759
There is a smallish but significant correlation between the Jevrejave annual MSL and sunspot area. This show clear evidence of a modulation due to interference pattern with a second periodicity. The other frequency causing the modulation can be estimated by:
1/79.1+1.10.49 = 9.23 years.
This suggestive of a lunar cycle. Periods of 9.1 +/- 0.1 years have been reported by Nicolas Scafetta and Berkeley Earth project.
I have been tyring to point for some time that you cannot detect or refute an particular driver (like solar) by trivial single variable regression of correlation.
Willis challenged me to show it and I think that does.
Greg Goodman says:
January 24, 2014 at 4:03 pm
“I have been tyring to point for some time that you cannot detect or refute an particular driver (like solar) by trivial single variable regression of correlation.”
Throw out all the other comments on this thread. That is the only one needed.
Jan Stunnenberg:
In my post at January 24, 2014 at 7:34 am I wrote
At January 24, 2014 at 9:20 am you have replied saying in total
The “must read” applies to you and not me.
Firstly, my words “at present” are true and have a meaning which your reply ignores.
Secondly, I said the possibility of a common cause exists but the study under discussion falsifies a direct cause.
Thirdly, you assert a common cause but provide no evidence for it.
So what? I see no reason to accept an unjustified assertion that something DOES exist when – as I pointed out – it may exist but there is no evidence for it.; e.g. the reason for fairies at the bottom of your garden has yet to be discovered.
Richard
Paul Westhaver:
Your post at January 24, 2014 at 12:29 pm begins saying
No! That is absolutely untrue except possibly that you may be “unconvinced”.
I provided no ad homs.
The only ad homs. were from you at me; e.g. untrue assertion that I used “snark”, my explanation is less cogent than the perception of an 8 year old child, I am “quite a tyrant want-to-be”, etc..
Both I and Willis explained to you that the human brain often generates the perception of patterns which do not exist so mathematics are needed to determine if perceived patterns are real. If you want to continue your pestering then try it on Willis because his tongue will be more lashing than mine and I cannot be bothered to reply to more of your insulting and offensive twaddle.
Richard
rgbatduke:
Your long but excellent post at January 24, 2014 at 2:47 pm contains much ‘good stuff’. One sentence jumped out at me because only yesterday I replied to a series of questions put to me on WUWT and I provided an answer which effectively said the same; i.e.
Indeed, I said that to my questioner, I explained why it is true and I suggested further study that would assist the questioner’s understanding of it. But the questioner then responded with a post which tried to pretend I had said that there would no global warming! I suspect his intention was to induce me to say that and when he failed he tried to pretend I had.
Reality is important. What we want reality to be is not important. But as this thread clearly demonstrates, some people want to pretend reality is as they ‘see’ it and not as available evidence indicates.
Richard
Richard: “Reality is important. What we want reality to be is not important. But as this thread clearly demonstrates, some people want to pretend reality is as they ‘see’ it and not as available evidence indicates.”
Indeed, and other some people want to pretend reality is as they ‘see’ it and do not understand what the available evidence indicates.”
http://climategrog.wordpress.com/?attachment_id=760
Greg Goodman:
At January 24, 2014 at 5:35 pm you say to me
I agree.
And I observe that your link provides some debatable data which its presenter says only “suggests” something and rightly does not claim it “indicates” anything.
Richard
More ad hominems.
I see what I see…
richardscourtney says:
January 24, 2014 at 7:34 am
But the analysis by Willis strongly suggests that the similarity of the SLR oscillation is purely by chance.
——————————————————————-
Richard,
if curve fitting is not regarded as science, computaton of a single p value is hardly an “analysis”, which “strongly” suggests something without having a “look” at more specifics and details.
That language is not matching the evidence brought forward in support.
And readers notice that, and rightfully may get annoeyed, because they are not used to be manipulated like that, but enjoyed much more sophisticated and constructive discussions, many of them thanks to Willis.
And curiously, exactly that overdoing of an argument may be rightfully critizised in a few PRP papers as well. and it is therefore neither a serious-minded nor a clever option to citizise their work.
You say, maths is superior to visible inspection. Maybe, maybe often, but not in this case. The opposite is true.
Firstly, what you call maths is just a single value. And by visible inspection you may not only find out as well, that the p value may be low, but also why.
It is due to 2 phase shifts at the start of the data set and around 1990. Without these phase shifts, the datasets would be beating nearly synchronously all along the data.
The p value may still be low due to different amplitude patterns, but the beat is obvious.
The first thing of interest would be to investigate if the phase shift around 1990 is real. Comparing with Shaviv’s data set suggests, that it may be not, as in his data set the spread thereafter is smaller. Perhaps part of it due to Mt Pinatubo ?
The second thing of interest would be, if the beat continues with recent data.
The third thing of interest would be, if the beat is also visible in the satellite sea-level data.
Then you may be closer to conclude, if the relationship is thermal / mechanical.or due to a third common driver or, perhaps more likely “purely by chance”.
Manfred says:
January 24, 2014 at 8:06 pm
Manfred, I have also shown that the correlation gets much worse when we use the sea level dataset that Nir Shaviv used … and gets much worse again when we look at the global Church and White sea level record, and when we look at the Jevrejeva sea level records its worse yet …
In addition, Robert Brown has pointed to the effect of the tiny number of sea level records (N = a pathetic 9) on the p value, making their claim even more unlikely (even less statistically significant).
So your idea, that the only evidence is a single “p” value, is far from the truth. We’ve discussed a host of evidence, all of it pointing to the conclusion that the relationship is a random occurrence.
w.
Manfred says:
January 24, 2014 at 8:06 pm
Oh, I do love this argument, you’re about the third person to put it forwards, and I’ve been meaning to discuss it, thanks for reminding me.
You are saying that if you ignore all the evidence that opposes your theory, and throw away that part of the data that shows results in total opposition to your theory, then the data shows that your theory works perfectly … and you know what?
I agree with you about that 100%.
I’m just not sure you’ve thought your claim through to the end …
w.
PS—if you still don’t get it, consider what you would conclude from the data if your theory was that sea level moves in opposition to sunspots. In that case, you’d say if you just throw away the parts of the data where the two move synchronously, that without those phase shifts, the datasets would be beating nearly ANTI-synchronously all along the data. And once again, I’d agree with that 100%.
Which means that using your method, the data equally supports both a synchronous relationship of sea level with sunspots, and an anti-synchronous relationship with sunspots.
This is called a “proof by contradiction” that your method of analysis (throw out the data that doesn’t fit) must be faulty.
Willis,
Regarding R^2 equals correlation.
R^2 isn’t very good at helping you find components of a signal. The correlation I (and others) are referring two is of the signal processing variety (Sum of the product, not sum of the square of the differences). This allows you to be wrong in phase, in scale, and still ‘detect’ a correlation. In addition there is another type of correlation known as synchronicity. R^2 is only valid for linear models. Consider Milankovitch cycles ‘correlation’ with ice-ages. If you were to do an R^2 you’d probably get a very a very low value. This is because the Earth is a complex non-linear feedback loop system and correlation assumes a linear model. Whatever Milankovitch cycles do and how they do it, it is probably more of a ‘trigger’ then a direct driver. Blindly using R^2 you’d probably get a value so low that you’d assume that Milankovitch cycles are irrelevant. However, synchronicity is amazing. No rational person can look at the synchronicity of Milankovitch cycles and ice ages and conclude “it’s just a coincidence”.
Regarding dismissing the graph:
However, you also linked to some other data that was far less flattering for sun spots (both synchronicity and statistical correlation). As I’ve said I’m not a ‘sun spot guy’. I objected to your use of R^2 meant for linear models and dismissal of the graph above when to the eye there appears an obvious synchronicity. Does the synchronicity hold for longer periods? Who knows? I thought you were too quick to dismiss it based on linear model metrics. However, If other data shows that this paper is essentially ‘cherry picking’, then that is a different story.
Has this paper been accepted for publication?
Solar Forcing of the Streamflow of a Continental Scale South American River
P. J. D Mauas, E. Flamenco, A. P. Buccino
http://ruby.fgcu.edu/courses/twimberley/EnviroPhilo/Mauas.pdf
http://arxiv.org/abs/0810.3882
From the Abstract:
Solar forcing on climate has been reported in several studies although the evidence so far remains inconclusive. Here, we analyze the stream flow of one of the largest rivers in the world, the Parana in southeastern South America. For the last century, we find a strong correlation with the sunspot number, in multidecadal time scales, and with larger solar activity corresponding to larger stream flow. The correlation coefficient is r=0.78, significant to a 99% level. In shorter time scales we find a strong correlation with El Nino. These results are a step toward flood prediction, which might have great social and economic impacts.
River stream flow may impact sea level change?
Hi Willis,
re failed Shaviv 2008 p=0.54 replication.
Shaviv used Lean 2000. Is your data identical ?
Shaviv removed secular trends.
Your zscore uses column D and not detrended values in E ?
=’C:\Applications\Microsoft Office 2011\Office\Add-Ins\myfunctions2.xla’!zscore(F305:F407,D305:D407)
Paul Westhaver says:
January 24, 2014 at 11:07 pm
Thanks, Paul. One river? They are basing their case on one river?
Anyhow, I took a quick look at the paper, and I was unable to reproduce their high correlation coefficient (r=0.78). Nor does it appear that high by eye to me in their graph. I get 0.56, high but not at all surprising given that both of the series have been detrended using a Fourier filter, then averaged with an 11-year running mean … for me, calculating the correlation of the resulting lines is a meaningless exercise. You can believe it if you want … but note that the r2 of the two actual datasets before munging is a pathetic 0.01 …
In addition, a running mean is a horrible way to do the averaging, because it introduces spurious data, sometimes reversing the peaks. Doing correlations after smoothing it with a running mean is a statistical joke.
They also claim a p-value of greater than 99.99%, where I find nothing of the sort. After all of the machinations, I get a p-value of 0.03 (97%), which is significant, but just barely.
However, then you have to consider how many rivers there are in the world, and how many rivers wherein people have tried to find correspondences with the sunspots. When you consider that, you’ll realize that if the researchers looked at only two rivers to find this example, it is no longer statistically significant, even by the lax standards of climate science …
Look, is it possible that somewhere on earth there is some signal or sign from the sunspots?
Sure, it’s possible. But given the number of places that people have looked, at river flows and air flows and sea levels and all of that, and given how few examples of even a very weak correlation we find, I’ve gotta say that the results might be there, but they are hardly a convincing case. If the effect had significant strength, we’d see it everywhere … but we find it almost nowhere.
w.
Ian Schumacher says:
January 24, 2014 at 10:54 pm
Thanks, Ian. I assumed that when you said “correlation” you were referring to … well, correlation. What you get from the “correl” function in Excel, or the “cor” function in R.
Are you saying that you are referring to something else? And if so, what?
All the best,
w.
Manfred says:
January 24, 2014 at 11:40 pm
I think so, but if that matters, then I fear the analysis has other problems …
w.
Ian Schumacher says:
January 24, 2014 at 10:54 pm
Actually that’s a terrible example which totally disproves your claim, because the correlations between the Milankovitch variations and the change in ice volume are quite high, and in fact is used to verify the existence of the relationship … just as I did in the head post. See here for details.
w.
” and when we look at the Jevrejeva sea level records its worse yet …”
What is your basis for that claim? What you posted earlier appears to be your own digitised version of Jevrejeva’s fig3 but you did not answer about where you got what you are calling “Jevrejeva sea level records “.
Their fig 3 is effectively 30 year smoothed, the J 2008 data does not look anything like what you posted. If you are now using the correct data, perhaps you need to clarify that you are not still referring to your previous error.
You asked for demonstration of a case in climate where the cause sometimes leads and sometimes lags the effect. That is the wrong question. That does not happen , neither did I suggest it did.
It a missing variable problem. What can give the _impression_ of effect periodically leading the cause is when you have two or more causes creating and interference pattern.
I demonstrated above, through cross-correlation of MSL and sunspot data, that there is a concurrent cause and that it has a frequency close to 9.2 years. The interference of those two causes produces the phase crisis that has already frequently been noted in trying to simplistically relate global temp change to SSN.
If there are two such causes, the result goes in and out of phase with both of them individually. If you calculate the corr-coeff over a full cycle of the interference pattern it will be ZERO.
If you do the CC over some non integral multiple of the interference patter as is the case here you will get a low correlation That does not mean the either of the two causes does not exist.
Is there any part of that , that you do not follow or disagree with?
Willis Eschenbach says:
January 25, 2014 at 1:08 am
Paul Westhaver says:
January 24, 2014 at 11:07 pm
Paul, I mentioned above that I strongly disapproved of taking a correlation after using a running 11-year average, because the average inverts parts of the signal. Here’s why, from the Parana river analysis:
Take a look at the large peak at about 1960. now look at what has happened to it in the running average. Rather than being a peak, it has turned into a valley … one that happens to match up with some part of the Parana streamflow data.
But that’s a totally bogus correlation, because there was no valley in the sunspot number at that time, in fact there was a peak. So when you do a correlation on the smoothed result, it is not a correlation with sunspots.
Next, let me draw attention to another problem, one specific to their analysis. If we use an 11-year average on a signal without a steady frequency, and that frequency varies above and below the length of the average as the sunspot signal does, the resulting average will contain a portion of the frequency signal aliased as an amplitude signal.
This is because the 11-year filter will respond differently to sunspot cycles longer than average, say 12 or 13 year cycles, than it it will to sunspot cycles shorter than average … so we end up with frequency information mixed into the amplitude information.
As a result of all that together, would you care to take a guess as to the correlation shown in the graph above, between the residual sunspots shown in blue, and the 11-year running mean shown in red?
The correlation is MINUS 0.11 … amazing, huh? Their averaging method is so poorly chosen that it has a NEGATIVE correlation with the actual data.
In other words, their analysis is worthless, because by the time they’ve taken their correlations and p-values, they are no longer comparing river flow to sunspots, but to something totally different, something that in fact is NEGATIVELY CORRELATED to the actual sunspot data …
As I said, in part this is because of the length of the 11-year average and the length of the sunspot cycle. So when we look at the correlation between the Parana River flow data and its 11 year average, unlike the sunspot data they are positively correlated (but again very poorly, correlation of 0.14)
So no, I fear the Parana study doesn’t show what they think it shows. They are comparing to something that has nothing to do with sunspots.
w.
Greg Goodman says:
January 25, 2014 at 1:42 am
I’ve been juggling a lot of balls, Greg, and I missed that one. My data is from Figure 5 in Jevrejeva’s study here.
All the best,
w.
Greg Goodman says:
January 25, 2014 at 1:42 am
Greg, I’m still waiting for a real-world example of such a system as you are describing. To be clear, to reproduce the activity of the putative relationship shown in Figure 1, it would have to do the following:
1. The response must move stably in opposition to the forcing for a decade. Then within a decade, it switches to a multidecadal period where the response runs stably in parallel with the forcing. Finally, within a decade, the response switches back to stably running the other way, in opposition to the forcing.
2. For part of the time, the response needs to steadily lag the forcing as we’d expect. Then, for a couple decades, the response needs to switch and to steadily lead the forcing. Then, the response needs to switch back again and lag the forcing.
Now, that is the pattern that is shown in Figure 1. Stably out of phase, a quick switch to decades the other way, then quickly switch back to out of phase. Steadily leading, then steadily lagging, then steadily leading again. You claim that this could be happening through some unspecified “interference” … well, I’ve looked at a whole lot of interference patterns in my life, and I’ve never seen one that acted like that.
So … I’m still waiting for an example of some real natural system that works like that. I don’t think such things exist. You do, and you certainly may be right, lots of things I don’t know. But to establish your claim, you need to come up with a real-life example.
w.
Manfred says:
January 24, 2014 at 11:40 pm
Sorry, Manfred, missed that question. I tried it both detrended and not detrended. There’s almost no difference.
w.
Richard: “I agree.
And I observe that your link provides some debatable data which its presenter says only “suggests” something and rightly does not claim it “indicates” anything.”
Try again to explicitly make your point, without picking single words out of context.
The basic unit of speech is a sentence. Try to use quotes of at least that length.
Paul Westhaver:
At January 24, 2014 at 7:01 pm
Yes, I too “see what I see” and – like you – I see your ad hominems.
The difference between us is that I object to ad hominems and you use them. Don’t.
Richard
OK, re fig 5 in that graph, which is the same as the paper I have calling it fig3 . What you are looking at is effectively 30y averaged data. The actual data looks more like fig2 though fig2 is a comparison to something else. I provided two links to the Jevrejeva data above, I suggest you grab one of them, run the first diff and use that.
That is the correct source of the data and is not 30 year smoothed.
Greg Goodman:
At January 25, 2014 at 2:32 am you demand
OK. I did use sentences but I will obey your command.
Here is my “try” to “explicitly” make my points.
1.
The data is processed and, therefore, it can be challenged.
2.
The provider of the data says the data “suggests”.
3.
The data does not indicate anything and its provider does not claim it does.
4.
Your link is here
http://climategrog.wordpress.com/?attachment_id=760
and the quotation you insist I copy from it says
5.
The demanded quotation admits there are “other variations” says “this seems to be the main effect” and concludes “This suggestive of” all of which confirms each of my points (2) and (3).
6.
My original statement says all that is said in my points (1) to (5) but is more clear and is more succinct.
I have fulfilled your instruction to the best of my ability. Can I now Stand At Ease or may I be Dismissed?
Richard
Willis: “Greg, I’m still waiting for a real-world example of such a system as you are describing. To be clear, to reproduce the activity of the putative relationship shown in Figure 1, it would have to do the following:
1. The response must move stably in opposition to the forcing for a decade. Then within a decade, it switches to a multidecadal period where the response runs stably in parallel with the forcing. Finally, within a decade, the response switches back to stably running the other way, in opposition to the forcing.”
Look, Willis, there’s a lot more going on than one or two variables in that data, there is not point in trying to get each segment described exactly with a trivial model.
The point I am trying to make is two simple harmonics of say 9.2 and 10.5 will interfere to produce a result that will sometimes be in phase with one of them and at other times will drift into anti-correlation and then back again. Thus using a simple correlation against one of the variables will give a low result. This is what you showed, and that is why you get that result.
You ask for a real example and we are looking at one. You have noted the phase goes wrong and there are time when it is inverted wrt SSN. Even if SSN is not all that regular, the spectrum is very strong around 10.5 and the cross-correlation provided the other frequency.
I used those data sets because I don’t trust the rigged satellite GMSL (Jevrejeva’s no GW sceptic but I think here work is genuine). I use sunspot area because it’s available as daily data and I prefer to do my own filtering and resampling, though don’t think that is an issue on this crude scale. I just used what I had.
I’m not sure whether eyeballing the CC lag plot or the spectral analysis is more accurate for the long period but that is immaterial to this discussion. The result is about 9.2 whichever we take.
Are you not able to see that lag-correlation plot derived from MSL and sunspots proves the presence of a circa 9y periodicity that WILL seriously reduce the correlation and produce exactly the kind of phase shifts you noted ?
Try messing around plotting a cos(9.2)+cos(10.5) and plot against either one and look how the phase drifts in and out.
Richard, ease up on the attitude a bit if you want to discuss.
1.The data is processed and, therefore, it can be challenged.
2.The provider of the data says the data “suggests”.
OK all climate data is suspect. Next point.
I used the word “suggests” in two contexts, not for the whole analysis. That is why I asked for specifics. “Visual estimation of the nodes suggests a long period of 79.1 years. ” , suggests because the period is not precisely determined. That the modulation is there is without question.
“This suggestive of a lunar cycle. ” because the attribution is speculative, not because the presence of a circa 9.2 is merely suggestive.
That analysis does show the presence of two cycles interfering, that part is not “suggests”, it is there. That is what is “indicated” by the data.
So now we have that clear, what is the point you were trying to make by saying:
“I agree.
And I observe that your link provides some debatable data which its presenter says only “suggests” something and rightly does not claim it “indicates” anything.”
Just for reference the Jevrejeva paper I referred to was this one (fig3 is Willis’ fig5 in the other paper).
http://dx.doi.org/10.1029/2008GL033611
Recent global sea level acceleration startedover 200 years ago?
Greg Goodman says:
January 25, 2014 at 2:43 am
Greg, I looked above and didn’t find any links to data. Sorry, what am I missing?
w.
Greg Goodman says:
January 25, 2014 at 3:06 am
Thanks for the answer, Greg, but no, you are claiming that’s what’s happening here—so you can’t use it as your example to show it’s happening here. That’s circular
I say that kind of relationship between forcing and respons (stable, phase reversal, stable, phase reversal, sometimes leading, sometimes lagging) doesn’t happen in real situations in the real world. I’m asking for some example of a real cause-and-effect relationship that does what you think this putative sea level/sunspot relationship is doing … so you can’t use this relationship itself as your example.
w.
Greg Goodman says:
January 25, 2014 at 3:06 am
That is true, Greg, and I’m well aware of that. However, that’s not the pattern we are trying to match. We are looking for a pattern which is stable, then quickly reverses, then is stable again, no change for some decades, then quickly reverses again.
A beat frequency of 9.2 and 10.5 years does nothing of that sort at all. As you said, it gradually shifts into and out of phase, and it is never stable … but that’s not at all what we see in Figure 1.
Regards,
w.
“I say that kind of relationship between forcing and respons (stable, phase reversal, stable, phase reversal, sometimes leading, sometimes lagging) doesn’t happen in real situations in the real world. ”
No, I’ve already said the idea the causal relationship flips for lag to lead does NOT happen, it just appears that way because of the missing variable. You can seen this phase shift in the pure maths of cos+cos , you don’t need a climate example which would only be anecdotal anyway and someone would say the data is dubious.
Understand the maths of how the two close harmonics interfere and you get the phase shift and you get low correlation if you ignore one or the other variable.
This is basic physics for stuff like acoustic beats, diffraction patterns, etc. , it’s just about the most basic wave interaction.
“That is true, Greg, and I’m well aware of that. However, that’s not the pattern we are trying to match. We are looking for a pattern which is stable, then quickly reverses, then is stable again, no change for some decades, then quickly reverses again.”
Great, at least we’re on the same wavelength now.
The solar cycle is not stable and there are other things happening (there’s 22.5 y and two close 5.42 and 5.85), that are small but enough to give the _visual impression_ of a sudden flip. The cross-correlation plot clearly shows the circa 10 y period (average _freq_ of 10.5 and 9.2) modulated by about 70-whatever years.
the data links you missed were:
Data should be available from PMSL but it’s not responding my end right now.
http://www.psmsl.org/products/reconstructions/gslGRL2008.txt
Suggest KNMI
http://climexp.knmi.nl/getindices.cgi?WMO=PSMSLData/gsl_ann&STATION=global_sea_level&TYPE=i&id=someone@somewhere&NPERYEAR=1
In fig1 cycle 18 seems to be where the ‘lunar’ signal is in anti-phase (1947 ?) , well aligned in 1980, a diff of 33 years. The full cycle in-out-in phase is twice that. That’s a little shorter than what I got from a mathematical analysis which uses all the data not eyeball pattern picking.
Also later 20th c solar cycle were on the shorter end of the variable solar cycle length.
Greg Goodman:
I am replying to your post at January 25, 2014 at 3:20 am
You say to me
Say what!?
I gave clear, succinct and polite correction to an assertion you made and your response was to instruct me to do what I had already done for you!
I obeyed your command but made clear that requests are preferred.
Now you say
That combines a falsehood with a misdirection.
My post which you demanded me to expand upon is at January 24, 2014 at 6:37 pm and it said in full
Clearly, I was pointing out that your cited “available evidence” only suggests and it does NOT “indicate” anything.
You say to me, “ease up on the attitude a bit if you want to discuss”.
I say to you, Be honest and stick to the subject then discussion will be possible.
Richard
Clearly, I was pointing out that your cited “available evidence” only suggests and it does NOT “indicate” anything.
No you were saying that I said it only suggested in did not indicate anything.
“…its presenter says only “suggests” something and rightly does not claim it “indicates” anything.”
And you accuse me of falsehood and misdirection and tell me to be honest. LOL.
You clearly do not have anything constructive to add the technical discussion. So I’ll leave it at that.
Greg Goodman:
Your semantic twaddle at January 25, 2014 at 5:03 am is interspersed with this falsehood addressed to me
No!
I made a technical observation which you disputed claiming that data you linked indicated something.
IT DOES NOT IINDICATE ANYTHING AND I POINTED OUT THAT IT DOES NOT.
You tried to wriggle out of having made a false assertion concerning an indication of a solar/climate link.
You clearly do not have anything honest to add the technical discussion. So I’ll leave it at that.
Richard
Willis,
Many of us are were implicitly using signal processing correlation [see here http://en.wikipedia.org/wiki/Cross-correlation%5D and also the qualitative synchronicity, which more qualitative and unfortunately doesn’t have a well defined metric that I know of.
This allows one to see how much of the ‘energy’ of a signal can be explained by our component. We can be off by phase, offset, amplitude and still that there is relationship. R^2 is about having all the component and arranged in the right way. We are also referring to synchronicity, for which I don’t know of any measures. Statistics and signal processing both don’t have good tools for when the environment is non-linear.
You referenced a paper for Milankovitch cycles and said it disproves my point because correlation is very high. Thanks for the paper, but:
1.) They are NOT correlating against temperature. They are correlating against insolation (or it’s proxy). Yes the correlation with insolation is good because there is a direct cause and effect there, so it’s to be expected. Let’s apply R^2 to this very small change in insolation versus temperature. Unfortunately that relationship is weak (in terms of R^2) because the changes in insolation are too small to directly drive climate from ice age and back again.
2.) Just like this paper, they are cherry picking times. While insolation correlation will probably still hold for longer periods, as there is a direct cause and effect, temperature correlation will get even worse as periodicity of ice ages completely changes [see – http://en.wikipedia.org/wiki/100,000-year_problem%5D
But it’s an excellent example because it also applies to sunspots. Sunspots are strongly correlated (in every sense of the word) with insolation to. People’s argument with sunspots is that there is somehow more to it. That the sunspots have a greater effect than that small changes in insolation would directly cause. So again the sunspot/Milankovitch parallel is excellent.
Also again, Climate is non-linear. While we should obviously still expect meaningful R^2 values, they will be lower than they would be in a linear system. Some of the things you say “can’t happen” in a causal relationship (phase shifts for example) ‘can’ happen and do happen in a non-linear system.
Ian Schumacher:
I await the answer of Willis to your post at January 25, 2014 at 9:06 am.
I write to request a clarification. You say to Willis
This gives me a problem, and I would welcome explanation. My problem is coherence and I exoplain it as follows.
If there is a direct causal relationship between two parameters then
(a) the causal parameter varies
and
(b) that variation induces an effect in the other parameter.
So, the effect cannot happen before the cause (in the absence of a time machine).
The time between the causal variation and the resulting effect may vary. As you say, this would induce a phase shift.
But the time between the causal variation and the resulting effect cannot go negative. That would make the cause happen after its effect has occurred. So, such a phase shift cannot occur (in the absence of a time machine). And that is what Willis rightly says “can’t happen”.
However, there could be an indirect effect whereby some third effect varies and is the cause of the variation of each of the other two effects.
I assume you are not claiming a time machine exists. So, my desired clarification is a request to know if you are claiming there is some parameter which has effect on the Sun and on the Earth’s climate and, if so, what is it?
Richard
richardscourtney says:
If a system is linear it means that supposition applies
f(a+b) = f(a) + f(b)
Many systems depend on inputs in a non-linear way. For example a signal below a certain threshold may result in nothing happening, and above a certain threshold results in an event e.g. avalanche, Earthquake, etc. Superposition does not apply to these systems.
Statistics also relies on systems being time-invariant [ http://en.wikipedia.org/wiki/Time_invariant ]
Unfortunately the climate is chaotic and neither of these conditions is valid.
In general, chaotic systems have what I would describe as a very long ‘memory’. An operation on a chaotic system is never operating on the same system twice (many other state parameters underneath have changed). Imagine we start with a fresh climate. Static and unchanging. To this insolation increases for the first time just a little bit. This triggers all sorts of events. Things warm up a little bit more than they did before, some more water vapor goes into the atmosphere and we can measure these things. Then insolation goes back down to normal, but the momentum of these events carries on. Cycles might be triggered, thresholds may have been exceeded for certain events (glacier melting, an avalanche to occur, and so on). These are non-linear events. Now insolation increases again a second time. This time the response is slightly different then before. This time the response is overlayed on top of the history of the past insolation event, the effects of which are still going on. Even if we assume that insolation is the ONLY external variable in this system the output response of the second insolation event may appear delayed, or even early, as it overlaps with the continuing effects of previous events. Statistics requires time-invariance. Chaotic systems are time variant. The climate is a chaotic system.
A completely different but important issue is that there are many signals operating at the same time. Statistics assumes all other inputs are random in nature (so they can be canceled out). This is often not the case. Many other signals such Milankovitch cycles, moon orbit, Earth’s magnetosphere, ocean mixing phases (PDO or whatever they call it) and so on are periodic and if they are a of a similar period to our signal of interest can very easily move the phase of the output one way or the other.
To be clear I don’t think sunspots have great influence. The are correlated with solar insolation and so there will be ‘some’ effect. That there can be no doubt. The argument is really about how big an effect they have.
lsvalgaard says:
January 24, 2014 at 2:12 pm
…..
As you have a tendency to misunderstand things, you should provide a link so we can see what you misunderstood.
It is the one who missed his understanding that is most keenly searching for it (vukcevic)
Or to put it in another way
“When a scientist is ahead of his times, it is often through misunderstanding of current, rather than intuition of future truth. JR
I admit to both of the above with exclusion of term ‘scientist’, since I am an engineer despite an MSc to my name.
http://cc.oulu.fi/~usoskin/personal/2007GL0330401.pdf
Paul Westhaver says:
January 24, 2014 at 2:32 pm
……
I suppose someone must have done good analysis. I tend to avoid clouds, they are source of strong both negative and positive feedback, and as far as I know global variability is on low side just 2-3%.
vukcevic says:
January 25, 2014 at 11:18 am
“When a scientist is ahead of his times, it is often through misunderstanding of current, rather than intuition of future truth”
well, you are not ‘ahead of your time’, just ignorant of the facts. And unwilling to learn.
vukcevic says:
January 25, 2014 at 11:18 am
“When a scientist is ahead of his times, it is often through misunderstanding of current, rather than intuition of future truth”
Well, you are not ‘ahead of your time’, just ignorant of the facts, and unwilling to learn.
rgbatduke says: January 22, 2014 at 1:32 pm
“The pause” is just such a stretch in the GASTA , where the 1997/1998 Super-ENSO is coincident with the preceding jump…
I have to be careful when addressing the most erudite of the commentators … (being cut to shreds or ignored, I suppose the second is less painful to an inflated ego), so I shall be brief:
ENSO, yes I agree; and if I may add: it is in the attendance of its travelling companion.
http://www.vukcevic.talktalk.net/ENSOa.htm
sorry for the double posting. WordPress is playing tricks on me 🙁
vukcevic says:
January 25, 2014 at 12:07 pm
Ah – But Sire Vuk of the clan cevic, thou shouldnest know that, though ’tis more painful to be shredded than ignored, ’tis less valued, even valuelest, to be ignored! Truly, ’tis actually a compliment to be corrected by another reader writing a wronged writing rightly, for then one knowest that the reader – and more proper! – the righter did deem it valuable of their time to spend their precious ducats and minutes correcting thy wrong wrightings rightly….
(besides, an over-inflated ego has no more value in it than a properly inflated ego, and is much closer to its bursting point.)
or something like that. (I’ll let Janice Moore translate. She does all that touchy-feely people-to-people stuff better.)
vukcevic:
“When a scientist is ahead of his times, it is often through misunderstanding of current, rather than intuition of future truth”
Dr. Svalgard:
Well, you are not ‘ahead of your time’, just ignorant of the facts, and unwilling to learn.
vukcevic:
“In science there is never any error so gross that it won’t one day, from some perspective, appear prophetic.”
Dear doc, here is an “error not so gross….”
http://www.vukcevic.talktalk.net/Ap-NHT.htm
RACookPE1978 says: January 25, 2014 at 12:31 pm
But Sire Vuk of the clan civic …
Err … .. in exactness ‘a sapling of the clan Vuk’
Sire, receive tis actually a compliment to be corrected by dependable reader writing a wronged writing rightly, for then one knowest that the reader – and more proper!
vukcevic says:
January 25, 2014 at 12:45 pm
“In science there is never any error so gross that it won’t one day, from some perspective, appear prophetic.”
Nonsensical self-glorification. Willful ignorance is the grossest sin.
lsvalgaard says:
January 25, 2014 at 1:06 pm
Nonsensical self-glorification. Willful ignorance is the grossest sin.
The Holy Ecclesiastical Court
Prosecutor: Higher Authority
Charge:
MAVukcevic is charged for theological offence: Committing ‘grossest sin’ by challenging The Doctrine.
Details of charge: Performing act of the exact calculation from the best available data, and so showing that the temperatures natural variability can be directly correlated to the solar activity by manipulating the Ap index (Ap index being the most accurate metric for the part of the solar magnetic activity impacting the Earth).
Exhibit: http://www.vukcevic.talktalk.net/Ap-NHT.htm
Defence: Barred
Verdict: Guilty as charged.
Sentence: Life long excommunication.
Willis Eschenbach says:
January 25, 2014 at 2:20 am
…Now, that is the pattern that is shown in Figure 1. Stably out of phase, a quick switch to decades the other way, then quickly switch back to out of phase. Steadily leading, then steadily lagging, then steadily leading again. You claim that this could be happening through some unspecified “interference” … well, I’ve looked at a whole lot of interference patterns in my life, and I’ve never seen one that acted like that.
So … I’m still waiting for an example of some real natural system that works like that. I don’t think such things exist. You do, and you certainly may be right, lots of things I don’t know. But to establish your claim, you need to come up with a real-life example…
———————————————-
I think, Greg already responded, that this is a missing parameter problem and hence leading nowhere to try to find a reason in the solar data at all. The sunspots are obviously not the only influence, if these phase differences are all real.
But, the first data point may be not reliable. The smaller phase differences afterwards up to about 1990 can’t be taken seriously conisdering the noise in the underlying raw data. Even the phase difference after 1990 may not be real, because it is much smaller in Shaviv’s dataset.
A natural system responding with varying lag would be a forced oscillator. The lag may be governed in addition by varying sun spot cycle lengths.
Hi Willis,
I tried to replicate your result of Shaviv 2008 using Lean 2005 data (also with 2 additional years) and arrived at:
r = 0.3076 and r2= 0.0946
For detrended solar data the result is:
r = 0.3480 and r2 = 0.1211
That confirms your result of r2 = 0.09, but suggests, you used detrended data.
However, that does not match, what Shaviv reports under figure 6 in his paper:
“Here r = 0.54 giving a p = 10−4 (for Neff = 47)”
and later on in the text on the same page:
“Here we find a correlation coefficient of r = 0.55 with the
solar luminosity reconstruction [Lean, 2000]. Unlike the previous
record, the correlation extends over many more solar
cycles. The high Neff = 67, gives rise to a 99.99% confidence
that random realizations with similar autocorrelation functions
as the actual signals cannot give such a high coefficient r.”
(I could not find the reason for the slight difference in these two statements r=0.54 or 0.55 ?, Neff=47 or 67 ?, is the second statement for a longer series ?).
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.2162&rep=rep1&type=pdf
However, I think, the main reason for the difference from your result is the removal of secular trends which was NOT done by detrending.
You see that, when you plot the detrended curve and compare it with the solar curve in Shaviv’s figure 6. So he did something else with a better match.
See also this quote from Lean 2000:
“Thus, the reconstructions of the 11-year spectral irradiance cycles are relatively robust whereas the multi-decadal to centennial changes on which the cycles are superimposed are far more speculative.”
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.393.1008&rep=rep1&type=pdf
Manfred,
“So … I’m still waiting for an example of some real natural system that works like that. I don’t think such things exist. You do, and you certainly may be right, lots of things I don’t know. But to establish your claim, you need to come up with a real-life example…”
He’s joking right. Ummm the chaotic system otherwise known as climate? Most systems that people study are not chaotic, so it’s easy to fool oneself into thinking “no such systems exist”. Chaos allows random-looking ‘echos’ to interfere with new incoming signals.
Here’s a fun one http://upload.wikimedia.org/wikipedia/commons/4/45/Double-compound-pendulum.gif
Imagine perturbing this system and measuring the output and doing a correlation 🙂
Of course climate isn’t ‘that’ bad, but there are still random ‘echoes’ of past perturbations inferring with new signals.
To me here is the dilemma in a nutshell. Theoretical conversation with anyone you choose to imagine:
Statement: “Based on R^2 correlations of sunspots and sea-level change, any proposed cause and effect is bogus.”
Question. “What about Milankovitch cycles R^2 correlation with temperatures. It’s low, really low if one considers it drastic changes periodicity in the past.”
Answer. “That’s different. Milankovitch cycles are highly correlated with insolation.”
Question. “Yes, but sunspots are also highly correlated with insolation. What makes you think Milankovitch cycles can have a larger effect than direct insolation changes predict, but sun spots can not?”
Answer. There is an obvious synchronicity of Milankovitch cycles and ice ages over a long period of time that is statistically impossible to reconcile without some sort of cause and effect mechanism.
Question. Even though R^2 is almost zero?
Answer. Yes.
Question. So if someone showed you a long time period of sunspot synchronicity, Would you accept that there may be cause and effect even if R^2 correlation is low?
Answer. [Long pause]. Yes, but such a record doesn’t exist.
[Note: I have no idea about the validity of the last answer.]
In conclusion, one can not accept that Milankovitch cycles somehow trigger ice-ages even though correlation with ice-age temperature changes is weak, while at the same time dismissing that sun spots might also trigger larger behavior because R^2 is too weak. To resolve this we need a much longer time period, or a much better model that can account for all external signals and all non-linear affects (next to impossible).
To add to that. One must be careful when using R^2 on non-linear systems. R^2 assumes linearity, time-invariance, and a stationary noise background – none of which can be assumed to exist for non-linear systems. It will probably still ‘correlate’ (as long as the chaotic system isn’t too ‘wild’), but we should expect correlations to be lower than for a corresponding linear system..
That said on balance I think Willis is probably right in this case. It’s an interesting graph, but it’s too short and it’s correlation is too weak to really claim anything. It merely hints at the possibility of a relationship. Those that believe will believe. Those that don’t, won’t. Those of us on the sidelines will continue to wait for better data before making a conclusion.
Remember the days described in history books when science wasn’t a proxy for statistics …. Signal to noise was off the charts. Cause and effect jumped out from your experiment and hit you in the face. Concise equations could actually be used to describe something meaningful. All low hanging fruit long gone now I suppose.
vukcevic says:
January 25, 2014 at 3:27 pm
Performing act of the exact calculation from the best available data, and so showing that the temperatures natural variability can be directly correlated to the solar activity by manipulating the Ap index
You have not even done that. Every statement you have made in the last several comments has been false or expressing a misunderstanding. That you are not ashamed of you unwillingness to learn something is a cause of concern.
Manfred says:
January 25, 2014 at 4:58 pm
Is there something confusing to you and Greg regarding my request for a real-life example of what you are referring to? Look, I understand beat frequencies and interference and the rest. I am asking for a PRACTICAL ACTUAL EXAMPLE of a natural system that works like the relationship up top in Figure 1 … namely steady, then reversal, then steady for decades, then reversal, then steady, AND sometimes the cause leads the effect and sometimes the effect leads the cause.
I’ve been asking Greg for an example of such a system, and I get things like “The solar cycle is not stable and there are other things happening (there’s 22.5 y and two close 5.42 and 5.85), that are small but enough to give the _visual impression_ of a sudden flip. The cross-correlation plot clearly shows the circa 10 y period (average _freq_ of 10.5 and 9.2) modulated by about 70-whatever years.” … well, yeah, whatever, but where’s the dang example?
And you, when I ask, you say it’s a “missing parameter problem”.
Well, yeah … but where’s my real-world example? Or are you saying that we’ve discovered the only real example of this kind of system and we see it in Figure 1?
Regards,
w.
Ian Schumacher says:
January 25, 2014 at 5:42 pm
Nope. As I understand their claims, Manfred and Greg are saying that this is a deterministic system where the sea level is a function of inter alia the sunspots … except sometimes sea level leads the sunspots and sometimes it follows them. And sometimes it stays neatly in sync for decades in a most unchaotic manner and then flips to being totally out of sync …
Now, you may be right that this is expected behavior in a chaotic system … but in that case, the claim Manfred and Greg are making, that sea level is a function of sunspots, falls apart entirely.
w.
vukcevic says:
January 25, 2014 at 3:27 pm
Performing act of the exact calculation from the best available data, and so showing that the temperatures natural variability can be directly correlated to the solar activity by manipulating the Ap index
‘Manipulating’? This is what Ap looks like: http://www.leif.org/research/Ap-1844-now.png it cannot be ‘manipulated’.
lsvalgaard says:
January 25, 2014 at 10:41 pm
‘Manipulating’? This is what Ap looks like: http://www.leif.org/research/Ap-1844-now.png it cannot be ‘manipulated’.
Of course, that is what the instruments see, not what the Earth’ and its magnetic field reacts to.
If you are interested in a serious dialog, rather than grumbling about learning (about that later) you should look more carefully into this graph
http://www.vukcevic.talktalk.net/Tromso.htm
One thing you would notice that the GMF responds half-heartedly to some strong Ap signals, while on other occasions it reacts strongly to much weaker Ap.
Why is that so?
You may think of number of ‘convenient’ reasons, but the simplest one is that the instruments do not take account of the incoming polarity, but the Earth does. Formally recorded ‘Ap signal value’ is always positive, but taking account of the NASA’s statement on the polarity orientation of the CMEs, I appropriately ‘sign’ the official Ap data. It is as simple as that. That is the ‘Ap MANIPULATION’, meaning manually correcting to what the Earth field reaction may be, and not COUNTERFEITING the data.
Once you accept that simple initial step than we can proceed to the next one, and check exactness of my calculations.
On learning: I read everything you write, but I’m selective in what I apply. I can’t aspire to acquire even a fraction of your knowledge, so I behave as a kind of a ‘science vandal’, look for a window which is not ‘armour plated’, throw a rock into it. Occasionally there is a break, then I can get clear view of the hidden treasures, report it, so the ‘authorities’ should know about.
Willis: “I am asking for a PRACTICAL ACTUAL EXAMPLE of a natural system that works like the relationship up top in Figure 1 … namely steady, then reversal, then steady for decades, then reversal, then steady, AND sometimes the cause leads the effect and sometimes the effect leads the cause.”
You are eyeballing certain patterns in that data at the same time as saying it’s a fools game because the human brain sees all sorts of things that do not exist. Why is your rather complex pattern as described above any more real that the patterns others are seeing?
I see not point in trying to explain such a visually derived “pattern”.
You also criticise the 9 station dataset. I have not looked into the reasons for choosing those stations so I don’t know whether they are objective. I suspect there may be some cherry picking involved so I looked at a dataset that I am familiar with, that I think is a genuine objective work.
I have shown cross-correlating that data to sunspots shows a clear peak that has a long period modulation that corresponds to interference with a 9.2 year cycle.
If there is a strong correlation with solar activity that would destroy it and give a low correlation like you found. It would also produce a pattern of behaviour where the phase would sometimes lead the SSN data and sometimes lag. So until that is accounted for the variable phase you correctly pointed to cannot be used to disprove a possible solar connection.
Since you seem to have accepted my point about the way interference patterns work it seems a diversion to carry on wanting an example from elsewhere in climate. All climate data is a nasty mix of noise resonance and interference and all we will get is another bunch of peaks to argue about.
It is still possible from cross-correlation that I did , that the whole solar+lunar interference is only a small part of variability in the Jevrejeva data. Maybe a multivariate linear regression may extract something.
There does appear to be a strongly repetitive pattern in MSL that is worth investigation. It should provide some information about the degree to which the thermal component of MSL being linked to the solar cycle since cross-corr shows there is something there.
It would also be worth looking at the geographic distribution of the Jevrejeva inputs (IIRC it’s NH Europe+Scandinavia). The 9.2 is strongly suggestive of long term tidal variation. Tides involve large horizontal displacements of water and hence thermal energy which could be a significant factor in regional patterns.
As a final point here, if decadal scale variations in lunar or luni-solar tidal forces import and export water from the tropics and Willis’ thermal regulator ensures tropics remain about the same SST, this implies a long term variation in the heat input to the tropics and hence global system.
Variations in the magnitude of those patterns could cause global warming/cooling on all scales.
We don’t understand the short tides, let alone the long ones.
vukcevic says:
January 26, 2014 at 2:02 am
Of course, that is what the instruments see, not what the Earth’ and its magnetic field reacts to.
Ap is a measure of the reaction of the Earth to the solar wind. Not something the ‘instruments’ see.
If you are interested in a serious dialog
What you are doing is not science and hence there is no basis for a dialog, let alone a ‘serious’ one. Rather, I try to educate you, but you are singularly learning-resistant.
One thing you would notice that the GMF responds half-heartedly to some strong Ap signals, while on other occasions it reacts strongly to much weaker Ap. Why is that so?
Simply because the field at an auroral zone station is very ‘local’ and does not reflect the global effect. For example: a strong geomagnetic storm moves the auroral zone equatorwards so that the effect at Tromsoe becomes less. In general, the effect at an auroral station is very localized and can be very different from one station to the next, see e.g. http://www.spaceweather.ca/auto_generated_products/stackplot_e.png and has nothing to do with the polarity of CMEs, but simply reflects the fact that an auroral station reacts strongly to currents that are only within a few hundred kilometers distant.
You may think of number of ‘convenient’ reasons, but the simplest one is that the instruments do not take account of the incoming polarity, but the Earth does.
Ap is a measure of the reaction of the Earth to ‘incoming polarity’. Given the solar wind parameters [in particular the polarity] we can calculate precisely the resulting Ap value. See for example http://www.leif.org/research/IAGA2008LS-final.pdf [especially Figure 6] or the extended discussion at http://www.leif.org/research/suipr699.pdf.
Formally recorded ‘Ap signal value’ is always positive, but taking account of the NASA’s statement on the polarity orientation of the CMEs, I appropriately ‘sign’ the official Ap data.
NASA did not [nobody can] know the orientation of CME’s back to the 1880s [no need to produce yet another link to a misunderstood paper]. And it is inappropriate to ‘sign’ the Ap-data as the sign is already in the Ap [part of what makes Ap].
On learning: I read everything you write, but I’m selective in what I apply.
That is not reasonable as that simply is confirmation bias: you apply what you like and disregard what doesn’t fit.
Ian Schumacher:
Thankyou for your response at January 25, 2014 at 11:13 am which is here here to my question concerning coherence at January 25, 2014 at 9:39 am which is here here .
I am grateful that you went to such trouble as to provide so long an answer. Unfortunately, your answer does not address my question.
For the benefit of others, I remind that I asked
Please note that my question was about coherence with particular attention to ‘a cause cannot follow its effect’.
I was not asking about linearity, chaos or system memory. I was asking about coherence and your answer does not mention it.
I notice that in your post addressed to Manfred at January 25, 2014 at 5:42 pm which is http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1549608
you say
OK. But that does not remove my puzzlement which is as follows.
1.
The putative chaos is in the climate system.
2.
I can understand how an “echo” can delay a climate effect which is caused by the solar cycle.
3.
But I fail to understand how an “echo” can move a climate effect forward in time such that the effect occurs before its cause.
4.
The problem of point (3) exists in the data under discussion if either of the two parameters (i.e. the solar cycle and the SLR variation) is causal of the other.
5.
I can think of ways to develop new philosophical concepts of time with potentially important practical applications if I were able to understand point (3).
So, please explain.
Richard
I appreciate your effort to finally engage in the meaningful discussion. I have read number of your points, and I’ll read them again, there isn’t as much disagreement there as you make out. Instead of instantly rejecting what you disagree with and arrive at a standstill (also exhibiting strong symptoms of confirmation bias), you could try to make a step forward and see why the alternative is more ‘nature like’.
There are other aspects to it beside the NASA’s statement. I have mentioned some here:
http://wattsupwiththat.com/2014/01/25/sunny-spots-along-the-parana-river/#comment-1550013
You are welcome to rubbish it, but results explain:
– Earth magnetic field and the global temperatures ~21year spectral component
– Ap and aurorea spectral components which do not exist in the sunspot cycles series.
– LOD variability correlated to signed sunspot cycles
– Physical process that could explain how the Earth’s core field dynamo is affected by impact of the solar energetic events (CMEs), via atmospheric events and subsequent exchange of the angular momentum.
Dr. S.
I forgot to ask: have you ever updated the Fig. 23 in http://www.leif.org/research/suipr699.pdf
If so I would like to see a copy of the graph, while data would be greatly appreciated, even if I have to suffer further ‘pseudoscience’ lapses.
vukcevic says:
January 26, 2014 at 7:48 am
I appreciate your effort to finally engage in the meaningful discussion.
There is no engagement as information flow is solely one-way.
you could try to make a step forward and see why the alternative is more ‘nature like’.
When ‘alternatives’ are rubbish they are not steps forward.
but results explain…
When garbage goes in, no explanation comes out.
vukcevic says:
January 26, 2014 at 8:12 am
I forgot to ask: have you ever updated the Fig. 23 in http://www.leif.org/research/suipr699.pdf
No, as there really is no need to elaborate further on this well-known result. But the effect shows up in modern data as well, of course, see e.g. Figure 17 of http://www.leif.org/research/2007JA012437.pdf and discussion in para [38]. See also section 5 of http://www.leif.org/research/Semiannual-Comment.pdf . In any case, it is just a small, second-order effect.
If so I would like to see a copy of the graph, while data would be greatly appreciated
The data are just the sunspot number and the aa-index which you can get from many places.
I have to suffer further ‘pseudoscience’ lapses
There is nothing wrong with pseudoscience [and make-believe] as long as you know it is pseudoscience. The problem comes in when you think it is [and advocate it as] science.
vukcevic says:
January 26, 2014 at 8:12 am
I forgot to ask: have you ever updated the Fig. 23…
The 22-year cycle was discovered by Ed Chernosky in 1966. Here is his original paper on that http://www.leif.org/EOS/JZ071i003p00965.pdf
If you go to his Figure 4 and look at the lower panel showing the sunspot number you may see that for the period covered odd sunspot cycles were more active in the first half of the cycle. If you ‘slide’ the B-curves down to match the A-curves during the first half of the cycles, you may see that the enhancement of activity in the first half of the odd cycles is simply due to those cycles being more active [per chance for the period covered]. The 22-year cycle in geomagnetic activity then becomes only something that happens in the last half of the cycle [after polar field reversal]. See also the discussion in para [20] of http://www.leif.org/research/Asymmetric%20Rosenberg-Coleman%20Effect.pdf
lsvalgaard says:
January 26, 2014 at 9:01 am
The 22-year cycle in geomagnetic activity then becomes only something that happens in the last half of the cycle [after polar field reversal].
I said that clumsily. The 22-year cycle enhancement is something that only occurs in one half of the 22-year cycle, namely from polar field reversal in even cycles to polar field reversal in the next odd cycle.
Thanks !
vukcevic says:
January 26, 2014 at 9:33 am
Thanks !
On second thought, it might be a good idea to re-visit the 22-year cycle some day as there are some misconceptions floating around…
The 22-year cycle in solar activity is called the Gnevyshev-Ohl rule [that odd cycles are intrinsically more active than even cycles]. Cycle 23 is a good example of the breakdown of the G-O rule and were cycle 18 and cycle 5 among others, so the ‘rule’ is not ‘robust’ [another word for ‘real’].
The 22-year cycle in geomagnetic activity is a geometric effect that relies on the solar polar fields regulating the length of solar sectors in just the right way [the Rosenberg-Coleman effect] such as they maximize the Russell-McPherron effect for enhancing the reconnection-generated geomagnetic activity. These effects are small and insignificant as far as being causative for any effects on the Geosystem, but important in the sense that they show us a very important aspect of solar cycles, namely that the solar polar fields have reversed normally since at least the 1840s.
Greg Goodman says:
January 22, 2014 at 4:33 pm
Good morning, Greg. I don’t know how to read the significance level from your plot … which peaks are significant?
I ask because I just looked at the cross-correlation of the annual change in the Jevrejeva data, versus the SIDC sunspot numbers from Leif. I don’t find a significant correlation at any lag. The maximum correlation is with the sea-level change occurring ten years after the sunspots, R^2 = 0.005 …
Best regards, and many thanks for your participation, always valuable,
w.
A number of people have recommended the use of the cross-correlation function to determine if a relationship exists. While this is valuable, and I use it often enough that long ago I wrote an Excel function that does cross-correlation, often the claim of significance is overblown. Here’s an example of why.
The figure below shows the cross-correlation between two datasets. At a lag of -12, we find a strong negative correlation, -0.25. Upon examination, this relationship is statistically significant at the 95% level (p-value less than 0.05), so I say “Whoa … big news, we’ve found a significant correlation”.
But how can that be, this is a pair of random datasets?
The answer lies in the fact that to find that “significant” correlation, the cross-correlation does an automated data dredge through no less than 33 separate comparisons. And as a result, the significance level we now need to achieve goes through the roof. If our significance level for one comparison is 95% (p-value of 0.05), the usual in climate change, then if we are examining 33 separate realizations, to be considered significant we need to find a relationship with a p-value of .0015 … which means, of course, that the relationship that I found in the graph above is NOT significant, it’s just random chance at work.
Now, it’s not quite that bad. I consider it this way. I’m starting my search at lag 0. I only consider lags on the logical side (effect lags cause). And I stop when I find a significant result. So for example, if I find a significant relationship after I’ve looked at lags of 0, 1, 2, and 3 time units, then I’ve examined four series. Accordingly, to achieve the equivalent of a 95% significance level (climate science standard) my adjusted signficance level is X^4 = 0.95, which solves to x = 98.7% or a p-value of 0.013. The series at lag 3 needs to have a p-value of 0.013 or better to be considered statistically significant.
The net result of all this is that as I look at longer and longer lags, the required p-value gets harder and harder to achieve.
Now, that doesn’t mean I don’t use cross correlations. I do. But I also know from experience that if I don’t find a strong relationship within a fairly short lag period, significance gets hard to achieve pretty fast.
And of course, sometimes the relationship is quite strong, and so none of this is an issue. In the world of sunspots and sea level, however, this is not common …
Finally, regarding the Holgate data shown in Fig. 1 above, the strongest relationship is at lag 0, and it isn’t statistically significant … nor are the lagged relationships.
Best regards,
w.
lsvalgaard says:
January 26, 2014 at 10:30 am
……….
I certainly would be interested in your updated observations. I did quick superficial ‘survey’ of the annual Ap index ( cycles 10-23) and found that (for cumulative value) the odd cycles have on average of 6% more than the even ones (two exceptions are SC10 and SC23)
Willis, the plot you pulled up there was a late night data processing mix-up which I corrected in another post a few minutes later.
A better labelled version of the latter graph is here:
http://climategrog.wordpress.com/?attachment_id=759
Both of those were power spectra. The correlation plot with the signif estimations is here ( I’ve already posted this too but here again for clarity ).
Now we’re using different sunspot data but that SSN is not substantially different from sunspot area in general form. It should not make a major difference, so I’m a little confused that your x-correl looks to be about pi.2 out of phase with mine. It looks like one of us has made slip up on d/dt, or something.
The other thing I don’t understand in what you show is “R^2” goes negative. Usually squares are positive, I thought R^2 went from 0..1 , but I could be wrong. If not , why do you discount the negative values which look more significant than the +ve ones. Neg. correl is just as important as +ve one.
Also typo “R^2 = 0.005” has an extra zero.
Finally 0.10 is presumably your signif estimation. As lag increases the dataset gets shorter and the required correl coeff for significance should rise. Can you run your plot from -60…+60 to see whether you get the same long term modulation I found?
We should be getting substantially the same thing, or at least have a good explanation why not.
richardscourtney,
The moving of phase is an illusion that is the result of echo interference (to over-simplify).
Imagine a signal
y(t) = Cos[w t]
Now imagine there is an signal an ‘echo’ signal
y(t) = 0.5 Cos[w (t-100)];
An echo could be at any delay and have any relative phase. This will interfere with our signal to move the phase of the output that we can see (even possibly moving it forward).
Now this is an exaggerated example of course and I’m just using it to illustrate how past signals can interfere. The reason that linearity is important is that echoes in linear systems will have a constant phase shift. In chaos we can not assume that. Amplitude, phase, and shape of the echo may be different every time.
Unfortunately correlation is all we have! When all you have is a hammer, every problem looks like a nail. We should still use it for chaotic systems (we don’t have a choice), but we should also be aware that things are harder to ‘see’ here. Correlation metrics may be lower than they would otherwise normally be.
vukcevic says:
January 26, 2014 at 11:58 am
I certainly would be interested in your updated observations. I did quick superficial ‘survey’ of the annual Ap index ( cycles 10-23) and found that (for cumulative value) the odd cycles have on average of 6% more than the even ones (two exceptions are SC10 and SC23)
The cycles should be counted from polar field reversal to polar field reversal which is roughly from max to max, not as normal cycles from min to min.
Willis,
Thanks for following up demonstrating a cross-correlation for the Jevrejeva data versus sunspots. You didn’t even have to bother with the random counter-example, but for completeness, sure. In signal processing if their are no spikes significantly above the background level – there is nothing there.
Now sorry if I’m confused, but is this the same data as in the graph above in your original post? I see different Scientists’ names and very different R^2 values being thrown around. I’ll assume it is, but just double checking.
OK, I’ve figured out your R^2=0.005 is from corr-coeff of -0.07 , fine.
BTW my comment was posted in between your two above and thus related to the first one January 26, 2014 at 10:52 am
Still don’t get the differences between our two graphs.
Ian Schumacher:
Thankyou for your explanation for me at January 26, 2014 at 12:15 pm.
Allow me to see that I have understood your mathematical illustration.
1.
The effect is periodic because its cause is periodic.
2.
Thus, the cause and the effect each consists of a series of peaks.
3.
A peak of the cause may have an ‘echo’ which is an additional small peak.
4.
The ‘echo’ induces an additional but small peak in the effect.
5.
The additional but small peak in the effect occurs shortly before the next periodic peak of the effect.
6.
Signals are additive, so the additional but small peak in the effect adds to the start of the next periodic peak of the effect.
7.
Thus, the next peak in the effect is distorted by the ‘echo’.
8.
The distorted peak in the effect occurs before the maximum of the next periodic peak in the cause.
9.
Thus the distorted peak seems to occur before its cause.
have I understood you correctly, please?
The reason I ask is that I have played around with your illustration and I fail to obtain an effect similar to cycle18 in the above graph. In that case a peak in SLR occurs before the solar peak but it is accompanied by a very small SLR peak coincident with the solar peak.
I appreciate that your maths were only an illustration and reality would differ. But I do not see how to amend the equations such that introduction of an ‘echo’ can both add to the effect before its next peak and also subtract from that next peak.
I am assuming you have been doing this stuff so would be grateful if you could explain this for me.
Richard
richardscourtney says:
I hope I understand your question properly. Here is a concrete I came up with. In Mathematica you can use
Plot[{Cos[t], Cos[t] + .5 Cos[t + .9]*Exp[-(t – 5)*(t – 5)]}, {t, 0, 14}]
There is an echo here that comes in ands fades away again.
The resulting plot looks like this:
http://imgur.com/Y23geir
The blue line is the unaltered signal. The purple line is the signal plus echo. [Weird that Mathematica chooses colors so close together].
Now I get an r value of 0.55 even with simple detrending for secular trend removal, consistent with Shaviv 2008, considering only data for the years 1919.5 – 2002.5.
Ian Schumacher:
I apologise if I am seeming obtuse. It is not deliberate. I am genuinely trying to get to the bottom of this in my understanding.
Thankyou for your fine illustration at January 26, 2014 at 1:37 pm .
Yes, your link does show what I found (several ways) as my points 1 to 9 state in my post at January 26, 2014 at 1:19 pm. As I there said
So, yes, you did answer my first question, and I apologise if I was not clear in saying that.
Thankyou for what you have provided. Clearly, I put you to additional and pointless work by failing to be clear and thanking you for what you have provided. Sorry.
But my playing around with the waveform additions posed another problem for me. And I asked a new question; viz.
The inherent assumption in my question is that the effect seen in ‘cycle 18’ is caused by ‘echoes’. This, of course, may not be true because SLR variation may be induced by several factors. However, if that assumption is not valid then any assessments which assume effect of ‘echoes’ are all also not valid for the same reason.
I genuinely appreciate your efforts so far and, therefore, I know my continued questions are an imposition. But I take the liberty of pressing the matter because, as I said,
Thanking you in anticipation
Richard
Greg Goodman says:
January 26, 2014 at 11:59 am
My plot is not showing R^2, it’s showing correlation, so it goes negative. I just noted the R^2 for the maximum correlation, which indeed is 0.005.
To avoid endless discussion, let me post the sunspot data, the jevrejeva data, and the R code I used. Hang on … OK, here’s the complete code to make the graph:
thespots=ts(read.csv("SIDC Sunspots.csv")[,2],start=1700.5,frequency=1) thesea=ts(read.csv("jevrejeva sea level.csv")[,2],start=1700.5,frequency=1) ccf(diff(thesea),thespots, main="Cross-Correlation, Sunspots and ∆ Jevrejeva Sea Level", ylab="Correlation",col="salmon",lwd=3)It pulls the data from the two csv files, “SIDC Sunspots.csv“, and “jevrejeva sea level.csv“, which need to be in the current workspace. The sunspot data is modified per Leif Svalgaard (and the upcoming SIDC revision) by increasing all pre-1947 values by 20% to account for the change in counting methods. Jevrejeva data is from KNMI.
Let me know what you find … code as used and data as used, the simplest way to answer all such questions. English is generally inadequate for resolving this kind of question.
w.
One more thing, Greg. I suspect the best way to see if the sunspot vs. Jevrejeva results are significant would be to do a monte carlo analysis against red noise with the same characteristics as the Jevrejeva data. The underlying problem is the strongly repetitive nature of the sunspot data. It’s not like comparing two datasets that are both non-cyclical.
Anyhow, I may get to that today, I’ll post up the results.
w.
@ richardscourtney
Sunspot 11 and 60+ years climate cycles among many others, are (in my view) forms of forced oscillations where the natural frequency (Fo) may be different from the forcing (driving) frequency Ff.
Under certain circumstances depending on the relationship of two frequencies (Ff = or > or < Fo) and when the driver isn’t very stable or contains number of higher harmonics i.e a pulse, the responding oscillator may occasionally (for a cycle or two) move in advance of the forcing one.
Greg, the question of a monte carlo analysis of the cross-correlation wouldn’t let me rest … it turns out that the cross-correlation between sunspots and the Jevrejeva data is nothing but chance. Here’s a typical run from a monte carlo analysis, many of them look like this:
For comparison, here’s the real cross-correlation:
I’m sure you can see the problem … as I said, the difficulty is that one of the two datasets is strongly cyclical, so it will generate cyclical cross-correlations against random data.
w.
[UPDATED TO ADD] The code, the code. I first measured the AR and MA components of the Jevrejeva data, and generated random ARIMA datasets with those values. Then I differentiated those proxy datasets to give me the delta values.
Then I simply measured the CCF of the sunspot data versus the random data. This should be turnkey, uses the same data as above:
thespots=ts(read.csv("SIDC Sunspots.csv")[,2],start=1700.5,frequency=1) thesea=ts(read.csv("jevrejeva sea level.csv")[,2],start=1700.5,frequency=1) detrend=function(x){ xlm=lm(x~c(0:(length(x)-1))) xlm$fitted.values=c(xlm$fitted.values,rep(NA,length(x)-length(xlm$fitted.values))) (x-xlm$fitted.values) } # get the arima variables seaarima=arima(detrend(as.vector(thesea)),c(1,0,1)) testsize=10 # make proxy data proxydata=arima.sim(list(ar= seaarima$coef[1],ma= seaarima$coef[2]),length(thespots)*testsize) # convert it to time series testbox=ts(matrix(proxydata,nrow=314,ncol=314*testsize),start=1700.5,frequency=1) # get the annual differences testdiff=diff(testbox) # cross-correlation ccf(testdiff[,3],thespots, main="Typical Cross-Correlation, Sunspots and Random Data", ylab="Correlation",col="salmon",lwd=3)</pre.vukcevic:
Thankyou for your post at January 26, 2014 at 2:38 pm.
Yes, I thought of that some time ago but I was concerned that stating the possibility would imply I was ‘playing games’ with Ian Schumacher, and I am not.
Basically, it is possible that the SLR oscillates naturally for some reason at near to the frequency of the solar cycle, and the possible small solar influence causes the SLR variation to adjust to get into synchronicity with it.
If so,
(a) the solar cycle is causal of the synchronicity of the solar cycle and SLR
but
(b) the solar cycle is NOT causal of the SLR fluctuation.
It is a nice idea because it overcomes the problem of coherence: the synochronicity would not be precise and would drift around the frequency of the solar cycle (coupled pendulums can do this).
Please note that I am NOT suggesting this is true. I am recognising a possible relationship between the two parameters which would overcome the coherence problem. And it overcomes the ‘cycle 18’ problem because the cause of the SLR fluctuation is something other than the solar cycle.
And that is why I have not mentioned it in my discussion with Ian Schumacher: it rejects a causal effect of the solar cycle on the SLR fluctuation. Induced synchronicity between the variations of the two parameters is very different from one parameter causing the other. It rejects that the fluctuation of the SLR is caused by the solar cycle and assumes the SLR fluctuation is caused by something else.
I hope this makes clear that I am not rejecting your suggestion, but I would like to ‘get to the bottom’ of the ‘echo’ suggestion.
Richard
richardscourtney,
Is it the small coincident peak combined with the delayed peak that’s bothering you? The small peak may just be noise or something else. In fact all of it could be noise. Who knows 🙂
My example shows how an echo could make a peak occur ‘before’ the signal peak. The purple line occurs ‘before’ the cause (the blue line). This is an illusion of shifted causality due to the periodic nature of the signals, it’s not a real shift in causality. I’m not sure if I’m able to explain it better.
In addition to that, three things:
– There is background noise. We are not looking at the ‘pure’ result. It is obscured by random ups and downs. A ‘peak’ may look to be shifted merely because random noise added and took away from the signal at the right time. A small false peak may just be noise and nothing more.
– There are other external signals at work here. PDO, or whatever it’s call. Volcanic eruptions, hurricanes, tsunamis, etc I don’t know which if any of these is important as I’m not a climate expert, but presumably they all could be. Overlapped at the right time it could cause peaks to occur and obscure our signal anywhere they occur, or their echo occurs.
– There may actually be no real correlation and I have conceded that. It’s an interesting overlap, but it may just be coincidental.
Not sure if that helps or not, but I don’t think I can add much more of value.
Ian Schumacher says:
January 26, 2014 at 12:15 pm
Interesting … and noted. However, see below.
The problem I’m having is in understanding how we can have an echo of say sunspot data in the climate system. The thing about an echo is that it is a reflection of something. With say an electrical signal or a sound signal, that makes perfect sense. The sound goes out … the sound comes back. All it takes is a signal, and something for it to be reflected from …
But what does a “sunspot echo” look like or consist of? And what is it bouncing off of? Most people, myself included, see the only possible means of the sunspot cycle affecting the planet as being electromagnetic. You know, via cosmic rays, or even directly via the heliomagnetic field.
But if that is the case, what would the “echo” of that magnetic field look like? And what would reflect it? In particular, one characteristic of an echo is that whatever it is echoing off of needs to be physically removed from the location where the echo is measured. An electric signal might bounce off of an impedance mismatch further down the wire. A sound echoes off of the distant cliff or the far side of the auditorium.
But what would a change in magnetic field echo off of?
This is why I have been pressing people for some other real-world example of what people think they see in figure 1, some process that is out of phase, switches to in-phase, stays that way for decades, switches back again, and where cause sometimes leads and sometimes follows effect …
Yes, as good old Joe Fourier showed, you are right—name a signal, we can gin up a combination of cosines that matches it exactly. Or we can go old-school, model it with delays and transformations and echoes of a couple of signals, phase-shift it forwards and back, anything we want.
But in the real world, only a very small subset of those infinite possibilities actually occurs. As a result, although it is possible to come up with waveforms that lead to what we see in Figure 1, in the real world that kind of thing is very rare. I look, day after day, at natural waveforms of all kinds. I see them inside my eyelids in bed at night. That kind of frequency flip-flop is very unusual.
Can’t say fairer than that, I agree entirely.
w.
Ian Schumacher:
You end your post to me at January 26, 2014 at 3:13 pm saying
You have helped a lot and provided much of value.
Many thanks.
Richard
Let me chime in with what I hope with become an “open question” in this topic, though perhaps not providing an answer in its own right amongst the actual specific issue of sunspots and lunar cycles.
We have two cyclical phenomena, right? The two appear to be slowly (over time) sliding from out-of-phase -> to in phase -> to (perhaps) back out-of-phase with each other.
Should we not question the relationship – but be ready at the same time to admit that the two may not, in fact, related at all? After all, sunspots are themselves only a “symptom” of the underlaying solar currents and local magnetic fields in localized “storms” on the sun’s surface. That the sunspots change over time is obvious, but that the sunspots themselves cause anything to change elsewhere is a “coincidence” of two “grandchildren” or two “cousin” effects being changing in phase (or out-of-phase or lagging-in-certain-phases) because the original “grandparent” cause is pulling their chains slightly differently. Should we not be willing to look very, very closely for the cause of the “original change” in the original effect that the sunspots themselves merely highlight?
For example, if I try to link the “cause” of the moon with tides, surely I’d first get a number of tidal charts from a number of different cities around the world, then plot the moon’s position overhead locally with the tides at those spots, right?
Doesn’t work to well though. London is upstream from the Thames outlet to the English Channel, Halifax is on the south side of a coastline mere few miles from the highest tides on the planet, the River Seine behaves differently that does the Sacramento River dumping into San Francisco’s Bay, Portland and Vancouver behave differently over the year but both are on bays on the Pacific Northwest, and finally New Orleans is completely out-of-phase with the Mississippi river and its irregular flood season and nearby hurricanes – completely different from Pensacola or Mobile right nearby on the Gulf Coast.
Just in this simple analogy, you need to separate irregular hurricane tides from the sun’s neap tides from river back-flooding in spring (maybe, or maybe not, or maybe twice) from the basic and initial not-quite-two-tides-a-day lunar effect. All of that with the reverse flooding and ebbing in the surrounding large bay or Channel (Chesapeake, Fundy, San Francisco, Med. etc) On a problem this simple, could you spot the very small change due to the sun’s position with respect to the moon each month?
OK, thanks for the code. Much quicker as you say.
Firstly sunspot daily data runs from 1877 and was similar to the period used in fig2. You are using all the data back to 1700. Cropping to 1887 got the phase of the first peak similar to my plot, however, the overall envelop is still like your plots not with modulation max in middle . It seems that is due to differences in sunspot data. I need to look in more detail to find out why. Could be a fundamental difference between SS area and SSN, or a result of the Svalgaard ‘correction’.
Next question is waht does R do when you give it monthly data and tell it frequency=1 ?
My guess is a straight annual average, which is not legit resampling. Could be producing aliasing that would produce spurious longer cycles.
I’ll look closer at whether it’s due to the different solar data.
Willis Eschenbach:
In your post at January 26, 2014 at 3:24 pm you say
Yes. And that is why I have been pressing the coherence issue and not the correlation issue.
As you note, Ian Schumacher has been very helpful on the ‘echo’ idea. However, despite his kind help, I remain unconvinced mainly because I have no idea what an ‘echo’ would be (which is why I have consistently put the word ‘echo’ in inverted commas).
Please read my post at January 26, 2014 at 3:08 pm which is here
I do not want to mislead by implying that I think there is any solar effect on SLR: I don’t. But I keep an open mind on possibilities and the synchronicity idea is plausible but – so far as I can see – not useful except as a spur to determine what could be a connection between solar variation and SLR.
Richard
” – but be ready at the same time to admit that the two may not, in fact, related at all? ”
Indeed. I’m still not convinced the link is strong. However, the cross-correlation looks very structured with the sunspot area data I used.
http://climategrog.wordpress.com/?attachment_id=760
RACookPE1978:
re your post at January 26, 2014 at 4:24 pm.
Yes, there may be no connection at all. Indeed, at present all we have is the analysis which Willis has conducted which indicates there is no direct causal relationship. (If you check up the thread you will see I had an unpleasant interaction with someone who refused to accept this.)
So, the present discussion is about finding ways in which the finding by Willis may be incorrect although his analysis has not been faulted.
Personally, I don’t think there is a relationship between solar cycle and SLR variation but – as in all things – I would welcome being shown to be wrong.
Richard
Willis Eschenbach,
“But what does a “sunspot echo” look like or consist of? And what is it bouncing off of?”
Well I’m speculating of course and know almost nothing about climate science, so humor me on the details.
Where does the PDO come from? Where do any of the ocean oscillations come from? I don’t mean it as a trick question. I hear PDO talked about all the time, but no one talks about why it exists in the exact form that it does (not that I have seen anyway). For the sake of argument I’m going to assume that there is some sort of delayed feedback (as that’s how most oscillations occur) through evaporation and ocean mixing. Evaporation creates a heavy salty layer, which reaches a threshold and falls deeper into the ocean bringing up cold water to replace it. Or at least ‘something’ like that. Presumably there are several of these types of oscillations all over the climate and this is just one we have noticed.
What are sunspots supposed to ‘do’ that alters climate. Well maybe they warm the Earth directly through increased insolation, or maybe the increase in cosmic rays create more clouds and so on. The end result is that ‘somehow’ there is supposed to be a corresponding change in energy.
Where does an increase in energy go? Well it should make the world a little warmer immediately of course, (unless it’s some really exotic mechanism like melting ice, or evaporating water directly through high-energy photons, that would be cool, pun intended), but also it goes into driving a slight increase in the amplitude of Earth’s natural oscillations and it’s disappearance of this extra energy allows the system to decay back to a previous level. Or maybe not. We can’t tell with chaos. It may require an off phase event to bring it back, or it may never come back. Anyways, that’s the ‘echo’ – the delayed response to a previous signal. Sunspots could change amplitude of existing cycles.
Now again, just guessing, but I bet these oscillation dynamics are non-linear. A particularly strong sunspot event may evaporate water faster than a weaker sunspot event (assuming that is the correct mechanism) and may actually speed up the PDO frequency for a time. That’s a different kind of echo. We are effecting how the system operates in the future by some event now. Depending on amplitude, sunspots could temporarily or permanently alter frequency and/or phase of existing cycles.
On a different note, even though many people on WUWT appear to treat PDO as sacrosanct. It may, in it’s current form, be a completely temporary phenomenon. Chaotic systems occasionally switch into a temporarily stable oscillation all the time, only to suddenly go off wildly with no warning into a different temporarily stable oscillation (think Lorentz butterfly – see http://en.wikipedia.org/wiki/Butterfly_effect in the unlikely event anyone reading this doesn’t know what I’m talking about). There are probably ‘butterfly’ equivalents all over the climate system.
I hope that was adequate. Chaos theory is a hobby of mine and I’m not an expert. I have a background in mathematical modeling and signal processing.
On a side-note and not to step on too many toes, PDO to me looks really noisy. I’m not actually convinced it’s a stable periodic phenomenon at all [http://en.wikipedia.org/wiki/File:PDO1000yr.svg ]
Willis,
It was hard work, but I think you managed to convince almost everyone here (including me) that the correlation (in every sense of the word) is too weak to be meaningful.
I downloaded the sunspot data from the link you recently posted took one look at the noisy sea level data and threw it away. The only signal in there I can see is the obvious one – ocean level is increasing over time. In many ways it feels like statistics has ruined science. When you can’t ‘see’ a signal and need to perform many dubious processing steps to tease out what you think should be there, the chance that you are just seeing an artifact of one of your processing steps is immense and difficult to completely compensate for.
RACookPE1978 says:
“Should we not question the relationship – but be ready at the same time to admit that the two may not, in fact, related at all?”
Is there something in the Universe that affects both Earth and the Sun? It’s a possible but very exotic theory 😉 What would be the other external thing be? Can we measure it? If you can measure it then you have a way to test your theory.
“The only signal in there I can see is the obvious one – ocean level is increasing over time. ”
Ian , you should read the Jevrejeva papers 😉
Willis, I’ve run Leif’s reworked SSN , 1877 onwards, through the same processing I did with sunspot area and get essentially the same result (just one minor peak moved from 16.75 to 17.4).
so the huge difference in form of the cross-correlation is in the processing not the data….
I said: ” Cropping to 1887 got the phase of the first peak similar to my plot, however, the overall envelop is still like your plots not with modulation max in middle .”
No. I’ve now got essentially the same x-correl plot from my code with data from 1750 on (earliest common date) .
Earlier incorrect report because setting start=1877.5 did not seem to do anything different and apparently still used all the data. That’s basically why I stopped using R. I had to spend more time double checking it did what I asked than I spent doing the job I itself.
thespots=ts(read.csv(“SIDC Sunspots.csv”)[,2],start=1877.5,frequency=1)
thesea=ts(read.csv(“jevrejeva sea level.csv”)[,2],start=1877.5,frequency=1)
ccf(diff(thesea),thespots, lag.max=60 , main=”Cross-Correlation, Sunspots and # Jevrejeva Sea Level”,
ylab=”Correlation”,col=”salmon”,lwd=3)
Now if you look at the jevre’ data there is a sudden doubling of the magnitude of variation pre-1900. This may be sampling errors or real climate variation: SST shows much larger variations too as does BEST land data.
Now I take the point made in your red noise tests about the solar cycle coming through, however this does not reproduce the 70 year envelop.
How about putting lag.max=100, limiting the data to 1877-ish, and seeing whether any random runs can produce anything like it? I rather doubt that will happen.
Ok, please allow me try to recapitulate:
1. We have that picture above, with clearly visible approx 11 year cycles in both data sets.
2. Actually, from first thought, doubt arises not whether a relationship exists, but much more why it appears to be so strong. Who would have thought that ?
3. For McIntyre this appears to be, offhand, the best decadal matching of any two climate variables.
4. Then, we have various analysis, applying methods which prima facie are not able to extract that 11 year pattern.
5. Correlation cannot handle the 1 or 2 major phase jump and also the amplitude mismatches.
Quote Willis Eschenbach says: January 23, 2014 at 8:05 pm : “… which just reveals the weakness of the method.”
6. Lagged correlation can’t do either.
7. Same problem with phase jumps and Fourier.
8. Then we have a discussion about a lacking physical explanation for small lags, which, due to noisy data and other influences working on sea level, may be totally explainable, and making that physical explanation redundant from the start..
9. Then there are comments, appearing to better address the visible relationship, but remaining totally ignored.
10. Shaviv 2008 reports to have improved correlation to 0.54 and p value to 0.0001 by removal of secular trends. How did he do that ?
11. I get a similar correlation value (for Shaviv 2008), if I remove the first 19 years of data, which may also be the most unreliable part of the data set.
12. Paul Westhaven reports an interesting strategy to extract a signal (by “forcing into correlation” and “examining the residuals”)…
Manfred says:
January 26, 2014 at 7:09 pm
10. Shaviv 2008 reports to have improved correlation to 0.54 and p value to 0.0001 by removal of secular trends. How did he do that ?
It seems to me that the secular trend would be the most important aspect of the whole matter. That would be the first-order response. Who would care about a small second-order wiggle on top of the dominant long-term trend?
“3. For McIntyre this appears to be, offhand, the best decadal matching of any two climate variables.”
I would not make too much of that comment which was clearly just an initial visual assessment. Note the “offhand” qualifier.
“7. Same problem with phase jumps and Fourier.”
Assuming there’s reason for the phase jumps spectral analysis will likely explain them.
One feature of interference patterns between close harmonics is an abrupt phase change. The point in the x-correl plot I marked at 38 years is a clear example.
http://climategrog.wordpress.com/?attachment_id=760
The 1975 flip in N. Pacific SST is another example but this gets lost when it is processed into PDO ” empirical orthogonal functions”.
lsvalgaard says:
January 26, 2014 at 7:27 pm
Manfred says:
January 26, 2014 at 7:09 pm
10. Shaviv 2008 reports to have improved correlation to 0.54 and p value to 0.0001 by removal of secular trends. How did he do that ?
It seems to me that the secular trend would be the most important aspect of the whole matter. That would be the first-order response. Who would care about a small second-order wiggle on top of the dominant long-term trend?
===
Detrending is similar to taking first difference since a linear “trend” becomes a constant and then would not affect the correlation. First diff would be a means of removing auto-correlation also. That would be a more justifiable strategy than arbitrarily subtracting an artificial linear trend from data that almost certainly has no reason to be displaying such linear behaviour.
I have not read Shaviv 2008 to see exactly what he did.
Greg Goodman says:
January 26, 2014 at 4:24 pm
OK, thanks for the code. Much quicker as you say.
Since both the Jevrejeva sea level data and the SIDC sunspot numbers went back to 1700, that’s what I used.
The function ts() in R converts a string of evenly spaced data into a timeseries object. The for annual data starting in 1700 like the sunspot data the syntax is something like
my_timeseries = ts(somedata, start= 1700.5, frequency = 1)
If you have monthly data that starts in March 2000, like the CERES data, the syntax would be
CERES_ts = ts(CERESdata, start = c(2000,3), frequency=12)
No, there’s no averageing or resampling. The ts() function doesn’t do anything at all to the data. It just adds an (invisible) time string to the data so you can window the data or plot it easily and without ambiguity. There’s another kind of object, a “zoo” object, for irregularly spaced data.
w.
Greg Goodman says:
January 26, 2014 at 7:40 pm
Detrending is similar to taking first difference since a linear “trend” becomes a constant and then would not affect the correlation.
Sometimes what is observed has an inherent trend. Let me give you a realistic example: it is generally accepted that magnetic fields on the Sun are the main cause of variations of TSI [the total Solar Irradiance, which a priori must have an influence on the climate]. Sunspots are manifestations of surface magnetic fields and so should have an influence on TSI [should make TSI smaller as spots are darker than the average surface]. This is observed, as TSI dips when a large spot is on the disk. Sunspots decay by having their magnetic field ‘shredded’ into many small ‘strands’ that are moving away from the spots. This creates an area around the spot where many such strands of magnetic field can be found. As the magnetic field in a strand exerts a pressure of its own, less gas is needed for pressure balance so we can see deeper into the partly evacuated atmosphere inside the strand [and in particular see the ‘hot walls’ inside the strand; this is also observed]. This makes the debris field brighter than the surrounding surface. In fact, outshines the deficit caused by sunspot by a factor of two, so on the whole, many spots + their debris fields increase TSI. This would create a solar cycle variation of TSI, as observed. Now, assume that there are an immensity of tiny [and therefore hard to observe] magnetic elements not directly associated with sunspots erupting all over the Sun and that their rate of eruption would be variable, perhaps on a longer time-scale [centuries or longer]. This would create a long-term, secular trend in TSI, potentially much larger than the variations due to sunspots and potentially of much larger importance for the climate. In fact, such a ‘background’ is a much discussed item in solar physics today [see http://www.leif.org/research/Long-term-Variation-Solar-Activity.pdf ]. We do not know if a background exists [personally I think not, but that is just my well-founded opinion], but some researchers claim so. So here you have a case where detrending could remove the real and important signal and thus critically alter the correlation between solar magnetism and climate. As I said: “Who would care about a small second-order wiggle on top of the dominant long-term trend?”
Ian Schumacher says:
January 26, 2014 at 4:46 pm
Thanks for the interesting thoughts. Yes, there can be delayed actions, no question. What I was questioning was your specific example from signal analysis, an “echo”. And you showed examples.
It seems now that you didn’t mean an echo, but some other delaying mechanism involving the transformation of the energy into another form. You see, a sound echo comes back as sound. An electronic echo comes back as an electromagnetic signal.
It seems now that you’ve abandoned that, and are talking about a different concept, which is fine. I’m just trying to follow. The new concept is that something associated with sunspots somehow (electro-magnetically?) affects some unknown process on the earth in such a way as to suddenly flip the phase of the response to the sunspots from negative to positive …
I’m still not seeing it, Ian.
Regarding your speculations on the PDO, interesting ideas on an interesting issue. You might enjoy my analysis called “Decadal Oscillations Of The Pacific Kind” …
Finally, you say:
Ummm … that chart is a reconstruction of the PDO from tree rings … possible, but the error bars would go floor to ceiling. Which may be why there aren’t any … Take a look at my paper linked above, Figure 4 shows the cumulative PDO which is much clearer about the existence of the phenomenon.
My best to you,
w.
Greg Goodman says:
January 26, 2014 at 7:40 pm
Detrending is similar to taking first difference since a linear “trend” becomes a constant and then would not affect the correlation.
Another realistic example: solar activity modulates the flux of Galactic Cosmic Rays [GCRs] by a few percent. The Earth’s magnetic field shields us from much of that GCR flux. The Earth’s magnetic field is a much stronger modulator of GHCs than the Sun is. The field varies on timescales of centuries or longer and cause the flux of GCRs to vary an order of magnitude more than the sorry wiggles due to the Sun. If GCRs have an influence on the climate, detrending the GCR flux will remove that large, important and first-order effect and leave us to fight with the noise. As I said: “Who would care about a small second-order wiggle on top of the dominant long-term trend?”
Manfred says:
January 26, 2014 at 7:09 pm
I love the plan. Who would have guessed that you could actually improve the correlation by the simple expedient of throwing away the data that doesn’t agree with your theory! Brilliant!
… [/sarc] …
Manfred, I’ll take that as your convoluted way of saying you couldn’t reproduce Shaviv’s results. Hey, I couldn’t either, as I reported above.
w.
lsvalgaard says:
January 26, 2014 at 7:27 pm
Thanks, Leif. That’s what I was pointing at in the Parana river post when I said:
In that case, it inconveniently didn’t fit with their narrative, so they just subtracted out the secular trend and went on … I just shook my head.
w.
For the nitpickers: “The Earth’s magnetic field is a much stronger modulator of GCRs than the Sun is”
lsvalgaard says:
January 26, 2014 at 8:19 pm
Dang, I love the web, it’s like attending the best school EVAR!
Leif, do you have a link to some kind of long-term measurement of the geomagnetic field? The sucker has to be monstrously complex, being as how it’s a 3-d vector or maybe tensor field. Dang … that kind of thing can’t be boiled down to one number, so I suppose they do what? Measure field strength in othogonal directions? Makes a man’s head hurt … any references to data and descriptions would be appreciated.
w.
Greg Goodman says:
January 26, 2014 at 7:40 pm
Detrending is similar to taking first difference since a linear “trend” becomes a constant and then would not affect the correlation.
Yet another example: due to various geometric effects there are [weak] 22-year cycles both in GCRs and Geomagnetic Activity. These [tiny] second-order effects are supposed to cause 22-year cycles in climate and all kinds of other things. If they do, one would expect a much larger effect from the much larger first-order variations that dominate over those tiny 22-year variations. As I said: “Who would care about a small second-order wiggle on top of the dominant long-term trend?”
Willis Eschenbach says:
January 26, 2014 at 8:35 pm
Leif, do you have a link to some kind of long-term measurement of the geomagnetic field?
Lots of such links. The single quantity that is much used is simply the dipole moment [the strength of the main field]. That quantity determines, among other things, the size of the Earth’s magnetosphere [stronger diple, larger magnetosphere]. Here are some links: [Google is your friend] http://seismo.berkeley.edu/~rallen/eps122/lectures/L05.pdf and http://earthref.org/ERDA/973/ and an old [but still valid] Figure: http://www.leif/org/research/CosmicRays-GeoDipole.jpg
haste is waste: http://www.leif.org/research/CosmicRays-GeoDipole.jpg
Greg Goodman says:
January 26, 2014 at 5:58 pm
Actually, you’re missing the beauty of timeseries objects. Methods like cross-correlation work differently on different kinds of objects in R. In this case, when you do a cross-correlation between two timeseries objects, the language is smart enough to only do the cross correlation on the period when they both have data. For example, the following creates two time series objects, called “thespots” and “thestream”:
thespots=ts(read.csv("SIDC Sunspots.csv")[,2],start=1700.5,frequency=1) thestream=ts(read.csv("parana streamflow.csv")[,2],start=1904.5,frequency=1)Note the different starting dates. They also have different ending dates, 2003 for the streamflow and 2013 for the sunspots.
Now, both of the following two statements do the same thing. The R function “window” extracts a section from a time series from some given start to end dates. The first statement below cross-correlates streamflow with a “window” on the sunspot time series object “thespots”:
ccf(thestream,window(thespots,start=1904.5,end=2003.5), main="Cross-Correlation, Sunspots and Parana River Flow", ylab="Correlation",col="salmon",lwd=3)That statement is correlating the full streamflow dataset, 1904-2013) with a window into the sunspot dataset that covers exactly that time period, so both timeseries objects have the same length, 1904 to 2003.
That code does exactly the same thing as the simpler version:
ccf(thestream,thespots, main="Cross-Correlation, Sunspots and Parana River Flow", ylab="Correlation",col="salmon",lwd=3)because R knows enough to just use the common time period if it is cross-correlating timeseries objects.
w.
Willis,
Yes, I can see how my ‘echo’ term could be misunderstood as an actual echo of sun spot energy or something like that. No, I did not mean that. I just used the word ‘echo’ originally because I wanted to emphasize that it was similar in shape to the original. Unlike linear systems the delay could be amplitude dependent and time-variant, which provides a mechanism to explain shifting phase relationships.
As for completely flipping phase 180 degrees this way, probably not. A Lorentz attractor type situation would be a much better way to explain such a sudden and hard phase shift like that. I’m not speculating that exists in this case, merely that such a system would occasionally show sudden drastic phase changes of exactly this nature.
I’ll look at your PDO research. I hope that wikipedia graph isn’t representative, otherwise, yes it would look of dubious utility.
Thanks for comments lsvalgaard .
Willis, re. circa 9.2 modulating the 10.x : cross correlation in tropical Atlantic:
http://climategrog.wordpress.com/?attachment_id=761
Link therein to Keeling & Whorf who did much of this 20 years ago. The manifestations of various lunar periods seems much stronger than the putative solar 10.x
A useful run of Arctic ice data were hot available to Keeling that far back but I find the 27.6 days he refers to and also the synodic lunar period is most evident though its “winter” period of 29.94 rather than the annual average of 29.53 usually quoted.
http://climategrog.wordpress.com/?attachment_id=756
DJF period of Arctic Oscillation is also noted to have most effect on coming ice year.
Now of course one isn’t going to find that kind of detail by looking at monthly averages and it does require a good understanding of interference patterns.
Here is my quick and dirty look at the data – http://i.imgur.com/siFb1Xx.png
I ‘detrended’ sea level by fitting to a parabolic equation and subtracting this trend.
First strange thing right away is that the data is completely non-stationary. Look the windowed variance:
http://i.imgur.com/GiuAcLV.png
Now this means one of two things:
1) Underlying data was actually smooth, but crude technology measurements initially added error.
2) The underlying system changed (chaos style).
At first I thought #2 was unlikely and probably sometime around 1880 they suddenly had better measurement techniques. But then look at where the best correlation occurs. Only before 1880. After that, the correlation completely falls apart.
In addition, the phase lag between sunspots and sea-level is growing throughout the first half of the graph (presumably until the synchronization simply ‘breaks’ around 1880).
Option #2 could explain a lot. It could explain the apparent ‘eye’ ball correlation before 1880, but the poor mathematical correlation (because phase lag is growing and linear correlation metrics don’t like that). The two systems start off in phase and slowly fall out of synch until finally becoming completely disconnected.
That’s my crazy theory for the day anyways 😉
lsvalgaard says:
January 26, 2014 at 8:46 pm
Google definitely is my friend … but it’s better to have a scientific advisor. The lecture notes you linked to in particular are very clear.
You mentioned why worry about the small variations, how about the big ones. From what I see, the earth’s geomagnetic at 3000-6000 nanoteslas is about a thousand times greater than the sun’s heliomagnetic fied at 3-6 nT … dang, I hadn’t realized that it was that great a disparity.
w.
Willis Eschenbach says:
January 26, 2014 at 11:19 pm
From what I see, the earth’s geomagnetic at 3000-6000 nanoteslas is about a thousand times greater than the sun’s heliomagnetic fied at 3-6 nT … dang, I hadn’t realized that it was that great a disparity.
It is even greater, namely 10,000 times as the Earth’s field at the surface is about 60,000 nT. Since the field comes from the core, it is much stronger [about 10 times] down there. The field falls off with the cube of the distance so it gets down to solar wind magnetic field strength pretty quick. The field holds off the solar wind at a distance about 11 Earth radii on the sunny side of the Earth.
Hi Willis
I think Dr. S sent you back to the secondary school. I think this may be more appropriate.
http://www.ngdc.noaa.gov/geomag/WMM/data/WMM2010/WMM2010_Report.pdf
My answer to your questions on Parana thread are stuck in the mod’s bin for the last 4 hours or so.
richardscourtney says:
January 26, 2014 at 3:08 pm
(a) the solar cycle is causal of the synchronicity of the solar cycle and SLR
but
(b) the solar cycle is NOT causal of the SLR fluctuation.
Thanks for the note.
I am not certain about point a, but would agree with point b, and here is an example why
http://www.vukcevic.talktalk.net/ENSOa.htm
Korte’s geomagnetic dipole ( http://earthref.org/ERDA/973/) is interesting, but due to axis inclination and the wandering of magnetic poles it does not represent magnetic field along the axis of Earth’s rotation, which may or may not have any significance, but here it is anyway, spanning 7 millennia
http://www.vukcevic.talktalk.net/GeoPolarMF.htm
vukcevic says:
January 27, 2014 at 1:33 pm
Korte’s geomagnetic dipole ( http://earthref.org/ERDA/973/) is interesting, but due to axis inclination and the wandering of magnetic poles it does not represent magnetic field along the axis of Earth’s rotation, which may or may not have any significance
You are correct it has no significance as the field at the surface is not of interest in solar-terrestrial relationships. The solar wind sees a very different pole. And what you plotted is not along
the axis but only where the axis cuts the surface. Yet another example of misunderstanding of the data and of the physics.
vukcevic says:
January 27, 2014 at 1:33 pm
it does not represent magnetic field along the axis of Earth’s rotation, which may or may not have any significance
So what, as the magnetic field at the geographic poles are of no interest in solar-terrestrial relations. When will you learn?
vukcevik says:
January 27, 2014 at 1:33 pm
it does not represent magnetic field along the axis of Earth’s rotation, which may or may not have any significance
So what, as the magnetic field at the geographic poles are of no interest in solar-terrestrial relations. When will you learn?
[test of moderation criterion]
lsvalgaard says:
January 27, 2014 at 1:47 pm
So what, as the magnetic field at the geographic poles are of no interest in solar-terrestrial relations. When will you learn?
[test of moderation criterion].
I (vukcevic) strongly disagree !
Currently magnetic pole in the S. Hemisphere has moved away from land into the ocean. However there is at the South pole or its near vicinity, great deal of instrumentation recording all sorts of data, among which I presume must be a magnetograph somewhere, not to mention cosmic rays, geomagnetic storms etc…
Any scientific data from the South pole, past or present is of importance, and my graph in the above link, may be only place available to anyone anywhere to se changes of the magnetic field at the South pole spanning 7,000 years; can you point to another web link?
vukcevik says:
January 27, 2014 at 2:12 pm
I (vukcevic) strongly disagree !
Might be, but that is of no consequence. You see, neither the solar wind [generated currents in the ionosphere and induced currents at depth] nor the cosmic rays [nor, of course TSI, UV etc] take any notice of the surface field, but instead react to the dipole field.
vukcevik says:
January 27, 2014 at 2:12 pm
I (vukcevik) strongly disagree !
Might be, but that is of no consequence. You see, neither the solar wind [generated currents in the ionosphere and induced currents at depth] nor the cosmic rays [nor, of course TSI, UV etc] take any notice of the surface field, but instead react to the dipole field.
[Testing continues — ]
Yes, but also the geomagnetic pole is in vicinity of the South Pole, it has drifted only few degrees, unlike the magnetic pole which has moved into Southern Ocean
http://www.geomag.bgs.ac.uk/images/polesfig2.jpg
Please note: geomagnetic poles are not at the same location as the magnetic poles.
(it is easier to use copy and paste, says vukcevic! ), good night.
vukcevik says:
January 27, 2014 at 2:51 pm
Yes, but also the geomagnetic pole is in vicinity of the South Pole, it has drifted only few degrees, unlike the magnetic pole which has moved into Southern Ocean
http://www.geomag.bgs.ac.uk/images/polesfig2.jpg
Please note: geomagnetic poles are not at the same location as the magnetic poles.
The only poles that are important are for the main field eccentric dipole [sometimes called the Corrected Geomagnetic Pole http://omniweb.gsfc.nasa.gov/vitmo/cgm_vitmo.html ], all the other ones are invisible to cosmic rays and geomagnetic activity [both above and below ground]. How the other poles move around and where they are irrelevant. Try to learn something.
I can’t make a lot out of that comment. Didn’t you (usually) say, the secular trend (at least over the time scales we talk about) is rather small ?
Manfred says:
January 27, 2014 at 7:01 pm
“Who would care about a small second-order wiggle on top of the dominant long-term trend?”
I can’t make a lot out of that comment. Didn’t you (usually) say, the secular trend (at least over the time scales we talk about) is rather small ?
Yes, but that is only my [well-founded] opinion. There are many people who advocate a large changing ‘background’ [presumably because that would correlate better with the long-term climate change we observe — you know, recovering from the LIA and all that]. See http://www.leif.org/research/Long-term-Variation-Solar-Activity.pdf for some discussion on this.
And due to that rapid fall off and the small reach, the earth’s field cannot hold off cosmic rays above a certain energy threshold, while the solar field can (up to a higher threshold), leaving some space for Svensmark’s theory to exist..
But what about the polar regions ? Charged particles coming in from directions along the axis of the earth’s magnetic dipole appear to be totally unaffected. More, according to the artiists expression from your Link (page 9),
http://seismo.berkeley.edu/~rallen/eps122/lectures/L05.pdf
part of the solar wind appears to get channelled around the magnetic field right into these “polar open doors.”
As the Arctic appears to be increasingly seen as the key climate region (see J. Curry’s Stadium Wave or Steven Wilde or Jet Stream ), may this “matter” ?
Manfred says:
January 27, 2014 at 10:41 pm
And due to that rapid fall off and the small reach, the earth’s field cannot hold off cosmic rays above a certain energy threshold, while the solar field can (up to a higher threshold), leaving some space for Svensmark’s theory to exist..
The Earth’s field is much more efficient than the Sun’s in screening out cosmic rays. The main factor in cosmic ray modulation is not the Sun, but the Earth.
But what about the polar regions ? Charged particles coming in from directions along the axis of the earth’s magnetic dipole appear to be totally unaffected.
These particles are precipitated in the upper atmosphere 100 km up and above and have no effect on our weather or climate.
svalgaard says:
January 27, 2014 at 11:25 pm
“The Earth’s field is much more efficient than the Sun’s in screening out cosmic rays. The main factor in cosmic ray modulation is not the Sun, but the Earth.”
Could you expand on that? Is it because the Earth’s field is symmetrical (more or less) about the Earth, and so tends generally to deflect charged particles away, while the Sun’s field at the Earth is as likely to deflect towards the Earth as away?
Of course, the Sun’s field is much weaker at the Earth, but a bit stronger at the Sun. My first thought was that it could deflect particles coming from that direction, but then I thought, yes but, they would be deflected away from the Sun, not the Earth in general.
Bart says:
January 27, 2014 at 11:55 pm
“The Earth’s field is much more efficient than the Sun’s in screening out cosmic rays. The main factor in cosmic ray modulation is not the Sun, but the Earth.”
Could you expand on that?
The solar modulation is only a few percent. The Earth’s modulation is a factor of two. Perhaps the best illustration of that is this http://www.leif.org/research/CosmicRays-GeoDipole.jpg
The small wiggles are the solar modulation.
The physics is quite different for the two cases. It is not just simple ‘deflection’. For the Earth this is what is going on http://www.nmdb.eu/?q=node/172
For the Sun http://arxiv.org/abs/1306.4421
Bart (and anyone who could be interested)
I have superimposed GeoPolar magnetic field data and the by science ‘accepted’ dipole graph
http://www.vukcevic.talktalk.net/GeoPolarMF.htm
agreement between two appears to be reasonable (except around 2KY BC ).
I am happy to email the data file for GeoPolar MF (5000BC – 1950AD resolution in 10 year steps).
vukcevik says:
January 29, 2014 at 8:57 am
I have superimposed GeoPolar magnetic field data and the by science ‘accepted’ dipole graph … agreement between two appears to be reasonable
The field at any point of the Earth’s surface would be a good fit too. There is nothing special about the ‘GeoPolar’ magnetic field, except perhaps that that field is affected by the largest uncertainties [as the are no paleomagnetic measurements at all up there] and so is the least well known field of all.
lsvalgaard says:
January 29, 2014 at 9:11 am
The field at any point of the Earth’s surface would be a good fit too. There is nothing special about the ‘GeoPolar’ magnetic field,
Dr. S
Let’s data speak for it, here is what Korte data show for your little California town
http://www.vukcevic.talktalk.net/DrS-town.gif
On that bet you would have lost your shirt.
GeoPolar MF is possibly most accurate data for Earth’s dipole during last 7,000 years.
I could email you file if you wish.
vukcevic says:
January 29, 2014 at 9:39 am
Let’s data speak for it, here is what Korte data show for your little California town
You do not say what you plot. The total field strength would be best.
GeoPolar MF is possibly most accurate data for Earth’s dipole during last 7,000 years.
‘possibly’? There are no measurements up there. If you would try to get a measure of the dipole strength you should plot the field at the corrected geomagnetic latitude of 90 degrees, not the meaningless geographic pole.
Have you a file of numbers, which you think is more accurate?
“There are no measurements up there.”, Doesn’t matter much, see http://www.ngdc.noaa.gov/geomag/WMM/data/WMM2010/WMM2010_Report.pdf
Plotted are values from CALSK7K (I assume radial F) Br N & S absolute values.
Current 90 degrees geomagnetic latitude may not be a good representation due to poles movements. For the last 2Ky the movements were more or less cantered around 90N (see http://dourbes.meteo.be/aarch.net/linford.pdf page5 ).
I have a program that will scan gumf file for a set Lat/Long box for max values, but from Korte’s file one might not be able to work it out by scanning data, since GMF pole is not same as the max Br. I suppose by spherical triangulation of declination data from two locations would it be possible.
If you can suggest alternative numbers (possibly for both N & S ) I’ll have a go at it and compare to the accepted dipole reconstruction.
vukcevik says:
January 29, 2014 at 11:55 am
Current 90 degrees geomagnetic latitude may not be a good representation due to poles movements
No location on the surface is a good representation of the dipole movements. All locations are bad for this, although some might be worse than others. Your whole exercise is meaningless as the surface field is not what cosmic rays and the solar wind see.
9:11 am The field at any point of the Earth’s surface would be a good fit too.
11:55 am All locations are bad for this, although some might be worse than others.
Well, well, well!
A scientist has file of raw C14 or 10Be data going back 6-7 millennia. To work out the heliospheric mf he has to subtract the Earth MF dipole.
Where the a file for the Earth MF dipole data is to be found?
‘Sine wave’ like curve from your link
http://www.leif.org/research/CosmicRays-GeoDipole.jpg demonstrably is no good !
If there is a raw C14 or 10Be data going back 2 or more millennia ?
let’s do the science properly and compare
I’ve got the data, do you?
As Willis implies : No data, no contest !
vukcevic says:
January 29, 2014 at 2:11 pm
Jeez, Vuk, lose the attitude, it just makes you look arrogant.
There’s a file here of five different estimates of the changes in the Earth’s geomagnetic dipole based on a variety of proxies proxies. I found them by actually following up the links that Leif posted upthread … you might consider doing the same.
I’ve graphed them up:
Happy now?
w.
vukcevic says:
January 24, 2014 at 1:49 pm
Usoskin & Korte:
We conclude that changes of the regional tropospheric ionization at midlatitudes are defined by both geomagnetic changes and solar activity, and none of the two processes can be neglected.
———————————————–
Nope can’t forget the regional tropospheric ionization at midlatitudes. Another way for GCR to enter the system. Just give them a little push out their homestead..pulses and waves or changes in solar winds.
I wish this movie would go a little slower though. You easily miss the polar, cusp and equator action ..
The constant in motion radiation belts remind me of magnetopause simulations..and the constant motion that creates WAVES………………………………………………………………………………………………………
Solar Anomalous and Magnetospheric Particle Explorer (SAMPEX)
Published on May 18, 2013
Movie of the changing radiation belts as measured by SAMPEX/LICA from January 1, 1998 to March 1, 2005
Thanks Vuks and Dr. S., for links and discussion.
Those pesky multi linked videos over there.
Last video good but not the intended video..
vukcevik says:
January 29, 2014 at 2:11 pm
If there is a raw C14 or 10Be data going back 2 or more millennia ?
Going back 10 millennia. And easy to find
let’s do the science properly and compare
You can’t do science and knowledgeable people have already done that. One problem is that the data is not good enough to settle some of the questions, e.g. how much of the variation is due to the Sun, the earth’s magnetic field, and to climate. These questions [on which I am sort of an expert] are active areas of research, see e.g. http://www.leif.org/research/Svalgaard_ISSI_Proposal_Base.pdf and http://www.leif.org/research/Long-term-Variation-Solar-Activity.pdf
Carla says:
January 29, 2014 at 6:16 pm
Nope can’t forget the regional tropospheric ionization at midlatitudes. Another way for GCR to enter the system.
You [as does also Vuk] misunderstand the science. The tropospheric ionization has nothing to do with the Radiation belts or waves or any of what else you rave about..
Usoskin & Korte: We conclude that changes of the regional tropospheric ionization at midlatitudes are defined by both geomagnetic changes and solar activity
All they are saying is the well-known fact that the GCRs that ionize the troposphere are modulated both by the slow, long-term changes of the main magnetic field of the Earth and by the activity of the solar magnetic field in the [outer] heliosphere. Nothing new there.
@ Willis
I knew I’ll get myself into trouble. Me “arrogant”, the man who claims to have ‘corrected’ all known and yet unknown science, surely you’re joking …
Dr. S and I, or our ‘conversations’ go long way back, I accept his put downs, he accepts my ignorance, and even managed to improve on the spelling of my name. We are kind of friends and occasionally exchange emails. Unfortunately, if it was not for Dr. S you wouldn’t ever heard of me.
Thanks for the graph, I actually did look at the files:
Korte’s geomagnetic dipole ( http://earthref.org/ERDA/973/ ) is interesting
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1551530
and plotted data against an earlier Korte file, as shown here, but didn’t link to it!
http://www.vukcevic.talktalk.net/MF-6D.htm
Differences are of similar order, except that in my set of calculated data (Korte’s radial geomagnetic model) the data go further back.
What I was actually looking for is the raw numerical data filesfor C14.and or 10Be and data for dipole which were used before Korte’s compilation (2009) for eliminating Earth’s field.
It does help if one is more precise with language, but being vague it does help one dig himself out of a deep hole, and some are in it more often than the others.
@ Dr. Svalgaard
Thanks doc, for the links, have seen one or two before but I’ll check it out again.
vukcevic says:
January 30, 2014 at 1:28 am
What I was actually looking for is the raw numerical data files for C14.and or 10Be and data for dipole which were used before Korte’s compilation (2009) for eliminating Earth’s field.
The raw data comes from a variety of sources as detailed in the reports in PNAS http://www.leif.org/EOS/PNAS-2012-Steinhilber.pdf and http://www.leif.org/EOS/PNAS-2012-Steinhilber-Appendix.pdf
Figures S1 to S7 shows ‘semi-raw’ data [top panels] which you can digitize yourself. The data presented has already been corrected for the varying strength of the Earth’s dipole. You cannot duplicate that by simple correlation as it is necessary to apply a complicated physical model [as described in the papers] and actually compute the correction. Using the surface field is meaningless as that is not what the cosmic rays see. You must calculate from the spherical harmonic coefficients and a model of the magnetosphere what the magnetic field in space looks like and then calculate the orbits followed by the cosmic rays for all energies and integrate the modulation in order to correct the observed flux.
Some of the ‘raw’ data can be found here: http://www.ncdc.noaa.gov/data-access/paleoclimatology-data/datasets/climate-forcing
Real science is not just comparing time series. There is physics in between and that physics must be understood and applied.
vukcevik says:
January 30, 2014 at 1:28 am
even managed to improve on the spelling of my name.
Your name is on WUWT’s ‘black list’ that causes a comment to go into ‘moderation’. Misspelling it [as done in this comment] avoids that…
vukcevik says:
January 30, 2014 at 1:28 am
What I was actually looking for is the raw numerical data files for C14.and or 10Be and data for dipole which were used before Korte’s compilation (2009) for eliminating Earth’s field.
The raw data comes from a variety of sources as detailed in the reports in PNAS http://www.leif.org/EOS/PNAS-2012-Steinhilber.pdf and http://www.leif.org/EOS/PNAS-2012-Steinhilber-Appendix.pdf
Figures S1 to S7 shows ‘semi-raw’ data [top panels] which you can digitize yourself. The data presented has already been corrected for the varying strength of the Earth’s dipole. You cannot duplicate that by simple correlation as it is necessary to apply a complicated physical model [as described in the papers] and actually compute the correction. Using the surface field is meaningless as that is not what the cosmic rays see. You must calculate from the spherical harmonic coefficients and a model of the magnetosphere what the magnetic field in space looks like and then calculate the orbits followed by the cosmic rays for all energies and integrate the modulation in order to correct the observed flux.
Some of the ‘raw’ data can be found here: http://www.ncdc.noaa.gov/data-access/paleoclimatology-data/datasets/climate-forcing
Real science is not just comparing time series. There is physics in between and that physics must be understood and applied.
Thanks doc, If I wasn’t ‘block-head ignorant’, wouldn’t know about latest Steinhilber’s paper link.
In my inimitable manner, I like to interfere with the ‘settled science’ and found these two oddities
http://www.vukcevic.talktalk.net/Stein-Vuk.htm
I expect you might say: So what? but would appreciate more elaborate comment
I will look into some of the ‘paleoclimatology-data/datasets/climate-forcing’ data, and then I will be off to my winter bolt-hole, and write a bit more about http://www.vukcevic.talktalk.net/Ap-NHT.htm
p.s
you are welcome to use
any spelling you choose.
vukcevik says:
January 30, 2014 at 6:06 am
In my inimitable manner, I like to interfere with the ‘settled science’ and found these two oddities
Since it is meaningless to compare with the surface field the ‘so what’ reply seems appropriate.
I am sure you can do better than that, perhaps Steinhilber needs to go back to his data.
If he does so, Steinhilber et al would find that the ‘paleo’ TSI is as flat as pancake (except tiny 11 year mod) as certain Dr. Svalgaard of Stanford and of the WUWT, has been telling us for some time now; ahh… but some people will never learn, let alone do the science.
vukcevic says:
January 30, 2014 at 1:28 am
Now you’re not only being arrogant, you are accusing me without quoting my words, which is the action of a willful, spiteful, spoiled child who thinks he can just ignore polite requests, viz:
w.
Willis Eschenbach says:
January 30, 2014 at 11:33 am
…………
Jeez, Vuk, lose the attitude, it just makes you look arrogant.
I really didn’t object to it, I heard a lot worse from elsewhere.
Sorry, it appears to be a misunderstanding, my response was meant solely as a bit of a ‘self deprecating humour’ with no intention of malice, etc .
Nevertheless, I offer an apology.
All the best.
Willis,
Now I see where the misunderstanding comes from, after reading your comment for the third time.
I knew I’ll get myself into trouble. Me “arrogant”, [from?] the man who claims to have ‘corrected’ all known and yet unknown science, surely you’re joking …
I was describing my ‘scientific endowers’ not yours.
Word ‘from’ it was not and never meant to be there, it was ‘ me’ and not you.
Oh, never mind.
Have a good day.
vukcevic says:
January 30, 2014 at 12:03 pm
my response was meant solely as a bit of a ‘self deprecating humour’ with no intention of malice,
If you were sincere there would have been no need for quote marks around self deprecating humour. There is but a thin line between the humorous and the ridiculous and it is not clear in each case on which side of the line you are at that time. Better simply to stick to your continuing science education.
I often put quote marks for something that is a colloquial phrase, which -self deprecating humour- is . Well you can make of it anything you like, At first I didn’t read italics since I know what I written and what I meant, only at third reading I did realise [from?] was inserted. . If you read whole post again
http://wattsupwiththat.com/2014/01/21/sunspots-and-sea-level/#comment-1554317
you can see that I was talking about myself, and that is how it was meant. It is not my habit to comment on anyone personally, regardless what others may or may not say or do. I shall leave it at that.
vukcevik says:
January 30, 2014 at 1:28 am
Me “arrogant”, the man who claims to have ‘corrected’ all known and yet unknown science
I think that claim is unfounded and plain silly. I think you actually [in your heart] mean what you claim which is quite sad.
vukcevic says:
January 30, 2014 at 12:03 pm
My apologies as well if I misunderstood your words.
w
Willis Eschenbach says:
January 30, 2014 at 7:43 pm
……………
Your comment is appreciated, no need to apologise, just simple misunderstanding.
All the best
lsvalgaard says:
January 30, 2014 at 6:58 pm
…………
Hi doc
What you believe, that I believe, believe me, is unbelievable.
Omnia et nihil
vukcevic says:
January 31, 2014 at 12:44 am
What you believe, that I believe, believe me, is unbelievable.
I take that as a retraction of your claim that you “have ‘corrected’ all known and yet unknown science”.
To be honest, I get rather tired of vukcevic and his constant wild claims and linkage to his website. Maybe it’s time for the permanent troll bin if he doesn’t dial it back some.