Guest Post by Willis Eschenbach
I came across a curious graph and claim today in a peer-reviewed scientific paper. Here’s the graph relating sunspots and the change in sea level:
And here is the claim about the graph:
Sea level change and solar activity
A stronger effect related to solar cycles is seen in Fig. 2, where the yearly averaged sunspot numbers are plotted together with the yearly change in coastal sea level (Holgate, 2007). The sea level rates are calculated from nine distributed tidal gauges with long records, which were compared with a larger set of data from 177 stations available in the last part of the century. In most of the century the sea level varied in phase with the solar activity, with the Sun leading the ocean, but in the beginning of the century they were in opposite phases, and during SC17 and 19 the sea level increased before the solar activity.
Let me see if I have this straight. At the start of the record, sunspots and sea level moved in opposite directions. Then for most of the time they were in phase. In both those cases, sunspots were leading sea level, suggesting the possibility that sunspots might affect sea level … except in opposite directions at different times. And in addition, in about 20% of the data, the sea level moved first, followed by the sunspots, suggesting the possibility that at times, the sea level might affect the number of sunspots …
Now, when I see a claim like that, after I get done laughing, I look around for some numerical measure of how similar the two series actually are. This is usually the “R2” (R squared) value, which varies from zero (no relationship) to 1 (they always move proportionately). Accompanying this R2 measure there is usually a “p-value”. The p-value measures how likely it is that we’re just seeing random variations. In other words, the p-value is the odds that the outcome has occurred by chance. A p-value of 0.05, for example, means that the odds are one in twenty that it’s a random occurrence.
So … what did the author of the paper put forwards as the R2 and p-value for this relationship?
Sad to relate, that part of the analysis seems to have slipped his mind. He doesn’t give us any guess as to how correlated the two series are, or whether we’re just looking at a random relationship.
So I thought, well, I’ll just get his data and measure the relationship myself. However, despite the journal’s policy requiring public archiving of the data necessary for replication, as is too common these days there was no public data, no code, and not even a Supplementary Online Information.
However, years of messing around with recalcitrant climate scientists has shown me that digitizing data is both fast and easy, so I simply digitized the graph of the data so I could analyze it. It’s quite accurate when done carefully.
And what did I find? Well, the R2 between sunspots and sea level is a mere 0.13, very little relationship. And even worse, the p-value of the relationship is 0.08 … sorry, no cigar. There is no statistically significant relationship between the two. In part this is because both datasets are so highly auto-correlated (~0.8 for both), and in part it’s because … well, it’s because as near as we can tell, sunspots [or whatever sunspots are a proxy for] don’t affect the sea level.
My conclusions from this, in no particular order, are:
• If this is the author’s “stronger effect related to solar cycles”, I’m not gonna worry about his weaker effect.
• This is not science in any sense of the word. There is no data. There is no code. There is no mathematical analysis of any kind, just bald assertions of a “stronger” relationship.
• Seems to me the idea that sunspots rule sea level would be pretty much scuttled by sunspot cycles 17 and 19 where the sea level moves first and sunspots follow … as well as by the phase reversal in the early data. At a minimum, you’d have to explain those large anomalies to make the case for a relationship. However, the author makes no effort to do so.
• The reviewers, as is far too often the case these days, were asleep at the switch. This study needs serious revision and buttressing to meet even the most minimal scientific standards.
• The editor bears responsibility as well, because the study is not replicable without the data as used, and the editor has not required the author to archive the data.
So … why am I bothering with a case of pseudo-science that is so easy to refute?
Because it is one of the papers in the Special Issue of the Copernicus journal, Pattern Recognition in Physics … and by no means the worst of the lot. There has been much disturbance in the farce lately regarding the journal being shut down, with many people saying that it was closed for political reasons. And perhaps that is the case.
However, if I ran Copernicus, I would have shut the journal down myself, but not for political reasons. I’d have closed it as soon as possible, for both scientific and business reasons.
I’d have shut it for scientific reasons because as we see in this example, peer-review was absent, the editorial actions were laughable, the authors reviewed each others papers, and the result was lots of handwaving and very little science.
And I’d have shut it for business reasons because Copernicus, as a publisher of scientific journals, cannot afford to become known as a place where reviewers don’t review and editors don’t edit. It would make them the laughing stock of the journal world, and being the butt of that kind of joke is something that no journal publisher can survive.
To me, it’s a huge tragedy, for two reasons. One is that I and other skeptical researchers get tarred with the same brush. The media commentary never says “a bunch of fringe pseudo-scientists” brought the journal down. No, it’s “climate skeptics” who get the blame, with no distinctions made despite the fact that we’ve falsified some of the claims of the Special Issue authors here on WUWT.
The other reason it’s a tragedy is that they were offered an unparalleled opportunity, the control of special issue of a reputable journal. I would give much to have the chance that they had. And they simply threw that away with nepotistic reviewing, inept editorship, wildly overblown claims, and a wholesale lack of science.
It’s a tragedy because you can be sure that if I, or many other skeptical researchers, got the chance to shape such a special issue, we wouldn’t give the publisher any reason to be unhappy with the quality of the peer-review, the strength of the editorship, or the scientific quality of the papers. The Copernicus folks might not like the conclusions, but they would be well researched, cited, and supported, with all data and code made public.
Ah, well … sic transit gloria monday, it’s already tuesday, and the struggle continues …
w.
PS—Based on … well, I’m not exactly sure what he’s basing it on, but the author says in the abstract:
The recent global warming may be interpreted as a rising branch of a millennium cycle, identified in ice cores and sediments and also recorded in history. This cycle peaks in the second half of this century, and then a 500 yr cooling trend will start.
Glad that’s settled. I was concerned about the next half millennium … you see what I mean about the absence of science in the Special Edition.
PPS—The usual request. I can defend my own words. I can’t defend your interpretation of my words. If you disagree with something I or anyone has written, please quote the exact words that you object to, and then tell us your objections. It prevents a host of misunderstandings, and it makes it clear just what you think is wrong, and why.
Nicola Scafetta says:
January 22, 2014 at 7:43 am
[snip – you are welcome to resumbit without the ad homs – mod]
Yes Nicola, behave yourself on his Nibs thread. Here’s an example of the sort of thing you can’t say:
“Copernicus, as a publisher of scientific journals, cannot afford to become known as a place where reviewers don’t review and editors don’t edit”
REPLY: That’s not ad ad hom, it isn’t directed at a specific person. You really don’t have much integrity to stand on here Mr. Tattersall, since you don’t allow Willis to comment at your own blog. And then there’s that other incident where you violated my trust. – Anthony
Anthony, by the way,
have you read Giovanni de Gregori letter?
http://tallbloke.wordpress.com/2014/01/22/giovanni-p-gregori-scientific-freedom-paradigms-and-dogma/
“A p-value of 0.05, for example, means that the odds are one in twenty that it’s a random occurrence.”
no, that is the “p-value fallacy”, the p-value is the probability of observing a result as extreme as that observed IF it ocurred due to random chance. This is not the same thing as the probability that it ocurred DUE to random chance.
REPLY: That’s not ad ad hom, it isn’t directed at a specific person. You really don’t have much integrity to stand on her Mr. Tattersall, since you don’t allow Willis to comment at your own blog. And then there’s that other incident where you violated my trust. – Anthony
That violation of trust was over Willis’ ad hom attacks too, remember?
Here’s Niklas Morner’s statement on the way we conducted peer review:
(2) “the editors selected the referees on a nepotistic basis”
Nepotism is to favor friends and relatives without respects to qualifications. We did the opposite; the reviewer chosen were all specialists on the topics in question.
It is true that they primarily were chosen among the authors of the special issue with some additional from outside. This does not mean “pal-reviewing”, but serious colleague reviewing. Most members of the author-team only new each other superficially or as authors. It is common practice when printing proceedings or collective volumes to seek the reviewers within the group, not in order to make the reviewing process less serious, but because those persons are the true experts within the field.
And almost always they do a tremendously good job to improve the papers in constructive ways. So also in our case: our reviewing was simply excellent, which I am sure all persons involved would happily testify. This includes strong points and forces for relevant changes and updating.
And what we achieved was a wonderful collection of papers that together make a very strong impact of elevating an old hypothesis into a firm theory saying that the solar variability is, indeed, driven by the planetary beat.
Amen to that Niklas, you did 100 hours a week for several months to make this happen, and I’m with you all the way. The 580+ peer reviewed papers you have to your name in the geophysics and oceanology fields enabled you to get some superb outside reviewers such as the very eminent Physicist Prof. Giovanni P. Gregori, who has written a brilliant letter to Martin Rasmussen. Anthony should read it.
http://tallbloke.wordpress.com/2014/01/22/giovanni-p-gregori-scientific-freedom-paradigms-and-dogma/
REPLY: So because you hate what Willis has to say, along with the maths to back it up (you’ve presented none), it was OK to violate my trust? Interesting. – Anthony
Dear Anthony,
a famous physicist wrote me after having read your recent posts. He said this about you
******
As for Anthony Watts, this reminds me of the old advice: when you circle the wagons to fend off an attack by wild Indians, direct your fire outward, not inward.
******
Don’t you think that would be better for you to close this post by Willis (it has been fully disproved by me above), and open a new post where you reproduce Giovanni de Gregori letter?
You may also make two posts by coping and past my articles on notrikzone:
(1) http://notrickszone.com/2014/01/21/lynch-mob-science-scafetta-accuses-publisher-of-skipping-trial-conducting-academic-lynching/
(2) http://notrickszone.com/2014/01/19/scientists-react-sharply-to-copernicus-publishing-censorship-of-alternative-scientific-explanations-do-you-realize-what-you-have-done/
Should add, it is better to perform a statistical test and not describe it completely accurately than not to perform the test at all! It is a common error even made by scientists on a regular basis.
Stephen Wilde says:
January 22, 2014 at 1:34 am
The Journal is about pattern recognition.
A pattern is being discussed in the paper.
=========================================
That is not what is meant by Pattern Recognition in science, to get a good idea of what it ought to be about, see the content of the journal “Pattern Recognition” http://www.journals.elsevier.com/pattern-recognition/
REPLY: So the Copernicus “Pattern Recognition in Physics” was nothing more than a ripoff of the Elsevier journal, right down to the orange cover?
– Anthony
Anthony, I think that may be another spurious correlation! ;o)
It is a pity really as PRIP was a really good idea for a journal, there are lots of good applications for pattern recognition in phsyics and the sciences, if it was the sort of pattern recognition methods published in Patter Recognition (which is a very good journal).
@ur momisugly Dikran Like you, I think the idea of such a journal was a good one, though the titles and covers are a bit too close for copyright comfort. Unfortunately, the people that ran the special edition bollixed the opportunity handed to them in the most self destructive way possible. It was a classic own-goal. As I pointed out previously Copernicus made equally bad errors. They should have spotted and fixed the problem before it exploded. – Anthony
Siberian_Husky says:
January 22, 2014 at 2:29 am
Ummm … I think “autocorrelated” means correlated to a lagged version of the data itself, but it appears you don’t have a clue what it means. Medice, cura te ipsum.
w.
Chris Wright says:
January 22, 2014 at 4:03 am
Dear God, would all of you folks making this or a similar claim please do the freakin’ math!
The human eyeball is famous for finding correlation where there is none at all. We invented statistics in part to get measurements of correlation, and you’re all “No, don’t bother me with all that nasty number stuff, I’m just going to squint at the graph and make my decision”.
DO YOUR HOMEWORK, DO THE MATH!
w.
Yes, I agree with much of that. The cover is probably just a coincidence and I don’t think the name is to big a deal either, it is quite common to see a journal titled X and then another International Journal of X and then a Journal of Applied X etc. It is difficult to find a good name for a journal after a while. Journals that have the most obvious short name for the topic are ususally good ones as it is an indication that they got there first and have survived for a while!
Also these days we mostly access the journals electronically and rarely see the cover, I’d forgotten PRs was even orange!
E.M.Smith says:
January 22, 2014 at 4:59 am
…..
Hi E.M
Thanks. The graph is based on the interaction of two magnetic fields. The Earth’s field apparently is not immune to tidal forces acting on the liquid core where it is generated. Andy Jackson, who is a world recognised authority on the mater, has publicly acknowledged (in a speech to some august science organisation) that they have found frequencies that are coincidental with certain E-M orbital numbers.
There is nothing gained, other than disappointment, concerning this affair. An extraordinary opportunity has been fumbled. Hindsight and finger pointing after the fact… brings no relief.
Personally, I always try to fail safe: If anything must be hidden, or secret… can that thing be good? The answer to that question, has kept me mostly out of trouble. GK
Anthony
Thats not the first time Rog has violated trust. Recall gavin’s letter and Lisbon.
Wilde
‘This paper does not purport to ‘prove’ anything.”
then it’s not even wrong.
Novels and poems dont purport to prove anything either.
Agust Bjarnason says:
January 22, 2014 at 5:11 am
The authors (not NASA but the authors) find cycles in the Nile high- and low-water records at NEAR 88 and 200 years. Here is their description of the significance of their finding:
Since the Nile is where Hurst-Kolmogorov dependence was discovered, and the Nile is justly famous for being the home of “strongly correlated fractional noise”, and their results are only significant at the one-sigma level when tested against that noise, this means that none of their results are statistically significant as the term is used in climate science.
In climate science, results need to be significant at the two-sigma level. It’s a weak standard, it means one result in twenty will come up by chance, it means that if you look at a dozen datasets you have a 50/50 chance of getting a “significant” result by chance, but two sigma is the standard … and their results are only significant at one sigma, which means not significant in any sense.
In addition, take a look at their Figure 3, right panel. The “peak around 260 years in the low water level” they discuss actually peaks at 303 years, and the “near 88 year periodicity in both records” they discuss at length actually peaks at 102 years in the low-water record. Since when is 102 years a “near 88 year periodicity”??
I always get nervous when I see this kind of fudging, claiming “near” for anything that vaguely resembles some astronomical cycle or other … overall, I’d give the paper low marks.
Definitely an interesting read, though, thanks.
w.
It is interesting that the two cycles appear to have similar frequencies…but a simple relation is agreed to not fit due to phase shifts. Think of an electrical analog — You can have a sinusoidal voltage source driving current in a resistor, and the measured current would be in phase. You could parallel that resistor with a capacitor, and as you varied the value of the capacitor, you would phase shift the current w.r.t. the drive. Pretty simple differential equations. Clearly, if there is a correlation & causation by solar forcing, than a complete model has to look into what other effects might be varying the energy storage. PDO? Something else? This is an intriguing pattern, but there is a lot to figure out here.
vukcevic says:
January 22, 2014 at 9:52 am
Thanks. The graph is based on the interaction of two magnetic fields.
No, just poorly done curve-fitting.
Nicola Scafetta says:
January 22, 2014 at 8:04 am
Thanks, Nicola, it’s always interesting to hear from you.
However, I just did a quick search of both the first WUWT post and the Shaviv paper that you reference … neither one of them contains the graph under discussion.
But its not just that … neither one of them even mentions sunspots once.
Something similar to the graph appears in the Archibald post you also reference in your comment. However, that post also says nothing about sunspots. And rather than tracing its derivation to the Shaviv paper (which doesn’t mention sunspots), Archibald says:
To sum up, none of the three references you give show the graph I discuss above or even mentions sunspots … and the one of the three above that is similar, the Archibald post, has no real provenance for the data (it appears to be smoothed sunspot data … but which sunspot data and what smoothing?)
Sadly, like the author under discussion, Archibald doesn’t provide us with either the R^2 value or the p-value for the claimed correlation. I can tell you right now, the p-value will be worse than I show above, because of the smoothing that has been applied to the sunspot data.
Short version? Your claim, that the graph in the head post is from the Shaviv document, is contradicted by the facts. Shaviv doesn’t mention sunspots.
Nor does Shaviv matter, because nothing in the provenance of the graph can overcome the lack of statistical significance of the claimed correlation in this graph.
Nor can all of your bluster rectify the fact that no peer-reviewer asked for that most elementary of calculations, the R^2 and the p-value.
Regards,
w.
PS—Your claim that the Shaviv paper was “discussed with Anthony’s great approval here” is an egregious misrepresentation. Of the entire post, Anthony only wrote the introductory paragraph, which says:
Since that was Anthony’s entire contribution, and he made no further comment on the paper in the comments section, this claim of yours about Anthony’s “great approval” is as false as your other claims … and you accuse me of not reading the paper under discussion?
Thanks Willis.
Doug Proctor says:
January 22, 2014 at 8:13 am
I don’t understand this. There is no trend in the ∆ sea level data, nor in the sunspot data. There is a claimed relationship where sometimes one leads the other, sometimes it follows the other, sometimes they move in parallel, and sometimes they move in opposition … perhaps that impresses some people as a prediction method. Me, not so much.
True. And pulling a random folk aphorism out of your fundamental orifice doesn’t make it a scientific objection to what I’ve written.
w.
I am not interested in reading the paper, however one can see from just looking at the graph that there is great correlation and that makes it fascinating. Sure it could just be a coincidence (which I assume is what you are essentially saying it is), but that seems extremely unlikely. There is ‘probably’ some reason for the close correlation.
It’s a similar situation with Milankovitch cycles. There is a great correlation. So great that people assume there is ‘something there’, they just aren’t sure exactly how the small-in-theory perturbation of the cycles translates into a huge difference in climate.
I won’t dispute your criticism of paper or journal, but I would like to point out your consistent closed-mindedness. There is an obvious correlation. Instead of being curious about why that is, you just choose to dismiss it, never to be thought about again as it doesn’t fit within your existing mental model. That’s poor science. Be curious! Being a criticizing curmudgeon is easy … anyone can do that.
Thank you. Another good presentation.
I would not automatically disparage a model fit with an R^2 = 0.15 and p = 0.08. Granted, it is not very strong evidence against a null hypothesis, so I wouldn’t believe the result either, but for a field in which nearly all relationships have small p values it is probably reportable (unless it has been selected out of a large number of results, in which case the observed significance level is really greater than the reported p value.) On the other hand, it is more probable than not that the reported association is closer to R^2 = 0.15 than R^2 = 0.
What’s curious, as you noted, is how quickly the curves changed from out of phase to in phase.
tallbloke says:
January 22, 2014 at 8:56 am
Thanks, Roger. I fear that peer review that does not require the mathematical calculation and reporting of the significance and R^2 value of a claimed relationship is not “serious colleague reviewing”, it’s a joke. The same is true of not requiring the “out of sample” testing of the Salvador model, plus not requiring at least some mention of the effect of using 20 tuned parameters in said model that Dyson warned about …
Look, you and Nicola and Gregori can puff and blow all you want about the high quality of the peer review, but the facts don’t bear you out. The peer review was horrendous, and you didn’t follow the requirements for reviewers or for the editors as well.
Show me how I can replicate Scafetta’s work without access to his data and code, for example … it’s written down as part of the Editors job to make sure that can happen. We’ve been fighting with Michael Mann and his ilk for years, fighting for the normal scientific transparency, fighting with Science magazine to enforce their own policies regarding data and code archiving … and you fools come along and you publish without including links to your data and code? What on earth were you thinking, Roger, that no one would care? That we wouldn’t read the papers critically?
Pathetic.
You guys signed up to play in the peer-reviewed swimming pool. Then you not only ignored the posted rules that you had agreed to follow, you smashed them to bits. Now you want to complain because the lifeguard threw you out of the pool? … color me unimpressed.
w.
Matthew R Marler says:
January 22, 2014 at 11:15 am
Thanks, Matthew. I’m skeptical because I’ve seen this same thing over and over in a wide variety of natural datasets. You find some relationship, it even might be strong … then it fades out, and some other relationship comes to the fore. Or it stays there, but suddenly changes phase. Or it’s there, but sometimes one leads the other, and sometimes one trails the other.
Causal relationships don’t do any of those things. When you walk out into the sun, you get warm. It doesn’t fade away and get replaced by warming from the stars. Nor does it change phase, so that next time you walk out into the sun it makes you cold. Nor does one sometimes lag and sometimes lead the other—I don’t get warm a few seconds before walking outside in some cases, and a few seconds after walking outside in other cases.
As I’ve said elsewhere, I spent a lot of time looking at, examining, and doing curve fitting on a whole host of natural datasets. The more I’ve done, the more skeptical I’ve become of this kind of claimed causal correlation, the kind that reverses phase and sometimes leads and sometimes follows and sometimes plays the Cheshire Cat …
w.