Levy walks, solar flares, and warming

On April 13 (12.57.00) Nicola Scafetta writes:
“Rypdal and Rypdal have proven that when the temperature data are altered, in their case by adopting several detrending procedures, the properties we found in the data, which are hidden in the smooth component of the temperature, disappear.
I suspect that R&R’s methodology can be used to disprove any study.
It is easy: take a study, alter the data in such a way to eliminate the part where the interesting properties are hidden, prove that the altered data do not contain any more the original properties and, finally, conclude that the original paper must be wrong! Great logic indeed!”
and on April 18 (11.47.47) he continues:
“1) Detrending a Levy-Walk signal of its smooth component kills its Levy-Walk properties.
2) Levy-Walk signals may present a distribution of events that looks Gaussian.
Therefore R&R methodology is not appropriate for the task.”
These statements illustrate very clearly the fallacy of S&W. Let me explain why:
Nicola’s idea is that the the Levy-walk scaling properties they found in the data are “hidden in the smooth component”. He contends tat our detrending procedure for the GTA “eliminates the the part where the interesting properties are hidden”. He is wrong. The Levy-walk properties that S&W claim to find by SDA and DEA analysis on scales up to 4 years are NOT contained in the much slower (but strong) multidecadal trend (a 4’th polynomial fit) that we slow in Fig. 1 of our paper.
The demonstration of this is easy. Produce an ensemble of synthetic Levy walk noises and make a 4’th order polynomial fit to each of them. Each fit will show a much smaller trend that what we get from the GTA data. If we make large ensemble, and compute the mean square deviation of these fitted curves from the flat mean curve, we find that it is much smaller than the mean square deviation for the trend curve obtained from the GTA data.
Hence the probability that a signal with such a strong slow component as the one observed in the GTA data should be produced by an underlying Levy-walk mechanism is practically zero.
The clue is that you should not eliminate components on scales that are comparable to or smaller than the scales you use for computing your scaling. In the GTA case you should only remove components on scales larger than the 4 year analyzing window.
Below I will elaborate a bit on how we proceed to find the proper degree of smoothing. Bear over with me if you think I am getting over-pedagogical. I am trying to avoid the patronizing evasiveness that characterizes Nicola’s style of arguing.
First I would like to stress that the difference between a Levy walk (LW) and fractional Brownian motion (fBm) on one hand, and a Levy walk noise (LWN) and a fractional Gaussian noise (fGn) on the other. A LWN is the differentiated time series of a LW time series. And an fGn is the differentiated fBm. In the discussions here we have talked about LWs and fBms, but the actual GTA time series are not thought of as walks or motions (which are non-stationary stochastic processes), but as noises (which are stationary). In our paper we contend that the INTEGRATED GTA is an fBm with Hurst exponent H=0.65. What this means is that the actual GTA time series is an fGn. Nicola’s position is that the actual GTA is a LWN (because in their papers S&W perform SDA and DEA analysis on the integrated signal). Realizing that we are both modeling the GTA signal as a noise, it becomes more evident that the slow trend that we eliminate is not an inherent property of the model.
S&W’s method of analyzing scaling properties is by SDA and DEA analysis. SDA is nothing but the first order structure function (first statistical moment of differences), and a more systematic approach to scaling analysis is to compute large number of empirical moments for many values of q (see our paper for details) as estimates of the structure functions. However, this implies creating PDFs of differences between values of the integrated time series separated by a time lag Dt. This difference is the same as the sum of the values of the noise time-series in the time window Dt. But we will encounter a problem if the the mean of the noise time series over this window is not zero, because then this mean will accumulate linearly with time when we sum them, and hence at large Dt this mean value will dominate the scaling and hide the actual microscopic scaling properties. If we don’t eliminate this local mean value in the noise signal, the integrated signal will appear as a smooth curve on sufficiently long time scales, and the Hurst exponent will approach 1. Hence, by performing proper detrending we do exactly the opposite of what Nicola says, we discern properties that would otherwise be hidden.
There are many methods of detrending. What we do is a minimal one, and it goes as follows: We start with the lowest possible polynomial order (i.e we subtract a linear fit over the entire record length) and compute the empirical moments and the associated scaling function \zeta(q) (see our paper). If there is a string trend we then get Hurst exponent close to H=1. Then we increase the order of the polynomial until the result appears to converge. If the result has converged at a polynomial order where the polynomial still appears very smooth on the scales for which we perform the analysis, and if the resulting structure functions and the scaling function are straight curves in log-log plots, we have a selfsimilar process. If I also do the DEA analysis on the detrended data I know that I will get that D=H (the selfsimilarity exponent equals the Hurst exponent), and I conclude that the noise signal analyzed is NOT a LWN. If the PDFs in addition are Gaussian, I know that my signal is an fGn. This is exactly what we got for the GTA signal, and is why we have concluded that it is an fGn with H=0.65 and that there is no evidence in the data that it could be modeled as a LWN.

Kristoffer Rypdal (04:37:59) :
I thank Dr. Rypdal of his comments, but I do believe that he is not getting the point.
1) We have already proven in our subsequent papers that the smooth component Rypdal extracts with a polynomial fit has a solar modulation. Taking off trends in the temperature signature cannot but eliminate a fundamental solar component in the temperature and make weaker its presence in the temperature. We have also proved that a decadal and bidecadal temperature oscillation have solar origin. Our paper in 2003 is part of our early results but it is not even central to the theory that has been developed since 2003 as Rypdal has claimed in his phyorg interview. Even if some part of our study of 10-8 year ago should be disproved, the most recent studies of the last five years are not conditioned at all by those. Therefore, claiming, as Rypdal has stated, that his study disprove our studies is a great exaggeration.
Rypdal’s statement include:
“This means that if a cornerstone hypothesis is proven to be false, the entire theory fails. A corresponding theory of global warming of solar origin does not exist. ” Then he starts talking about “to shoot down every new missile” etc which is not really appropriate in a scientific debate.
2) In our pre 2003 studies where we talk about solar flares we are using the waiting time distribution of “large” flare as proxy of the solar dynamics, not their intensity as measured by SFI. We not only never claimed that solar flare energies are the origin of the climate increments, we have explicitly excluded it.
For example in we stated (page 8)
http://www.fel.duke.edu/~scafetta/pdf/PRE26303.pdf
“The solar flare intermittency is not the direct cause of the earth temperature fluctuations, since the radiation energy of the solar flares is relatively small.”
Thus, Rypdal’s discovery that the increments of the temperature record do not present the peaked Levy-Flight structure of the SFI does not disprove, but confirm our statements!
3) About the effect of the detrending procedure. Let us explain it with a simple example. Let us suppose to have to coupled systems. One is characterized by an intermittent nature, so we have spikes of vary amplitude that are separated by a given time interval. The other is sensitive to the spike frequency of the first one, not to the amplitude of the same. Now when the frequency of the first signal increases this causes a trend increase in the second signal. (For example think about the relation between the cycle frequency of a car engine and the velocity of the car). We study this kind of property and conclude that the two systems are linked.
Rypdal confuses the topics and claims that we are relating the spike amplitudes of the first signal to the increments of the second signal. To better isolate the increments of the second signal he has the great idea to eliminate the trending component of the second signal (the smooth component of the velocity of the car) that is what is related to the peak frequency function of the first signal (the frequency cycle of the engine) .
Finally he claims that he has disproved our result (that the frequency cycle of the engine is related to the velocity of the car) and he adds that his result has disproved all our results including those that are nor related at all with our 2000-2003 early studies!
This looks “propaganda” to me!
What we have proven in our studies is that climate system dynamics present a significant similitude with solar dynamics at multiple time scales. This similitude is present in the temporal patterns of the signal. This result will be reinforced by new studies! 🙂

Leif Svalgaard (07:44:04) :
in physics the term “proven” is always used in a “weak” term.
There are several other patterns beside the 11-year cycle that are “proven” to correspond between the solar indexes and the temperature: for example during the solar Maunder minimum is was quite cold! And the same happened during the other little ice ages and so on.
The mechanisms are still “unproven”, not the existence of those correlations, which are “proven”.
About the ‘Science Nugget’. We discuss about the different interpretations of the solar flare intermittency, Levy or Poisson, and concluded that the data we analyzed present a clear levy structure. Moreover, inverse power law distributions can also be obtained with incremental Poisson distributions.
All figures shown in your web-site indicate a Levy-structure. note that the graphs are all in a log-log plot

Nicola Scafetta (08:42:38) :
whether a detrending procedure can keep intact the Levy-walk properties of complex Levy-walk signals such as those where the increments are noised. This is pure mathematics and the detrending procedure will destroy such memory definitely.
It is difficult to do “pure mathematics” if you don’t specify the model. Please answer my question: Is the Levy walk a valid stochastic description of the GTA? If so, specify the construction and the parameters.

Interesting stuff.

Nicola Scafetta (16:02:38) :
The model is specified in the papers we wrote in 2003. Your way to do calculation should be tested first. Which is what you did not do.
I have done it now, and our method is fine!
Nicola Scafetta (16:02:38) :
Moreover we are not talking about the increments of the climate system, we explicitly excluded that such “increments” have a levy-statistics.
 This was my position in 2003.
I am not sure I know what you are talking about here, but nobody is saying the increments of the GTA have heavy tailed distributions.
You are still avoiding my question. I repeat: Should we understand the model specified in the 2003 papers as a “toy” or as a realistic description of the GTA? You are still not clear on this point.

In his later messages Nicola Scafetta has begun to admit that there may be flaws in his 2003 papers, but he does not think these papers are important in the light of his later work. We have shown that the strong slow component in the GTA signal cannot be a consequence of Levy-walk statistics in the model realization of a Levy walk using the prescription given in the 2003 papers (where DEA and SDA analysis is applied the frequency signal of temporally Levy distributed unit pulses). He now stresses that the slow component cannot be removed because his later work has “proven” that this slow component is (partly) of solar origin. But this is completely irrelevant for the issue we are discussing. In our paper we explicitly avoid discussion of the physical mechanism of the slow component, because the issue in the S&W 2003 paper is the assertion that the LW statistics and its characteristic exponent can be detected in the GTA signal. WE have shown that if you want to discern this kind of statistics in a truthful way you MUST eliminate any strong slow component, whatever the physical origin of that component may be.
Our criticism of the S&W 2003 paper concerns the scientific methodology, not the larger issue of the solar origin of climate change. Nicola’s tendency to divert the focus to other issues when he is pressed on methodology is known from other debates. We will be happy to discuss his later work in another setting, but not here.
The flawed logic in the S&W paper is quite simple to understand, and can be summarized as follows:
They contend that the waiting-time statistics satisfies a power-law with a specific value of characteristic exponent describing the weight of the power-law tail of the waiting-time distribution. This is the essential ingredient in a Levy walk (LW). We think that this assertion is true, but only up to time-scales of about 3 months.
The controversial point, however, is their assertion that this LW- statistics is detectable in the GTA signal. The only evidence they present for this assertion is the result of so-called SDA and DEA analysis of the GTA signal, which yields the value of two exponents.
Our main objection is that the values they find for these exponents do not represent evidence for underlying LW-statistics, only that the undetrended signal is strongly persistent and not selfsimilar. There is an infinity of mechanisms other than Levy walk noises that will generate the same result. In our paper we show one such mechanism, namely a weakly persistent fractional Gaussian noise with H=0.65 superposed on the slow, but strong, multidecadal component found in the GTA signal.
S&W 2003 make the fundamental logical error: A implies B, observe B is true, then A is true.
I will conclude this discussion from my side (after all, we have job to look after) with a quote from a well-known satirical play by the Norwegian-Danish writer Ludvig Holberg, (1684-1754), called Erasmus Montanus. What is written below are not my words, but taken from the web-site
http://www.twinkle.ws/docs/erasmus-montanus.html
In this satire, a farmer’s son, Rasmus Berg (or Erasmus Montanus in Latin), goes to the big city to study, and become “an academic” in lack of a better term. Upon returning to his home in the country, he displays arrogance against his peers, now a know-it-all (borderline snob), hence causing quite a stir in the village. For instance, he deploys invalid rhetorical logic to prove that his mother is a stone, and in a discussion with the church bell ringer, Per, he sets out to prove that Per is a cock:
“A cock crows; so do you. A cock is proud of its crow; so are you, I can clearly hear that. A cock crows when it is time to get up. You ring the bell, when the folks go to church.
Ergo, you are a cock!”

Leif Svalgaard (05:01:11) :
Nonsense. Then Solanki’s is wrong too. What about the situation that the radionuclide used is a ten-year average it self, e.g. derived from a layer that represent 10-years of ice [or three rings]?
What is nonsense is trying to establish a reliable TSI figure from a proxy record. Solanki and Steinhilber have provided a detail solar record over the Holocene that both more or less agree, the records are an indication of overall solar activity and not an accurate TSI reconstruction. The very interesting aspect is that both sets of records do not show a flat solar floor.
I am with Nicola on this one, the weight of evidence is not on your side.

Kristoffer Rypdal (03:09:55) :
“S&W 2003 make the fundamental logical error: A implies B, observe B is true, then A is true. A implies B, observe B is true, then A is true. ”
In science the logic is: A implies B, observe B is true, then A is possible.
You methodology has removed a known solar signature on climate which is associated to the Gleissberg (50-80 years) and Suess (160-260
years) solar variability. Moreover, if the record were twice as long what kind of fit would you use? A 4th, 5th or 6th order polynomial?
Levy-walk noises may present very long trending properties because of their fat temporal tail distributions.
In any case, the real problem with your work is that you mistake the increments with the time structure. We talk about waiting time distribution, you are talking about increment distributions. The scaling is in the time structure, not in the increments.
You cannot disprove anything by mistaking the correct observable and the correct interpretation. The logic in your work is the following:
‘A’ does not have anything to do with ‘B’, ‘B’ is false therefore ‘A’ too must be false!

Leif Svalgaard (17:37:45) :
in science nothing is proven. I thought that you knew that. 🙂
Only in mathematics the theorems can be proven! Don’t you know it?
The logic used in climate science and in several observational sciences is that: A implies B, observe B is true, then A is possible.
For example:The big-bang theory predicts the 3K background temperature of the universe; the 3K background temperature is observed; therefore the big-bang theory is plausible.
“Gleissberg (50-80 years) and Suess (160-260
years) solar variability. These may be solar signals”
really? You surprise me! So, after all the sun is not constant (plus a 11-year cycle)!
Just a curiosity. During the Maunder minimum no sunspots were seen. Do you believe the TSI was completely and perfectly constant during that period?
I propose to call such a value the Leif’s constant 🙂