Guest Post by Willis Eschenbach
Once again, Dr. Curry’s “Week in Review-Science and Technology” doesn’t disappoint. I find the following:
Evidence of a decadal solar signal in the Amazon River: 1903 to 2013 [link] by Antico and Torres
So I go to the link, and I find the abstract:
It has been shown that tropical climates can be notably influenced by the decadal solar cycle; however, the relationship between this solar forcing and the tropical Amazon River has been overlooked in previous research. In this study, we reveal evidence of such a link by analyzing a 1903-2013 record of Amazon discharge. We identify a decadal flow cycle that is anticorrelated with the solar activity measured by the decadal sunspot cycle. This relationship persists through time and appears to result from a solar influence on the tropical Atlantic Ocean. The amplitude of the decadal solar signal in flow is apparently modulated by the interdecadal North Atlantic variability. Because Amazonia is an important element of the planetary water cycle, our findings have implications for studies on global change.
The study is paywalled, but to their credit they’ve archived the data here as an Excel workbook. Let me start where I usually start, by looking at all of the raw data, warts and all.
Figure 1. Monthly average Amazon river flow (thousands of cubic metres per second). The violet colored sections are not observations. Instead, they are estimations based on the river levels in two locations on the Amazon.
Now to me, that’s a big problem right there. One violet section is based on river levels at one location, and the other violet section is based on river levels from another location. It’s clear from the annual average (red/black line) that the variance of those two river level datasets are very different. One river level dataset has big swings, the other has small swings … not good. So first I’d say that any results from such a spliced dataset need to be taken, as the old Romans said, “cum grano salis” …
Setting that question of spliced data aside, I next looked at the periodogram of the data. This shows the strength of the signal at various periods. If the ~11-year solar cycle is affecting the river flow, it will show a peak in the 11-year range.
It appears at first blush as if there is a very small 11-year signal in the full data (black), about 6% of the total range of the overall data swing. But when we split the data into the first half and the last half (red and blue), the 11-year signal disappears. This is not at all uncommon in observational datasets. Apparent cycles are often just the result of the analysis method averaging a changing signal.
Next, in Antico2015, the authors use the annual average data. To me, this is a poor choice. If you wish to remove the annual fluctuations, that’s fine … but using annual average data cuts your number of data points by a factor of 12. And this can lead to spurious results by inflating the apparent significance. But let us set that aside as well.
Finally, there is no statistically significant correlation between sunspots and Amazon river flow levels at any lag (max. monthly correlation ~ 0.1, p-value = 0.3 …).
Having seen that, my next step was to see how the authors of Antico2015 decided that there was a solar signal in the Amazon. And this was a most fascinating voyage. The best thing about climate science is that there is no end of the opportunities to learn. In this case, I learned from the Supplemental Online Information that they were using a method I’d never heard of, ensemble empirical mode decomposition, or EEMD. It’s one of many methods for decomposing a signal into the sum of other signals. Fourier analysis is the best known type of signal decomposition, and I’ve written before about the “periodicity” decomposition of Sethares, but there are other methods..
The details of EEMD are laid out by its developers in a paper called “Ensemble Empirical Mode Decomposition: A Noise Assisted Data Analysis Method” (hereinafter “EEMD2005”) … how could a data junkie like myself not like something called “noise assisted data analysis”?
Figure 3. Graphic explaining the EEMD process, from the EEMD2005 paper. ORIGINAL CAPTION: The very first sifting process. Panel a is the input; panel b identifies local maxima (red dots); panel c plots the upper envelope (red) and low envelope (blue) and their mean (black); and panel d is the difference between the input and the mean of the envelopes.
Then after you identify local maxima (panel b) and local minima (not shown), you draw two splines, one through the local maxima and the other through the local minima of the dataset (red and blue lines, panel c). The first component C1 is the difference between the data and the local mean of the two splines (panel d).
Then you take the resulting empirical mode C1 as your dataset and do the same—you draw two splines, one through the local maxima and the other through the local minima of C1. The second component C2 is again the difference between the data and the local mean of those two splines.
Repeat that until you have a straight line.
How do you aid that with noise? Well, you repeat it a couple thousand times using the original data plus white noise, and you average the results. According to the paper, this acts as a bank of bandpass filters, and prevents the mixing of very different frequencies in any one component of the decomposition. What do I know, I was born yesterday … read the paper for the math and the full explanation.
In any case, when they use EEMD to decompose the Amazon flow data, here’s what they get. Each panel shows the resulting curve from each step in the decomposition.
Figure 4. This shows Figure S1 from the Supplementary Online Information of the Antico paper. ORIGINAL CAPTION: (Left) Annual mean (October-September) Amazon flow record at Obidos station, its oscillatory EEMD modes (C1-6), and its residual trend. (Right) Raw periodograms of flow modes. In these power spectra, the frequency band of the decadal sunspot cycle, at 1/13 to 1/9 cycles per year, is depicted by the shaded region, and the oscillatory period of the most prominent spectral peak of C3 is given in years. In the left panels, the fraction of total variance accounted by each mode is shown in parentheses. For a particular mode, this fraction is the square of the Pearson correlation coefficient between the mode and the raw data record. The sum of these fractions may be greater than 100% because EEMD is a nonlinear decomposition of data; therefore, the EEMD modes are not necessarily linearly independent. To obtain the EEMD decomposition of the annual mean flow record, we considered an ensemble number of 2000, a noise amplitude of 0.6 standard deviations of the original signal, and 50 sifting iterations.
This is a curious kind of decomposition. Because of the use of the white noise, each panel in the left column shows a curve that contains a group of adjacent frequencies, as shown in the right column. No panel shows a pure single-frequency curve, and there is significant overlap between the groups. And as a result of each panel containing a mix of frequencies and amplitudes, each curve varies in both amplitude and frequency over time. This can be seen in the breadth of the spectral density plots on the right.
For the next obvious step, I used their data and variables, and I repeated their analysis.
Figure 5. My EEMD analysis of the Amazon river flow. Like the paper, I used an ensemble number of 2000, a noise amplitude of 0.6 standard deviations of the original signal, and 50 sifting iterations maximum.
I note that while my results are quite similar to theirs, they are not identical. The intrinsic modes C1 and C2 are apparently identical, but they begin to diverge starting with C3. The difference may be due to pre-processing which they have not detailed in their methods. However, I tried prefiltering with a Hanning filter, it’s not that. Alternatively, it may have to do with how they treat the creation of the splines at the endpoints of the data. However, I tried with end conditions of “none”, “wave”, “symmetric”, “periodic”, and “evenodd”. It’s none of those. I then tried an alternative implementation of the EEMD algorithm. The results were quite similar to the first implementation. Finally, I tried the CEEMD (complete ensemble empirical mode decomposition) method, which was nearly identical to my analysis shown above in Figure 5 .
I also could not replicate their results regarding the periodograms that they show in their Figure S1 (shown in Figure 4 above), although again I was close. Here are my results:
This makes it clear how the modes C1 to C6 each contain a variety of frequencies, and how they overlap with each other. However, I do not see a strong signal in the 9-13 year range in the intrinsic mode C3 as the authors found. Instead, the signals in that range are split between modes C2 and C3.
Now, their claim is that because mode C3 of the intrinsic modes of the Amazon River flow contains a peak at around 11 years (see Figure 4 above), it must be related to the sunspot cycle … while I find this method of decomposing a signal to be quite interesting, I don’t think it can be used in that manner. Instead, what I think is necessary is to compare the actual intrinsic modes of the Amazon flow with the intrinsic modes of the sunspots. This is the method used in EEMD2015. Here are the modes C3 of the Amazon flow and of the sunspots:
Now, it is true that intrinsic modes C3 of both the sunspot and the Amazon data contain a signal at around the general sunspot frequency. But other than that, the two C3 modes are quite dissimilar. Note for example that the sunspot mode C3 is phase-locked to the raw data. And in addition, the sunspot C3 amplitude is related to the amplitude of the raw sunspot data.
But to the contrary, the Amazon mode C3 goes into and out of sync with the sunspots. And in addition, the amplitude of the Amazon mode C3 has nothing to do with the amplitude of either the sunspot data or the sunspot C3 mode.
This method, of directly comparing the relevant intrinsic modes, is the method used in the original EEMD2005 paper linked to above. See for example their Figure 9 showing the synchronicity of the intrinsic modes C3 – C7 and higher of the Southern Ocean Index (SOI) and the El Nino Cold Tongue Index (CTI).
I find this to be a fascinating way to decompose a signal. It is even more interesting when all of the intrinsic modes are plotted to the same scale. Here are the sunspot intrinsic modes to the same scale.
Note that the overwhelming majority of the information is in the first three intrinsic modes. Beyond that, they are nearly flat. This is borne out by showing the periodograms to the same scale:
Now, this shows something fascinating. The EEMD analysis of the sunspots has two very closely related intrinsic modes. Mode C2 shows a peak at ten or eleven years, plus some small strength at shorter periods. Mode C3 shows a smaller peak at the same location, ten or eleven years, and an even smaller peak at sixteen years. This is interesting because not all of the strength of the ~ eleven-year sunspot signal falls into one intrinsic mode. Instead it is spread out between mode C2 and mode C3.
DISCUSSION: First, let me say that I would never have guessed that white noise could function as a bank of bandpass filters that automatically group related components of a signal into a small number of intrinsic modes. To me that is a mathematically elegant discovery, and one I’ll have to think about. Unintuitive as it may seem, noise aided data analysis is indeed a reality.
This method of signal decomposition has some big advantages. One is that the signal divides into intrinsic modes, which group together similar underlying wave forms. Another is that as the name suggests, the division is empirical in that it is decided by the data itself, without requiring the investigator to make subjective judgements.
What is most interesting to me is the showing by the authors of EEMD2005 that EEMD can be used to solidly establish a connection between two phenomena such as the Southern Ocean Index (SOI) and the El Nino Cold Tongue Index (CTI). For example, the authors note:
The high correlations on interannual and short interdecadal timescales between IMFs [intrinsic mode functions] of SOI and CTI, especially in the latter half of the record, are consistent with the physical explanations provided by recent studies. These IMFs are statistically significant at 95% confidence level based on a testing method proposed in Wu and Huang (2004, 2005) against the white noise null hypothesis. The two inter-annual modes (C4 and C5) are also statistically significant at 95% confidence level against the traditional red noise null hypothesis.
Indeed, Jin et al. (personal communications, their manuscript being under preparation) has solved a nonlinear coupled atmosphere-ocean system and showed analytically that the interannual variability of ENSO has two separate modes with periods in agreement with the results obtained here. Concerning the coupled short interdecadal modes, they are also in good agreement with a recent modeling study by Yeh and Kirtman (2004), which demonstrated that such modes can be a result of a coupled system in response to stochastic forcing. Therefore, the EEMD method does provide a more accurate tool to isolate signals with specific time scales in observational data produced by different underlying physics. SOURCE:EEMD2005 p. 20
Now of course, the question we are all left with at the end of the day is, to what extent do these empirical intrinsic modes actually represent physical reality, and to what extent are they merely a way to mathematically confirm or falsify the connections between two datasets at a variety of timescales? I fear I have no general answer to that question.
Finally, contrary to the authors of the paper, I would hold that the great disparity between all of the intrinsic modes of the Amazon flow data and of the sunspot data, especially mode C3 (Fig. 7), strongly suggests that there is no significant relationship between them.
Always more to learn … I have to think about this noise assisted data analysis lark some more …
My Usual Request: If you disagree with me or anyone, please quote the exact words you disagree with. I can defend my own words. I cannot defend someone’s interpretation of my words.
My New Request: If you think that e.g. I’m using the wrong method on the wrong dataset, please educate me and others by demonstrating the proper use of the right method on the right dataset. Simply claiming I’m wrong doesn’t advance the discussion.
Data: Available as an Excel workbook from the original article.
Code: Well, it’s the usual ugly mish-mash of user-aggressive code, but it’s here … I used two EEMD implementations, from the packages “hht” and “Rlibeemd”. If you have questions about the code, ask …