Guest Post by Willis Eschenbach
Once again, Dr. Curry’s “Week in Review-Science and Technology” doesn’t disappoint. I find the following:
Evidence of a decadal solar signal in the Amazon River: 1903 to 2013 [link] by Antico and Torres
So I go to the link, and I find the abstract:
Abstract
It has been shown that tropical climates can be notably influenced by the decadal solar cycle; however, the relationship between this solar forcing and the tropical Amazon River has been overlooked in previous research. In this study, we reveal evidence of such a link by analyzing a 1903-2013 record of Amazon discharge. We identify a decadal flow cycle that is anticorrelated with the solar activity measured by the decadal sunspot cycle. This relationship persists through time and appears to result from a solar influence on the tropical Atlantic Ocean. The amplitude of the decadal solar signal in flow is apparently modulated by the interdecadal North Atlantic variability. Because Amazonia is an important element of the planetary water cycle, our findings have implications for studies on global change.
The study is paywalled, but to their credit they’ve archived the data here as an Excel workbook. Let me start where I usually start, by looking at all of the raw data, warts and all.
Figure 1. Monthly average Amazon river flow (thousands of cubic metres per second). The violet colored sections are not observations. Instead, they are estimations based on the river levels in two locations on the Amazon.
Now to me, that’s a big problem right there. One violet section is based on river levels at one location, and the other violet section is based on river levels from another location. It’s clear from the annual average (red/black line) that the variance of those two river level datasets are very different. One river level dataset has big swings, the other has small swings … not good. So first I’d say that any results from such a spliced dataset need to be taken, as the old Romans said, “cum grano salis” …
Setting that question of spliced data aside, I next looked at the periodogram of the data. This shows the strength of the signal at various periods. If the ~11-year solar cycle is affecting the river flow, it will show a peak in the 11-year range.
Figure 2. Periodogram of the monthly Amazon river flow data shown in Figure 1.
It appears at first blush as if there is a very small 11-year signal in the full data (black), about 6% of the total range of the overall data swing. But when we split the data into the first half and the last half (red and blue), the 11-year signal disappears. This is not at all uncommon in observational datasets. Apparent cycles are often just the result of the analysis method averaging a changing signal.
Next, in Antico2015, the authors use the annual average data. To me, this is a poor choice. If you wish to remove the annual fluctuations, that’s fine … but using annual average data cuts your number of data points by a factor of 12. And this can lead to spurious results by inflating the apparent significance. But let us set that aside as well.
Finally, there is no statistically significant correlation between sunspots and Amazon river flow levels at any lag (max. monthly correlation ~ 0.1, p-value = 0.3 …).
Having seen that, my next step was to see how the authors of Antico2015 decided that there was a solar signal in the Amazon. And this was a most fascinating voyage. The best thing about climate science is that there is no end of the opportunities to learn. In this case, I learned from the Supplemental Online Information that they were using a method I’d never heard of, ensemble empirical mode decomposition, or EEMD. It’s one of many methods for decomposing a signal into the sum of other signals. Fourier analysis is the best known type of signal decomposition, and I’ve written before about the “periodicity” decomposition of Sethares, but there are other methods..
The details of EEMD are laid out by its developers in a paper called “Ensemble Empirical Mode Decomposition: A Noise Assisted Data Analysis Method” (hereinafter “EEMD2005”) … how could a data junkie like myself not like something called “noise assisted data analysis”?
The concept itself is quite simple. First, you identify local maxima and minima. See e.g. Figure 3 Panel b below, from the EEMD2005 paper, that shows the local maxima.
Figure 3. Graphic explaining the EEMD process, from the EEMD2005 paper. ORIGINAL CAPTION: The very first sifting process. Panel a is the input; panel b identifies local maxima (red dots); panel c plots the upper envelope (red) and low envelope (blue) and their mean (black); and panel d is the difference between the input and the mean of the envelopes.
Then after you identify local maxima (panel b) and local minima (not shown), you draw two splines, one through the local maxima and the other through the local minima of the dataset (red and blue lines, panel c). The first component C1 is the difference between the data and the local mean of the two splines (panel d).
Then you take the resulting empirical mode C1 as your dataset and do the same—you draw two splines, one through the local maxima and the other through the local minima of C1. The second component C2 is again the difference between the data and the local mean of those two splines.
Repeat that until you have a straight line.
How do you aid that with noise? Well, you repeat it a couple thousand times using the original data plus white noise, and you average the results. According to the paper, this acts as a bank of bandpass filters, and prevents the mixing of very different frequencies in any one component of the decomposition. What do I know, I was born yesterday … read the paper for the math and the full explanation.
In any case, when they use EEMD to decompose the Amazon flow data, here’s what they get. Each panel shows the resulting curve from each step in the decomposition.
Figure 4. This shows Figure S1 from the Supplementary Online Information of the Antico paper. ORIGINAL CAPTION: (Left) Annual mean (October-September) Amazon flow record at Obidos station, its oscillatory EEMD modes (C1-6), and its residual trend. (Right) Raw periodograms of flow modes. In these power spectra, the frequency band of the decadal sunspot cycle, at 1/13 to 1/9 cycles per year, is depicted by the shaded region, and the oscillatory period of the most prominent spectral peak of C3 is given in years. In the left panels, the fraction of total variance accounted by each mode is shown in parentheses. For a particular mode, this fraction is the square of the Pearson correlation coefficient between the mode and the raw data record. The sum of these fractions may be greater than 100% because EEMD is a nonlinear decomposition of data; therefore, the EEMD modes are not necessarily linearly independent. To obtain the EEMD decomposition of the annual mean flow record, we considered an ensemble number of 2000, a noise amplitude of 0.6 standard deviations of the original signal, and 50 sifting iterations.
This is a curious kind of decomposition. Because of the use of the white noise, each panel in the left column shows a curve that contains a group of adjacent frequencies, as shown in the right column. No panel shows a pure single-frequency curve, and there is significant overlap between the groups. And as a result of each panel containing a mix of frequencies and amplitudes, each curve varies in both amplitude and frequency over time. This can be seen in the breadth of the spectral density plots on the right.
For the next obvious step, I used their data and variables, and I repeated their analysis.
Figure 5. My EEMD analysis of the Amazon river flow. Like the paper, I used an ensemble number of 2000, a noise amplitude of 0.6 standard deviations of the original signal, and 50 sifting iterations maximum.
I note that while my results are quite similar to theirs, they are not identical. The intrinsic modes C1 and C2 are apparently identical, but they begin to diverge starting with C3. The difference may be due to pre-processing which they have not detailed in their methods. However, I tried prefiltering with a Hanning filter, it’s not that. Alternatively, it may have to do with how they treat the creation of the splines at the endpoints of the data. However, I tried with end conditions of “none”, “wave”, “symmetric”, “periodic”, and “evenodd”. It’s none of those. I then tried an alternative implementation of the EEMD algorithm. The results were quite similar to the first implementation. Finally, I tried the CEEMD (complete ensemble empirical mode decomposition) method, which was nearly identical to my analysis shown above in Figure 5 .
I also could not replicate their results regarding the periodograms that they show in their Figure S1 (shown in Figure 4 above), although again I was close. Here are my results:
Figure 6. Periodograms of the six flow modes, Amazon River data.
This makes it clear how the modes C1 to C6 each contain a variety of frequencies, and how they overlap with each other. However, I do not see a strong signal in the 9-13 year range in the intrinsic mode C3 as the authors found. Instead, the signals in that range are split between modes C2 and C3.
Now, their claim is that because mode C3 of the intrinsic modes of the Amazon River flow contains a peak at around 11 years (see Figure 4 above), it must be related to the sunspot cycle … while I find this method of decomposing a signal to be quite interesting, I don’t think it can be used in that manner. Instead, what I think is necessary is to compare the actual intrinsic modes of the Amazon flow with the intrinsic modes of the sunspots. This is the method used in EEMD2015. Here are the modes C3 of the Amazon flow and of the sunspots:
Figure 7. Raw data and intrinsic empirical mode C3 for the Amazon (top two panels) and for the sunspots (bottom two panels)
Now, it is true that intrinsic modes C3 of both the sunspot and the Amazon data contain a signal at around the general sunspot frequency. But other than that, the two C3 modes are quite dissimilar. Note for example that the sunspot mode C3 is phase-locked to the raw data. And in addition, the sunspot C3 amplitude is related to the amplitude of the raw sunspot data.
But to the contrary, the Amazon mode C3 goes into and out of sync with the sunspots. And in addition, the amplitude of the Amazon mode C3 has nothing to do with the amplitude of either the sunspot data or the sunspot C3 mode.
This method, of directly comparing the relevant intrinsic modes, is the method used in the original EEMD2005 paper linked to above. See for example their Figure 9 showing the synchronicity of the intrinsic modes C3 – C7 and higher of the Southern Ocean Index (SOI) and the El Nino Cold Tongue Index (CTI).
I find this to be a fascinating way to decompose a signal. It is even more interesting when all of the intrinsic modes are plotted to the same scale. Here are the sunspot intrinsic modes to the same scale.
Figure 8. EEMD analysis of the annual mean sunspot numbers. All panels are printed to the same scale.
Note that the overwhelming majority of the information is in the first three intrinsic modes. Beyond that, they are nearly flat. This is borne out by showing the periodograms to the same scale:
Figure 9. Periodograms of the EEMD analysis of the annual mean sunspot numbers. All panels are printed to the same (arbitrary) scale.
Now, this shows something fascinating. The EEMD analysis of the sunspots has two very closely related intrinsic modes. Mode C2 shows a peak at ten or eleven years, plus some small strength at shorter periods. Mode C3 shows a smaller peak at the same location, ten or eleven years, and an even smaller peak at sixteen years. This is interesting because not all of the strength of the ~ eleven-year sunspot signal falls into one intrinsic mode. Instead it is spread out between mode C2 and mode C3.
DISCUSSION: First, let me say that I would never have guessed that white noise could function as a bank of bandpass filters that automatically group related components of a signal into a small number of intrinsic modes. To me that is a mathematically elegant discovery, and one I’ll have to think about. Unintuitive as it may seem, noise aided data analysis is indeed a reality.
This method of signal decomposition has some big advantages. One is that the signal divides into intrinsic modes, which group together similar underlying wave forms. Another is that as the name suggests, the division is empirical in that it is decided by the data itself, without requiring the investigator to make subjective judgements.
What is most interesting to me is the showing by the authors of EEMD2005 that EEMD can be used to solidly establish a connection between two phenomena such as the Southern Ocean Index (SOI) and the El Nino Cold Tongue Index (CTI). For example, the authors note:
The high correlations on interannual and short interdecadal timescales between IMFs [intrinsic mode functions] of SOI and CTI, especially in the latter half of the record, are consistent with the physical explanations provided by recent studies. These IMFs are statistically significant at 95% confidence level based on a testing method proposed in Wu and Huang (2004, 2005) against the white noise null hypothesis. The two inter-annual modes (C4 and C5) are also statistically significant at 95% confidence level against the traditional red noise null hypothesis.
Indeed, Jin et al. (personal communications, their manuscript being under preparation) has solved a nonlinear coupled atmosphere-ocean system and showed analytically that the interannual variability of ENSO has two separate modes with periods in agreement with the results obtained here. Concerning the coupled short interdecadal modes, they are also in good agreement with a recent modeling study by Yeh and Kirtman (2004), which demonstrated that such modes can be a result of a coupled system in response to stochastic forcing. Therefore, the EEMD method does provide a more accurate tool to isolate signals with specific time scales in observational data produced by different underlying physics. SOURCE:EEMD2005 p. 20
Now of course, the question we are all left with at the end of the day is, to what extent do these empirical intrinsic modes actually represent physical reality, and to what extent are they merely a way to mathematically confirm or falsify the connections between two datasets at a variety of timescales? I fear I have no general answer to that question.
Finally, contrary to the authors of the paper, I would hold that the great disparity between all of the intrinsic modes of the Amazon flow data and of the sunspot data, especially mode C3 (Fig. 7), strongly suggests that there is no significant relationship between them.
Always more to learn … I have to think about this noise assisted data analysis lark some more …
w.
My Usual Request: If you disagree with me or anyone, please quote the exact words you disagree with. I can defend my own words. I cannot defend someone’s interpretation of my words.
My New Request: If you think that e.g. I’m using the wrong method on the wrong dataset, please educate me and others by demonstrating the proper use of the right method on the right dataset. Simply claiming I’m wrong doesn’t advance the discussion.
Data: Available as an Excel workbook from the original article.
Code: Well, it’s the usual ugly mish-mash of user-aggressive code, but it’s here … I used two EEMD implementations, from the packages “hht” and “Rlibeemd”. If you have questions about the code, ask …
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
“…our findings have implications for studies on global change.”
————
“Global change” – we’re doooomed!!!!
Wondering if you have seen Paul Pukite’s work on SOI and QBO.
http://contextearth.com/2015/09/04/the-qbom-part-2/
It makes for a very long punch-line but thank you for the LOL, Willis!
Thanks for the post Willis.
Have to figure out how to apply this type of analysis to my data, Center of Pressure data on a force plate. But looks like a promising method of analysis. Been typically using Correlation Dimension, Sample Entropy, and Higuchi’s Fractal Dimension.
Next the adjustments will be installed on the satellites via the operating system NOAA installs before launch.
Picking cherries no longer is sufficient. To get data that supports the preordained conclusion now requires picking fruit salad.
The periodogram shows peaks in the Fourier space that are not sharp. There are ‘sidebands’ of the main peak, especially at the longer periods such as 2.5 yr and 3.7 yr. The ‘beating’ of these frequencies is responsible for the periodic modulation of periodic signals (e.g. C2, C3, and C4 in figure 5). To some extent there is a contribution to these rounded peaks due to the truncation of the raw data at the ends of the domain. These rounded peaks are what is passing through the computational ‘bandpass filters’ as closely related cycles.
A more ‘elegant’ approach, in my opinion, would be to work in the Fourier space, to sequentially identify statistically significant peaks (amplitude > 2 sigma for the linear regression of the FFT) one at a time, separate them from the data, subtract the Inverse FFT of the peaks (boxcar truncation) from the original data, and reprocess the reduced data set. This can be repeated as long as statistically significant peaks can be identified within the FFT space.
A familiarity with the FFTs of pure noise of various types is helpful here. The FFT of white noise is ‘flat’ – the linear regression of amplitude vs frequency (alternatively amplitude vs period) has no statistically significant slope. Brown noise has an FFT in which the ‘envelope’ of amplitude shows a linear dependence on frequency/period, as does the periodogram of this data when one looks at the non-peak amplitudes in Fourier space.
But what do I know?
tadchem said December 11, 2015 at 9:02 am in part “ The FFT of white noise is ‘flat’ “
Surprisingly, NOT so. Here I am NOT referring to the fact that any one white sequence is FAR from flat. Rather, even if you average the FFT magnitudes of perhaps a million different white sequences, there will always be a (very significant) dip in the magnitude at DC (k=0), and at k=N/2 (only when N is even). This dip is to just about 90% of the “plateau” level: it’s 2(sqrt(2))/pi or 0.9003163. For k=0 and k=N/2 (for N even) we have, for a REAL time signal (usual case), an ordinary one-dimensional “Drunkard’s walk” with the mean of a folded normal distribution. For all other k, the “walk” is two-dimensional and the mean is that of a Rayleigh distribution.
http://electronotes.netfirms.com/NotFlat.jpg
A full detailed description from 2012 is here:
http://electronotes.netfirms.com/EN208.pdf
You probably will want to verify this is TRUE before considering my explanations! Email me for additional details. Yes – another way the FFT can sneak up and eat your lunch on you.
Bernie
We are a way beyond picking data. Today we are adjusting data.
As always, quite interesting, and thank you again.
Some time ago I read papers on how the sensory organs (of which animals, I do not remember) make use of the random thermal noise to sharpen discrimination. This is at least “associatively” related, if not directly comparable in detail, to the method you presented here.
Recordings on CD used to have noise added to the analogue signal to cover up quantisation distortion (audible volume stepping) at low signal levels to compensate for the logarithmic response of the ear.
Probably still do, in fact.
Excellent article Willis. Thank you.
What I noticed in your journey through EEMD and CEEMD land was that most of the solar signal appears in the reconstructed data.
Another tidbit caught my eye. That is the Amazon river output has increased.
From around 1930 to 1948-9 the Amazon river flow appears to be approximately 16,000 – 17,000 cubic meters of water per second. This range appears limited in variability.
From approximately 1968 to the current period, the flow has greater variability. What is interesting is how the overall flow increases during that time period; with peaks as high as 20,000 cubic meters/sec. The average also appears to be higher, perhaps a 1,000 cubic meter/sec, though that is a guess.
Looking around seeking Amazon River discharge rates was enlightening.
From Earth Observatory:
Research carried out in:
So, beginning around 1996-1998, Amazon River discharge rates are measured via satellite.
Meaning that Willis was trying to analyze data originating from four, perhaps five data sources, not three. Two separate reconstruction locations,
One, likely two different river mouth estimation series; the second estimation is when the series begins again in 1968-9 until replaced by satellite measurement.
One satellite measurement series.
Without trying to locate Vorosmarty’s paper, I am left wondering how the satellite measures water levels where fresh water meets salt water and South Atlantic tidal forces.
I no longer worry about the Amazon River discharge rates perhaps increasing; now that the latter period is measured by satellites.
Doesn’t anyone in NASA ever wonder about the conjugal habits of their data?
I haven’t bothered to seek the ‘Global River Discharge Database’ yet. Because that way lies more work, as I am curious just what Amazon river discharge levels are used in calculating sea level rise.
IMO largely from deforestation rather than increased precipitation.
Willis, you seem to find surprises in every study you undertake and you don’t tend to leave stones unturned. Thank you again for your fine work. I’m left with some questions as I’m sure you are, but maybe not the same questions.
1) I haven’t read the paper and won’t pay for it, but I’m puzzled if they didn’t look at the baseline causes of flooding in this mighty river. First, the baseline cause is precipitation- rain and snowfall/melt. If this information is available, it should give an even more convincing connection between sunspots and flooding if there is one. Also, flooding is unlikely to be a simple what goes in must come out situation. In dry years, there is low ground (swamps) that may drain and in some basins, aquifers decant. Heavy precipitation may be partly or largely swallowed up by these and thirsty vegetation by the time the water has traversed a good part of the basin. Also, after a dry period, sloughing of the banks are likely dam water temporarily or divert it through other low ground causing delay. Precipitation is best for the analysis.
2) You said there were some other methods that might have been used. The suspicious nature I’ve developed over the past decade or so gives me cause to think that the authors likely tried the usual methods and had to go to an unusual one to get their fit. Also, with two stations in the stew, I’m seeing red flags. I didn’t grasp the reason for this but assumed one would be a check for the other, or some such (perhaps revealing the data hiding in the noise). I can imagine if they initially tried several other stations on the river, they could cull out the ones they didn’t like and go with what they did go with. This seems to me a good way to get a high probability of a fit that probably is meaningless.
In any case, I note you are intrigued with the method and that tells me you will be reporting this in detail at some later date. I look forward to it.
Rainfall time series do not contain periodic components other than those due to the seasonal cycle. Once that is eliminated, the remaining variability is entirely stochastic; introducing white noise adds nothing to the discovery of random signals. Apparently unaware of rigorous cross-spectral methods to establish relationships between stochastic time series (and entirely ignoring the very apparent lack of coherence between Amazon rainfall and the sunspot cycle), Antico and Torres resort to analysis methodology that is as quaint as it is inappropriate, Sadly, analytically misguided attempts to squeeze a tendentious result out of the data are par for the course in “climate science.” They’re simply not worth the baffled attention given them.
Well doen, that exactly the unsubstantiated assumption that the whole the AGW is based on. ” the remaining variability is entirely stochastic; ”
Everything is “stochastic” except CO2, so whatever happens it must be due to CO2. QED.
Thank you 1sky1, that would answer my question as to why they didn’t use precipitation /snow melt. What else can they imagine causes flooding of a river? Could they really have not asked this question first and if the answer was that of 1sky1’s information above, then why do the study?
One possible way stochastic precipitation could be irrelevant to the flood cycle(?) would be if snow tended to have net accumulation for 10yrs and then melts were greater at the end of a sunspot cycle. But surely this would be well known for a long time and would be obvious without the need for iffy methods to tease out a signal. This one using interacting data from two stations on the river is what has got me. One station in the lower reaches of the river should do the job if it has any significance at all. What is the difference between floods on the river in between sun cycle maxima and minima? Volumetrically it must be small or it would be well known long before now. If small, we get into the problem of “small differences” with large error bars. I’m assuming there aren’t numerous, high precision stream flow guages, snow pack and melt guages. This isn’t Squaw Valley.
I can understand why Willis hasn’t been able to find any sunspot cycle signal in climate. With only a small change (1%) in TSI, and large error bars in all climate data, it’s more than just noise. One percent sounds non existent when you are measuring with axe handles and plugs of chewing tobacco and then adjusting the data on an algorithm daily. I think this quest to find a connection with the sun spots is becoming pathological. Whatever there may be has to be small. Other very long term variations in the sun’s output are another thing, but Willis has already pointed out that the much stronger annnual variations from orbital eccentricity don’t even show, that clouds forming 15 minutes earlier in the afternoon can wipe out this difference in potential heating. This all makes this paper ridiculous.
Willis Eschenbach wrote December 10, 2015 at 11:54 pm:
“Sorry, guys, but the original paper differentiates this use of white noise from the “stochastic resonance” use. See page 7 for their discussion of the differences between the two.
w.”
I assume”original paper” meant Zhaohua Wu and Norden E. Huang (2005) and that the relevant comments on page 7 are:
“Adding noise to the input to specifically designed nonlinear detectors could be also beneficial to detecting weak periodic or quasi-periodic signals based on a physical process called stochastic resonance. The study of stochastic resonance was pioneered by Benzi and his colleagues in early 1980’s. The details of the development of the theory of stochastic resonance and its applications can be found in a lengthy review paper by Gammaitoni et al. (1998). It should be noted here that most of the past applications (including these mentioned earlier) have not used the cancellation effects associated with an ensemble of noise-added cases to improve their results.”
I had already considered this and it was the basis for which I commented about averaging stochastic resonance. It seemed quite clear and I don’t see what else it could mean.
It is in fact, apparently, according to my quick look below, a pretty good idea, although I doubt that there is any averaging in most SR situations except as I noted for audio dither. But I am impressed. The figure shows the average output of 400 runs of SR where the signal is five cycles of a sinewave of amplitude 0.1, the added noise is uniform of amplitude 0.5, and the threshold (non-linear detector) is set at 0.5. Hence there is a detection when the noise boosts the sinewave peaks to between 0.5 and 0.6, which seems to be an average of about 25 of the 400 trials for any time point. Here is my result:
http://electronotes.netfirms.com/AvStoRes.jpg
I am impressed. Beyond this finding, the local jargons get in the way (as Joe Born suggested).
Bernie
Bernie Hutchins:I assume”original paper” meant Zhaohua Wu and Norden E. Huang (2005) and that the relevant comments on page 7 are:
thank you for your post. That looks like a good example. I am impressed.
Nobody’s mentioned this yet. This whole EEMD process was invented to deal with (among other things) non-stationary data. In other words, data where the solar signal is there…but not there either all the time or at the same frequency, phase or amplitude all the time. If the signal is not there all the time then Fourier analysis would have a harder time seeing it. That would have the effect of spreading out the energy over a range of frequencies and lowering the peaks, thus making it more difficult to spot.
However, what it means if the solar signal really is there but varies significantly over time is beyond me. Anyway, just an observation here.
Dear Willis,
As a late-career physicist who routinely uses advanced signal-processing techniques, all I can say is that this approach is BS. First, the addition of truly “stochastic” noise to data is a well known technique and has applicability IF the recording technique is non linear or quantized. This should be obvious in the case where a digitization recording technique has a bit resolution SMALLER than the signal. The addition of STOCHASTIC noise LARGER then the signal can raise the signal level above the one bit resolution of the digitizer. Then a repetitively recorded signal can eliminate the noise through ensemble averaging (whose definition includes the removal of stochastic noise – but I digress). FUNDAMENTALLY, the addition of stochastic noise to a signal is easily discernible in Wigner Space where the data is spread out in phase space. Stochastic noise is 2-D randomly scattered in Wigner Space. Localization of energy density in Wigner space corresponds to a true “non-stochastic” information. Stochastic noise addition will not help this situation with the exception noted above. Finally, the use of multi-resolution decomposition is well known in Wavelet Theory. The approach described in the the article is a mathematically non-rigorous decomposition. True wavelet mathematicians would laugh at their approach. (See Ingrid Daubechies’ books.) I don’t laugh, I just shake my head. There are numerous ways to mathematically attack this problem in a rigorous fashion. You described one approach. The other is a true time domain autocorrelation approach which would give nearly the same result. — I have no bias as to the result of the authors’ study but such a data analysis used in a physics article in any reputable journal (Phys Rev comes to mind) would have NEVER made it past the reviewers. C students – what can you say. Thanks so much for all of your effort. It is always a pleasure.
Rick
rbspielman December 11, 2015 at 4:30 pm
Rick, as a late-career troglodyte who routinely uses a stone for a hammer, all I can say is that you are arguing with the wrong man. At present, there are 31,000 hits on google for “EEMD signal”, and another 3,600 hits on Google Scholar. Go impress them all by telling them all about how you are a “late-career physicist”, and let them know that the 3,600 scientific studies referencing EEMD are “BS”. Here are the top four from google’s list:
I note that just these four studies of what you call “BS” have been cited by 439 other studies … I suppose those are all BS too …
Report back with your findings, Rick. I’m interested in what those folks say when you tell them that they don’t know what they are talking about, and that they are peddling BS and should read Daubechies …
w.
Willis Eschenbach: At present, there are 31,000 hits on google for “EEMD signal”, and another 3,600 hits on Google Scholar.
Excellent!
[Reply: ‘Chaam Jamal’ is a sockpuppet. Also posts under the name ‘Richard Molineux’ and others (K. Pittman, etc.) As usual, his sad life writing comments has been completely wasted, as they are now deleted. –mod]
Chaam Jamal December 11, 2015 at 5:34 pm
Yeah, but that’s just because it’s true …
It appears you missed the small print. When you do a Google search for “Eschenbach is a jerk”, it says at the top of the page:
Since Google ignores “a” and “is”, you’ve counted web pages with the words “Eschenbach” and “jerk” on them somewhere. I note that the top hit is for Canadian circle dancing … me, I do much more focused searches than that.
So I’d agree that in a meaningless search such as your example, the Google results are meaningless. I encourage you to do the narrow focused search I did for EEMD signal, and note what you find. There is not one single article in the top thirty that is not about the subject under discussion, including things like “Search for the 531-day-period wobble signal in the polar motion based on EEMD”. I got tired after reading the names of the first thirty or so of them. I strongly encourage you to do the same.
In any case, I started by mentioning 31,000 Google hits in the hopes he’d look at them. Then I noted the 3,600 Google Scholar hits, and I listed the top four of those google scholar hits. They show the EEMD analysis being used in the real world. I have conclusively shown that EEMD is not “BS” as the commenter rashly claimed.
w.
[Reply: ‘Chaam Jamal’ is a sockpuppet. Also posts under the name ‘Richard Molineux’ and others (K. Pittman, etc.) As usual, his sad life writing comments has been completely wasted, as they are now deleted. –mod]
And about 27,300 results for “Chaam Jamal is a jerk” too.
You don’t understand how to use Google, do you?
[Reply: ‘Chaam Jamal’ is a sockpuppet. Also posts under the name ‘Richard Molineux’ and others (K. Pittman, etc.) As usual, his sad life writing comments has been completely wasted, as they are now deleted. –mod]
All this is way, way over my head, but still fascinating. I have a question: Does it make a difference what kind of noise is used? What if one uses red noise instead of white?
I tried both uniform and gaussian noise, made no difference. Don’t know about red noise.
w.
A form of this matter of noise addition to improve detection was put to me as follows by an analytical chemist in my lab about 1970. We were using the new technique of atomic absorption spectrometry on an instrument with a logarithmic calibrated dial/needle readout.
He theorised “At very low concentrations the needle barely moves up from zero. Therefore, a bias is introduced because all readings below zero are assigned a zero, all just above have discrete values and therefore the mean of repeated readings is affected. If, before analysis, we add a small, known amount of extra analyte, we will shift the base from zero and allow an unbiased average.”
Eventually he agreed that the step of addition of analyte was equivalent to adding a source of noise and more error; and that in this case, the situation was made worse by taking the meter needle into the more compressed part of the visual scale, adding more noise through worse visual discrimination.
In the case being discussed by Willis, one approach is to categorise cases such as this and then strip them from the list of possible ways that the signal is allegedly enhanced.
Other bloggers have been attempting this by reference to human detection, such as adding noise to CD music, adding noise to visual imagery, dithering, etc. Examples that rely on human response should not be used because only a mathematical expression can be devoid of human frailties. The mathematical approach should avoid relative comparisons like whether the Floyd-Steinberg or Stucki dither method is the best. We need illustrations of how the addition of (first, perhaps) white noise has improved signal strength, expressed in quantitative mathematics.
(Unrelated factors have not allowed me to study the original paper yet – please excuse, I am trying. The example I gave is not trivial. I suspect it applies at least in part to sampling sub-pixel sizes.)
Thanks for the example, a great lesson in thinking.
The problem in the lab would be to ensure precision and accuracy in measuring the small amount of extra analyte. If the resulting value after subtracting the bias is close to zero there is a risk of multiplying any error in measuring the analyte. In a lab with closely controlled conditions this may be routine.
But in the wild, such precision is rare and that is the problem we find in climatology.
There is also an improved method, called CEEMD, for “Complete EEMD”. It appears to be a significant improvement on the original EEMD method in the higher numbered intrinsic modes, and is able to reproduce the original signal with greater fidelity.
?w=640
The discoverers’ paper explaining the method is here. I tried the CEEMD method as an alternate analysis method in the head post, but the differences were minor and I didn’t want to try to explain the CEEMD method, start at the start I figured … plus it’s slooow to compute, and the authors used EEMD, so that’s what I used.
The CEEMD function is a part of the “hht” package in R.
Anyhow, here’s a lovely test of the two methods from the paper above, that explores how the two methods decompose the Dirac delta function (a one-time jump in a signal);
Gotta say, that is sweet as!
w.
Wow, that is nice, Willis . Certainly puts a bit of context for those dismissing EEMD as BS.
I’ve got the SFT of the flowts with and without removeann=T , nearly identical in where peaks lie.
Now I want to get the C3 spectrum at monthly resolution, yearly is too crude to be much use. I’ve tried to adapt the code but it’s not going too well . Could you suggest how to do this?
Thx.
Mike –
Yes – it looks neat. But don’t you think it looks very much like a wavelet decomposition. (Not to mention the Fourier decomposition of a Delta.)
And I thought we were talking about detecting a very weak periodic component – exactly what a Delta function ISN’T. Your own work seems related to the periodic case?
yes Bernie. I think that plot shows the result is very similar to FFT, SFT or wavelet decomposition. The claim is that it is more robust in noisy data. Remains to be tested.
It’s quite surprising in view of the difference in method but again reassuring that it is consistent with FT.
I have now managed to do EEMD on the monthly Amazon data and the peaks are very close though not always identical to what I get using Willis’ SFT . The periods around 20-odd years changed by a couple of months , the other were exactly the same number of months.
The band-pass effect may be very useful in more noisy data but here it was detrimental in the case of the 18.6y peak. On C6 is in one transition band of the filter; on C7 it is on the other side. Due to very poor resolution of the adjacent peaks, this meant that I could not determine the peak centre when it got bent downwards.
Using SFT I get a peak value. Here is what I find from the Amazon rain data:
I did the SFT of the monthly flowts with and without the annual cycle and it was nearly identical in where the peaks lay.
8.91
10.75
13.5
18.58
21.25
26.08 y
I’m tempted to see 8.91y as the lunar apside period of 8.85 years but I’d be a little cautious since I would have more convinced it were nearer in view of the length of the data sample.
18.58 is clearly a lunar cycle. This year is a “minor lunar standstill” which is when the latitude of the moon comes closest to the ecliptic ( plane of the solar system ). Thus it also was in 1997 when when the last major El Nino developed.
10.75 and 21.25 are strongly suggestive of Schwabe and Hale, though the 10.75 is fairly small. Finding a solar signal and demonstrating it is small is probably as informative as not finding one.
However, with equally strong 18.58 and 21.25 barely being resolved from each other it is clear that ignoring possible lunar influence will beggar attempts to find or dismiss a solar signal. Those periods will go from in phase to opposite phase in about 74 years.
I was unimpressed by this paper when I read it last week. Having analysed the data I’m even less impressed since I think they failed to see the stronger Hale cycle and did not even consider the presence of lunar influence.
Anyway, mighty thanks to Willis for digging into this EEMD method and making his code available.
This is another useful tool to have available.
Just ran thsi through the spectral software I usually use and I don’t see any sign of the circa 21y peak !
The circa 18 is sitll there though nearer 18 than 18.6, ; circa 8y is 8.8 just the other side of 8.85 to that found with this technique. 10.8 is still there and is the strongest in decadal scale peaks.
Looks like Amazonian climate is affacted by both the sun and the moon, but I suppose the Azetcs could have told us that. 😉
Willis –
What is the provenance of the paper you have linked? No date! It looks like IEEE format, but even students submit assignments using that! Thanks.
After I found the paper on the web, I checked it against the CEEMD implementation in R that I’d used in the head post, and found that they were both the same. Hang on … OK, the documentation for the package “hht” (Hilbert-Huang Transform) gives it as:
w.
Frederick Colbourne — you presented a good work on 11 year cycle with Indian monsoon rainfall.
Solar radiation presented 11 year cycle but not rainfall
All-India Southwest Monsoon rainfall presented 60-year cycle. The third cycle started in 1987 [the starting year of Astrological calandar of 60 year but lagging by three years to Chinese 60 year astrological cycle.]. You can go back ward and forward from 1987 and take 10 year averages and plot on a graphy. You get clear sine curve.
In the case of Southwestern parts of India with northeast monsoon and pre-monsoon and post-monsoon cyclonic activity, the annual rainfall presented 132 year cycle. The new cycle started in 2001. If look at the data by separating for SWM and NEM, they presented 56 year cycles but in opposite direction. NEM 56 also reflected in the cyclonic activity. Both NEM & SWM precipitation showed an increasing trend basically because the first 66 years come under the below the average and next 66 years comes under above the average.
So, Indian monsoon is a complicated system as they are modified by orographic systems.
Dr. S. Jeevananda Reddy
Willis – What we really need here is:
(1) A well-defined (toy) test signal with the results compared for FFT, stochastic averaging, and any proposed new method. Has this been done?
(2) The essential, smiling first-class grad students at our doors to tell us what, if anything, it means and is good for!
Alas – I am retired and only hoping for (1).
Thanks, Bernie. Since the EEMD method is widely used in industry and science, I’m not sure that we need some abstract test.
Next, if you just take a look at the Google search results for “EEMD signal” (without quotes) you’ll find literally dozens of places where people are using it for a variety of things that it is good for.
w.
Thanks Willis –
I think this EEMD was new to you until a few days ago, and I had never heard of it until you posted. What I have not seen here or immediately adjacent is a consistent description of HOW one computes this. Thus I can not duplicate or assess the EEMD procedure at present. [ I was however, familiar with the use of noise to enhance detection (stochastic resonance) for some 22 years (thanks C.H.).]
I do not work well in a mode where I have to rely on a canned program (in R or even in Matlab). I prefer to (first) write my own often cumbersome equivalent code. Alternatively, I can consider running a whole menagerie of test signals (sines, steps, ramps, sines plus noise etc) through the can to see WHAT the EEMD does DO. The last figure you posted of the decomposition of the Delta is the sort of thing that helps – but it does look very wavelet-like.
An additional difficulty here is the application of an unfamiliar (to me) function to look for solar signals that most likely don’t even exist. These are poor tests. I don’t know what species of failure a negative result would mean.
I do greatly appreciate your calling attention to interesting things hiding in the corners. Do keep doing that for us. Thanks.
Bernie
Bernie Hutchins December 12, 2015 at 4:35 pm Edit
Indeed, until I read the Amazon paper I’d never heard of EEMD or CEEMD.
I can only agree wholeheartedly with the idea that the more of the nuts and bolts we understand, the better off we are. Here’s the matchbook explanation of EMD (not EEMD), which is the underlying algorithm.
Identify the local maxima and minima, and run a spline through them.
Average the two splines, and subtract that curve from the signal. What remains is the first intrinsic mode.
For EEMD, you do that say a hundred times or so, adding random noise each time, and average the results.
However, that is generalities, not the details wherein the devil resides. However, there’s a way around that. One of the beauties of R is that if you type in the function name, you get the the nuts and bolts of the function. For example, here’s the standard deviation function “sd”, obtained by typing in “sd” and hitting Enter:
> sd
function (x, na.rm = FALSE)
sqrt(var(if (is.vector(x)) x else as.double(x), na.rm = na.rm))
Here, we can see that R calculates the standard deviation function “sd” as the square root of the variance. (The variable “na.rm” specifies whether to ignore missing values indicated by NA).
If you use this same technique on the R function “EEMD”, you will assuredly have a “consistent description of HOW one computes this”. You’ll find the actual calculations are done by a call to a more basic function, “Sig2IMF”, which in turn calls a more basic function yet called “emd”.
Alternatively, the R program “seewave” contains a function “discrets” which identifies the local minima and maxima. You could use that plus the normal spline function to create the two splines yourself.
Since the Dirac delta is a simple function, the last figure I posted was a proof of concept that the CEEMD function operates the same as the more familiar versions. Where it differs is with complex functions, where the signal varies in both frequency and amplitude over time. Here, for instance, is the CEEMD analysis of 315 years of sunspot data.
?w=640
Go figure …
Nor do I. However, in the original 2005 EEMD paper they showed how they used EEMD to establish the similarity by comparing the intrinsic modes C3 and up of the SOI and the El Nino CTI.
In the Amazon paper, all they did was show that one of the intrinsic modes had power in the sunspot range, which they define as 9-13 years. To me, that seems far from adequate.
You are more than welcome. Life is an unending mystery to me, and I can only report back on what I find in my wanderings.
Regards,
w.
Willis –
Much thanks. Even as I admire your initiative and curiosity I also admire your patience and energy, quantities I myself find in decreasing supply in my dotage!
So you show me that the EEMD is really TWO things: the detection and removal of patterns, in turn; and the addition of noise.
In a noise-free signal, the pattern detection is essentially just what the human brain does without being specifically instructed. It is the basic approach of the reduced math (just “eyeball” it) Fourier Series articles of the 1950’s era Popular Electronics Magazine. We easily extend this, even to impromptu basis functions, and welcome the mathematical aids as encountered. So EEMD looks like a method of teaching a robot to do our own pattern recognitions. Fair enough.
The addition of noise is not so clearly warranted, except as one compares it to classic stochastic resonance. In the case of a noise-free signal, we can contemplate adding intentionally generated random noise – for some purpose. In the case of an already noisy signal, why add more? Indeed! You may already have enough, or more than enough. For example, “adding dither” may be just a matter of not trying so hard to reduce noise coming in.
With stochastic resonance however, it is CLEAR that the addition of noise HELPS. Consider the crayfish in the stream not wishing to encounter a bass who is looking for lunch. In a very quiet stream, the stealthy swishing tail of the bass may be insufficient to trigger the crayfish’s sensors. Now, the crayfish does not add noise, but the stream does: the turbulent “babbling” as water randomly encounters rocks. Enough that the peaks of the swish are now a thump, thump, thump. (My figure at December 11, 2015 at 3:39 pm) Essential here is the non-linearity (threshold). Once again (as with sonar), nature got there first.
So the noise-free case of eyeballing components is perhaps efficient (computation cost of one-time processes don’t really matter), it is not clear to me why and how noise helps, unless it is similar to stochastic resonance. Perhaps.
As for computer code, I did find some Matlab functions which I need to study and put together.
Thanks again Willis.
Bernie
The Amazon doesn’t pass through a flow meter. River and stream flows are estimates.
A common method the USGS uses is to set up a gauge to measure stream depth. The surface area of a cross section of the stream at the site is ascertained. The velocity of the stream is measured. From that the flow is estimated and tables produced giving so much flow for so much depth. Periodically that area of the cross section and the velocity are checked and the tables adjusted.
Methods may have changed over the years. I don’t know how they measure the flow of the Amazon but it all does remind of surface stations and temperature.
How accurately do the numbers reflect reality? The numbers, such as they are, may be the best we have but adjusting an estimate using another estimate might give you more decimal points but no more accuracy.
Willis,
After more digging I will accept that this is a neat example of counter intuition. I still have uncertainties about identification of types of data when noise should be added, but that is me.
It is a bad day when one does not learn something new.
Geoff.
Thanks, Geoff.
w.
for some reason it more feels like what they call in digial audio as the process of “dithering”. just in reverse mode.
Dithering adds and substracts noise for known audio signal distortions when you convert the bit depth of an audio signal. It adds general background noise but does reduce the more audible bit reduction sharp distortion that is audible.
the difference here is that it uses “known” outcomes of distorted patterns in the sound and fils it in with the interpolated values of the reduced grid.
why do i say reverse? Well here the signals are not known, you got like the “first step of dithering (=applying noise) that has been done” but this first step is unknown, some cycles are known so then you can “guess dither” the second step with the amazone values for any known cycle. However as fourrier analysis from the raw data shows, that may just be a small signal nearly undetectable that may add to the reduction towards a straight line.
EEMD is the second step of dithering but with the first step unknown, it can therefore magnify this signal to a level that’s blown out of proportions This because the added “noise” of your “audio” (here the values of the amazon river) are decomposed is “unknown” and all the sine waves that compose the “noise” are unknown. Therefore ther may be a catch in this that can put scientists/mathematicians on a wrong leg, even if the whole methodology is scientifically or mathematically correct.
so in short you can find with EEMD any cycle that can link, however as the interaction is unknown and the weather related noise follows an unknown, working this way backwards can be misleading. the best is to just apply a fourrier analysis of the raw data and then make conclusions.
it’s a bit like “torturing the data till it shows what you are looking for” but then in a scientific way: nothing is wrong with the methods used, but it can give false (mainly exagerated) correlations
Frederik –
I, like you, would like to understand EEMD based on ideas from digital audio like dither (and stochastic resonance). I have been unable so far to complete the connections!
One thing that I think helps is to turn the problem around. You are not adding noise to a perfectly good signal, but adding a signal to perfectly good noise.
Another thing is to recognize that the digital audio “art” is not so much science as it is (fiendishly clever) ENGINEERING aimed at a practical product. Essential here are the ideas of “over-sampling” and “noise-shaping” to manage noise. Over-sampling drastically increases the sampling rate during playback far far above the audio needs. This is by temporarily generating extra samples (locally on the fly by interpolation), using them, and discarding them. The “quantization noise” is then shaped into the high frequency range thus opened up (inaudible). In addition, the number of bits can thereby be reduced (resolution below LSB), even to just one bit! Deterministic, but the various waveforms look a lot like dither had been used.
The value of comparing something new to something you already understand is of immense value.
Bernie
Thanks Bernie you added nicely in what i tried to explain with the fact that i said “in reverse”
Actually i am a digital sound creator and from any random noise of nature i can make beats and sounds that sound melodic.
To achieve this, i filter out random noises of thunders and other sounds to a specific range so that it sounds like a beat or instrument. Thus filtering out the noise till you get a left over of a specific set of harmonics. by “filtering out the interfering harmonics that create noise”.
This is what i would call to apply EEMD on perfect noises: take out the blur that masks the sound and intonation you hear in for example a thunder, to create a sound more perfect by eliminating the other freuqencies.
So i do these practises daily but then just on the ear to have a good sounding beat without too much interfering random noise
This principle does remind me very hard of what i do. The problem is that the focus on the inexistant sine waves with natural events will give one anyway. The Amazon river data will show an 11 year cycle or actually more accurate: “an 11 year sine wave with variety in intensity.”
I believe they hurried to make their conclusions. however in correct noise it is not good to average an “in intensity varying wave that is inherent to noise”
that effect explains why willis can’t find an 11 year cycle when he splits the data in half. here instead of cycles per second for audio, we talk about cycles of 11 years and more..
so this means that in 40 years suddenly the fourrier analysis may suddenly show the 11 year cycle got canceled out or increased in strength. Both can happen
I often use fourier analysis in my soundwork to find the “base wave” so now i’m in for an analogy: when you take the amazon raw data: here we have an exception: seasonal variability that gives a clear signal, Like this i would not be surprised to see also a signal for other events in our solar system that can or intensify if well aligned or cancel each other out if aligned in opposition. this will create “noise” even in the “clear signal you see “noise” as every wave is not of the same amplitude.
so yes with enough data and very precise data you will see our “celestial harmonics” in yhe fourrier analysis (in fact all of them). The question then becomes: “If this signal is a range of 6% is it significant ? or is the signal of the seasonal variability that strong that it can easyly and repeatedly cancel out the 11 year cycle?”
when i look at the split data Willis provided i suspect the second question will be answered with “yes”, which then would make the influence insignificant.
if this wave would then change amplitude like the basic wave varying in a chaotic pattern, then it more looks like an artifact of the “noise of the seasonal cycle” rather then a real cycle
i hope i made some sense as English is not my native language… if not clear just ask.
“…for some reason it {EEMD} more feels like what they call in digial audio as the process of “dithering”. just in reverse mode.”
Nice example and exposition.
Frederik Michiels December 14, 2015 at 8:53 am
Frederik, you seem to have missed the fact that EEMD is an analysis method that is used successfully in a wide range of real-world applications. Do you think that they are doing that because EEMD “can give false (mainly exagerated) correlations”. Do you reckon they use EEMD because it “can be misleading”, or because using EEMD you can find “any cycle that can link?
All you’ve given us are handwaving objections and imaginary problems. You have not shown that even one of those objections is true; you have not supported even one of those objections with a single example; and you have not explained the details of even one of your claims that EEMD might give you the wrong answers.
In short, you’re just waving your hands and saying that you think EEMD is “torturing the data till it shows what you are looking for” … and without a single fact, citation, or reference to back you up, I’m sorry, but I simply don’t believe you know what you are talking about.
w.
Willis –
Right – BUT. Recall that Principal Components was a well-established method at the point where Mann invented a new way of normalizing the data which brought out a hockey stick. Who knows for sure, but don’t we all at times suppose that we have “finally done something right” when we see what we were expecting, if not outright hoping for? There seem to be a fair number of users who compare EEMD to wavelets and prefer wavelets. Caution per se has merit.
Bernie
Bernie, I’m not following you. What is your point here?
Should we use caution? Sure. Can EEMD be compared to wavelets? Sure, you can compare anything to anything. Is EEMD the same as wavelets? Absolutely not.
Next, Mann did not “invent a new way of normalizing the data”, I hope that was sarcasm. He made a stupid mistake and didn’t notice it because it fit his preconceptions … but what on earth does that have to do with whether EEMD is a valuable tool? We know for a fact already that it is a valuable tool, because people are using it all over the world for real-world problems.
I say again … what is your point here?
w.
Willis my friend – you said December 14, 2015 at 11:17 am, in part:
“I say again … what is your point here?”
Three points.
POINT 1: First, you said in the top post:
“Finally, contrary to the authors of the paper, I would hold that the great disparity between all of the intrinsic modes of the Amazon flow data and of the sunspot data, especially mode C3 (Fig. 7), strongly suggests that there is no significant relationship between them.”
Now, if I understand you correctly (here and in the past) you don’t see evidence of any 11 year cycles, specifically not in the case of river flow rate. So a tool that shows such a mode is in some way flawed or being misused? The fact that the same tool is properly used elsewhere by others is not relevant to the Amazon River. Thus if it found an artifact, it is misleading us all. It’s like claiming that a FFT could find a linear trend, and pointing out that the FFT is highly regarded as useful.
POINT 2: (peripherally related) you said:
“….He made a stupid mistake and didn’t notice it because it fit his preconceptions …”
while I said:
“….we all at times suppose that we have ‘finally done something right’ when we see what we were expecting, if not outright hoping for?….”
These are much the same – the same human foible.
POINT 3 – Certainly MY failing but it still do not have much idea how or why noise is used in EEMD (unless it is a stochastic resonance means of detecting weaker modes) and I have not seen any basic “tutorial” on EEMT (one good ppt never gets to noise) that demonstrates the procedure and puts it through its paces, comparing to FFT, wavelets, etc.
Thanks for your time.
Bernie
Bernie Hutchins December 14, 2015 at 1:23 pm
It’s hard for me to say much about them purportedly finding the 11-year cycle, mostly because I found no such cycle either in my Fourier analysis or the EEMD analysis. I can’t reproduce their results. So I don’t know whether “it found an artifact” as you say, or whether they were using on a different dataset, or using it incorrectly.
I also disagreed with their method, which was different from the method of the authors of the EEMD2005 paper. The Antico2015 author merely determined if one isolated frequency were present, while the EEMD2005 authors compared four different intrinsic modes.
I was just objecting to your claim that “Mann invented a new way of normalizing the data”, when all he did was make a math error.
Did you read the EEMD2005 document? I thought it explained the reason for the noise quite clearly, and why noise improved the previously-used EMD method. Even the abstract explains what the noise does …
And it is discussed in greater detail further down in the study, viz:
Not sure what more in the way of explanation you were looking for. They also have precise step-by-step details on the method in section 3.3.
And yours as well.
w.
will uote yourselfon this:
when you add noise to a noisy signal and then decompose it, into group related compinents you are on a dangerous zone where i thus say that though entirely correct in all ways to do it this does not work entirely correct on noisy data.
the data of the amazon river is noise, but it is not only “white noise only” so like with sound it will resonate with parts of the white noise and not with other parts so when used as a bandpassfilter it will indeed divide the signal in intrinsic modes it will do that with every signal you do it with so yes some “harmonics” will pass through
i think you missed the point about the “dithering in reverse”point i made as that was the point i was trying to make
The point with “torturing the data” here is thus a “reverse one” with the point being: “you will find with EEMD in all river patterns an 11 year cycle even more on every aspect of our planet’s weather behavior you will find it with this method. I even do not need a proof for that, it’s obvious that this influence is there and that it is measurable,
i think you missed the analogy with sound i made i that regard: in sound each IMF would be a harmonic component of the result of white noise band pass a sound with flutter in it consistent with the variable frequency of EMD processing. each IMF would then be seen in the spectral analisys of that resulting sound (sound works additive which is why the analogy is maybe not entirely empirical correct)
does it also prove a huge impact or is that impact too small on these scales to make a difference?” so if yes for which parts would it be significant and fot which parts would it be too small to have an impact?
Frederik December 14, 2015 at 8:00 pm
Frederik, thanks for your thoughts. No, you will NOT “find with EEMD in all river patterns an 11 year cycle even more on every aspect of our planet’s weather behavior you will find it with this method”. In fact, I didn’t find the 11-year signal in the Amazon using this method.
How about you actually USE THE METHOD AND GET SOME RESULTS before lecturing us on what the method will and will not do?
Come back with some real information, like an actual river or other observational data series that you have actually analyzed and actually found the 11-year cycle in BEFORE you try to lecture us all about how much you know.
Finally, your idea that you “do not need proof” for your claims merely reveals that you don’t understand the scientific method. First, you can’t prove anything in science. More importantly, you need support (logic, math, observations, previous studies, etc.) for any scientific claim that you might make. It’s a brand new method to you, near as I can tell you’ve never actually used it. Get some backup for your claims.
w.