Guest Post by Willis Eschenbach
Are there cycles in the sun and its associated electromagnetic phenomena? Assuredly. What are the lengths of the cycles? Well, there’s the question. In the process of writing my recent post about cyclomania, I came across a very interesting paper entitled “Correlation Between the Sunspot Number, the Total Solar Irradiance, and the Terrestrial Insolation“ by Hempelmann and Weber, hereinafter H&W2011. It struck me as a reasonable look at cycles without the mania, so I thought I’d discuss it here.
The authors have used Fourier analysis to determine the cycle lengths of several related datasets. The datasets used were the sunspot count, the total solar irradiance (TSI), the Kiel neutron count (cosmic rays), the Geomagnetic aa index, and the Mauna Loa insolation. One of their interesting results is the relationship between the sunspot number, and the total solar irradiation (TSI). I always thought that the TSI rose and fell with the number of sunspots, but as usual, nature is not that simple. Here is their Figure 1:
They speculate that at small sunspot numbers, the TSI increases. However, when the number of sunspots gets very large, the size of the black spots on the surface of the sun rises faster than the radiance, so the net radiance drops. Always more to learn … I’ve replicated their results, and determined that the curve they show is quite close to the Gaussian average of the data.
Next, they give the Fourier spectra for a variety of datasets. I find that for many purposes, there is a better alternative than Fourier analysis for understanding the makeup of a complex waveform or a time-series of natural observations. Let me explain the advantages of an alternative to the Fourier Transform, which is called the Periodicity Transform, developed by Sethares and Staley.
I realized in the writing of this post that in climate science we have a very common example of a periodicity transform (PT). This is the analysis of temperature data to give us the “climatology”, which is the monthly average temperature curve. What we are doing is projecting a long string of monthly data onto a periodic space, which repeats with a cycle length of 12. Then we take the average of each of those twelve columns of monthly data, and that’s the annual cycle. That’s a periodicity analysis, with a cycle length of 12.
By extension, we can do the same thing for a cycle length of 13 months, or 160 months. In each case, we will get the actual cycle in the data with that particular cycle length.
So given a dataset, we can look at cycles of any length in the data. The larger the swing of the cycle, of course, the more of the variation in the original data that particular cycle explains. For example, the 12-month cycle in a temperature time series explains most of the total variation in the temperature. The 13-month cycle, on the other hand, is basically nonexistent in a monthly temperature time-series.
The same is true about hourly data. We can use a periodicity transform (PT) to look at a 24-hour cycle. Here’s the 24-hour cycle for where I live:
Figure 2. Average hourly temperatures, Santa Rosa, California. This is a periodicity transform of the original hourly time series, with a period of 24.
Now, we can do a “goodness-of-fit” analysis of any given cycle against the original observational time series. There are several ways to measure that. If we’re only interested in a relative index of the fit of cycles of various lengths, we can use the root-mean-square power in the signals. Another would be to calculate the R^2 of the cycle and the original signal. The choice is not critical, because we’re looking for the strongest signal regardless of how it’s measured. I use a “Power Index” which is the RMS power in the signal, divided by the square root of the length of the signal. In the original Sethares and Staley paper, this is called a “gamma correction”. It is a relative measurement, valid only to compare the cycles within a given dataset.
So … what are the advantages and disadvantages of periodicity analysis (Figure 2) over Fourier analysis? Advantages first, neither list is exhaustive …
Advantage: Improved resolution at all temporal scales. Fourier analysis only gives the cycle strength at specific intervals. And these intervals are different across the scale. For example, I have 3,174 months of sunspot data. A Fourier analysis of that data gives sine waves with periods of 9.1, 9.4, 9.8, 10.2, 10.6, 11.0, 11.5, and 12.0 years.
Periodicity analysis, on the other hand, has the same resolution at all time scales. For example, in Figure 2, the resolution is hourly. We can investigate a 25-hour cycle as easily and as accurately as the 24-hour cycle shown. (Of course, the 25-hour cycle is basically a straight line …)
Advantage: A more fine-grained dataset gives better resolution. The resolution of the Fourier Transform is a function of the length of the underlying dataset. The resolution of the PT, on the other hand, is given by the resolution of the data, not the length of the dataset.
Advantage: Shows actual cycle shapes, rather than sine waves. In Figure 2, you can see that the cycle with a periodicity of 24 is not a sine wave in any sense. Instead, it is a complex repeating waveform. And often, the shape of the wave-form resulting from the periodicity transform contains much valuable information. For example, in Figure 2, from 6AM until noon, we can see how the increasing solar radiation results in a surprisingly linear increase of temperature with time. Once that peaks, the temperature drops rapidly until 11 PM. Then the cooling slows, and continues (again surprisingly linearly) from 11PM until sunrise.
As another example, suppose that we have a triangle wave with a period of 19 and a sine wave with a period of 17. We add them together, and we get a complex wave form. Using Fourier analysis we can get the underlying sine waves making up the complex wave form … but Fourier won’t give us the triangle wave and the sine wave. Periodicity analysis does that, showing the actual shapes of the waves just as in Figure 2.
Advantage: Can sometimes find cycles Fourier can’t find. See the example here, and the discussion in Sethares and Staley.
Advantage: No “ringing” or aliasing from end effects. Fourier analysis suffers from the problem that the dataset is of finite length. This can cause “ringing” or aliasing when you go from the time domain to the frequency domain. Periodicity analysis doesn’t have these issues
Advantage: Relatively resistant to missing data. As the H&W2011 states, they’ve had to use a variant of the Fourier transform to analyze the data because of missing values. The PT doesn’t care about missing data, it just affects the error bars.
Advantage: Cycle strengths are actually measured. If the periodicity analysis say that there’s no strength in a certain cycle length, that’s not a theoretical statement. It’s a measurement of the strength of that actual cycle compared to the other cycles in the data.
Advantage: Computationally reasonably fast. The periodicity function I post below written in the computer language “R”, running on my machine (MacBook Pro) does a full periodicity transform (all cycles up to 1/3 the dataset length) on a dataset of 70,000 data points in about forty seconds. Probably could be sped up, all suggestions accepted, my programming skills in R are … well, not impressive.
Disadvantage: Periodicity cycles are neither orthogonal nor unique. There’s only one big disadvantage, which applies to the decomposition of the signal into its cyclical components. With the Fourier Transform, the sine waves that it finds are independent of each other. When you decompose the original signal into sine waves, the order in which you remove them makes no difference. With the Periodicity Transform, on the other hand, the signals are not independent. A signal with a period of ten years, for example, will also appear at twenty and thirty years and so on. As a result, the order in which you decompose the signal becomes important. See Sethares and Staley for a full discussion of decomposition methods.
A full periodicity analysis looks at the strength of the signal at all possible frequencies up to the longest practical length, which for me is a third of the length of the dataset. That gives three full cycles for the longest period. However, I don’t trust the frequencies at the longest end of the scale as much as those at the shorter end. The margin of error in a periodicity analysis is less for the shorter cycles, because it is averaging over more cycles.
So to begin the discussion, let me look at the Fourier Transform and the Periodicity Transform of the SIDC sunspot data. In the H&W2011 paper they show the following figure for the Fourier results:
Figure 3. Fourier spectrum of SIDC daily sunspot numbers.
In this, we’re seeing the problem of the lack of resolution in the Fourier Transform. The dataset is 50 years in length. So the only frequencies used by the Fourier analysis are 50/2 years, 50/3 years, 50/4 years, and so on. This only gives values at cycle lengths of around 12.5, 10, and 8.3 years. As a result, it’s missing what’s actually happening. The Fourier analysis doesn’t catch the fine detail revealed by the Periodicity analysis.
Figure 4 shows the full periodicity transform of the monthly SIDC sunspot data, showing the power contained in each cycle length from 3 to 88 years (a third of the dataset length).
Figure 4. Periodicity transform of monthly SIDC sunspot numbers. The “Power Index” is the RMS power in the cycle divided by the square root of the cycle length. Vertical dotted lines show the eleven-year cycles, vertical solid lines show the ten-year cycles.
This graph is a typical periodicity transform of a dataset containing clear cycles. The length of the cycles is shown on the bottom axis, and the strength of the cycle is shown on the vertical axis.
Now as you might expect in a sunspot analysis, the strongest underlying signal is an eleven year cycle. The second strongest signal is ten years. As mentioned above, these same cycles reappear at 20 and 22 years, 30 and 33 years, and so on. However, it is clear that the main periodicity in the sunspot record is in the cluster of frequencies right around the 11 year mark. Figure 5 shows a closeup of the cycle lengths from nine to thirteen years.:
Figure 5. Closeup of Figure 4, showing the strength of the cycles with lengths from 9 years to 13 years.
Note that in place of the single peak at around 11 years shown in the Fourier analysis, the periodicity analysis shows three clear peaks at 10 years, 11 years, and 11 years 10 months. Also, you can see the huge advantage in accuracy of the periodicity analysis over the Fourier analysis. It samples the actual cycles at a resolution of one month.
Now, before anyone points out that 11 years 10 months is the orbital period of Jupiter, yes, it is. But then ten years, and eleven years, the other two peaks, are not the orbital period of anything I know of … so that may or may not be a coincidence. In any case, it doesn’t matter whether the 11 years 10 months is Jupiter or not, any more than it matters if 10 years is the orbital period of something else. Those are just the frequencies involved to the nearest month. We’ll see below why Jupiter may not be so important.
Next, we can take another look at the sunspot data, but this time using daily sunspot data. Here are the cycles from nine to thirteen years in that dataset.
Figure 6. As in figure 5, except using daily data.
In this analysis, we see peaks at 10.1, 10.8, and 11.9 years. This analysis of daily data is much the same as the previous analysis of monthly data shown in Figure 5, albeit with greater resolution. So this should settle the size of the sunspot cycles and enshrine Jupiter in the pantheon, right?
Well … no. We’ve had the good news, here’s the bad news. The problem is that like all natural cycles, the strength of these cycles waxes and wanes over time. We can see this by looking at the periodicity transform of the first and second halves of the data individually. Figure 7 shows the periodicity analysis of the daily data seen in Figure 6, plus the identical analysis done on each half of the data individually:
Figure 7. The blue line shows the strengths of the cycles found using the entire sunspot dataset as shown in Figure 6. The other two lines are the cycles found by analyzing half of the dataset at a time.
As you can see, the strengths of the cycles of various lengths in each half of the dataset are quite dissimilar. The half-data cycles each show a single peak, not several. In one half of the data this is at 10.4 years, and in the other, 11.2 years. The same situation holds for the monthly sunspot half-datasets (not shown). The lengths of the strongest cycles in the two halves vary greatly.
Not only that, but in neither half is there any sign of the signal at 11 years 10 months, the purported signal of Jupiter.
As a result, all we can do is look at the cycles and marvel at the complexity of the sun. We can’t use the cycles of one half to predict the other half, it’s the eternal curse of those who wish to make cycle-based models of the future. Cycles appear and disappear, what seems to point to Jupiter changes and points to Saturn or to nothing at all … and meanwhile, if the fixed Fourier cycle lengths are say 8.0, 10.6, and 12.8 years or something like that, there would be little distinction between any of those situations.
However, I was unable to replicate all of their results regarding the total solar irradiance. I suspect that this is the result of the inherent inaccuracy of the Fourier method. The text of H&W2011 says:
4.1. The ACRIM TSI Time Series
Our analysis of the ACRIM TSI time series only yields the solar activity cycle (Schwabe cycle, Figure 6). The cycle length is 10.6 years. The cycle length of the corresponding time series of the sunspot number is also 10.6 years. The close agreement of both periods is obvious.
I suggest that the agreement at 10.6 years is an artifact of the limited resolution of the two Fourier analyses. The daily ACRIM dataset is a bit over 30 years, and the daily sunspot dataset that he used is 50 years of data. The Fourier frequencies for fifty years are 50/2=25, 50/3=16.7, 50/4=12.5, 50/5=10, and 50/6=8.3 year cycles. For a thirty-two year dataset, the frequencies are 32/2=16, 32/3=10.6, and 32/4=8 years. So finding a cycle of length around 10 in both datasets is not surprising.
In any case, I don’t find anything like the 10.6 year cycle they report. I find the following:
Figure 8. Periodicity analysis of the ACRIM composite daily total solar irradiance data.
Note how much less defined the TSI data is. This is a result of the large variation in TSI during the period of maximum solar activity. Figure 9 shows this variation in the underlying data for the TSI:
Figure 9. ACRIM composite TSI data used in the analysis.
When the sun is at its calmest, there is little variation in the signal. This is shown in the dark blue areas in between the peaks. But when activity increases, the output begins to fluctuate wildly. This, plus the short length of the cycle, turns the signal into mush and results in the loss of everything but the underlying ~ 11 year cycle.
Finally, let’s look at the terrestrial temperature datasets to see if there is any trace of the sunspot cycle in the global temperature record. The longest general temperature dataset that we have is the BEST land temperature dataset. Here’s the BEST periodicity analysis:
Figure 10. Full-length periodicity analysis of the BEST land temperature data.
There is a suggestion of a cycle around 26 years, with an echo at 52 years … but nothing around 10-11 years, the solar cycle. Moving on, here’s the HadCRUT3 temperature data:
Figure 11. Full-length periodicity analysis of the HadCRUT3 temperature record.
Curiously, the HadCRUT3 record doesn’t show the 26- and 52-year cycle shown by the BEST data, while it does show a number of variations not shown in the BEST data. My suspicion is that this is a result of the “scalpel” method used to assemble the BEST dataset, which cuts the records at discontinuities rather than trying to “adjust” them.
Of course, life wouldn’t be complete without the satellite records. Here are the periodicity analyses of the satellite records:
Figure 12. Periodicity analysis, RSS satellite temperature record, lower troposphere.
With only a bit more than thirty years of data, we can’t determine any cycles over about ten years. The RSS data server is down, so it’s not the most recent data.
Figure 11. Periodicity analysis, UAH satellite temperature record, lower troposphere.
As one might hope, both satellite records are quite similar. Curiously, they both show a strong cycle with a period of 3 years 8 months (along with the expected echoes at twice and three times that length, about 7 years 4 months and 11 years respectively). I have no explanation for that cycle. It may represent some unremoved cyclicity in the satellite data.
SUMMARY:
To recap the bidding:
• I’ve used the Periodicity Transform to look at the sunspot record, both daily and monthly. In both cases we find the same cycles, at ~ 10 years, ~ 11 years, and ~ 11 years 10 months. Unfortunately when the data is split in half, those cycles disappear and other cycles appear in their stead. Nature wins again.
• I’ve looked at the TSI record, which contains only a single broad peak from about 10.75 to 11.75 years.
• The TSI has a non-linear relationship to the sunspots, increasing at small sunspot numbers and decreasing a high numbers. However, the total effect (averaged 24/7 over the globe) is on the order of a quarter of a watt per square metre …
• I’ve looked at the surface temperature records (BEST and HadCRUT3, which show no peaks at around 10-11 years, and thus contain no sign of Jovian (or jovial) influence. Nor do they show any sign of solar (sunspot or TSI) related influence, for that matter.
• The satellite temperatures tell the same story. Although the data is too short to be definitive, there appears to be no sign of any major peaks in the 10-11 year range.
Anyhow, that’s my look at cycles. Why isn’t this cyclomania? For several reasons:
First, because I’m not claiming that you can model the temperature by using the cycles. That way lies madness. If you don’t think so, calculate the cycles from the first half of your data, and see if you can predict the second half. Instead of attempting to predict the future, I’m looking at the cycles to try to understand the data.
Second, I’m not blindly ascribing the cycles to some labored astronomical relationship. Given the number of lunar and planetary celestial periods, synoptic periods, and the periods of precessions, nutations, perigees, and individual and combined tidal cycles, any length of cycle can be explained.
Third, I’m using the same analysis method to look at the temperature data that I’m using on the solar phenomena (TSI, sunspots), and I’m not finding corresponding cycles. Sorry, but they are just not there. Here’s a final example. The most sensitive, responsive, and accurate global temperature observations we have are the satellite temperatures of the lower troposphere. We’ve had them for three full solar cycles at this point. So if the sunspots (or anything associated with them, TSI or cosmic rays) has a significant effect on global temperatures, we would see it in the satellite temperatures. Here’s that record:
Figure 12. A graph showing the effect of the sunspots on the lower tropospheric temperatures. There is a slight decrease in lower tropospheric temperature with increasing sunspots, but it is far from statistically significance.
The vagaries of the sun, whatever else they might be doing, and whatever they might be related to, do not seem to affect the global surface temperature or the global lower atmospheric temperature in any meaningful way.
Anyhow, that’s my wander through the heavenly cycles, and their lack of effect on the terrestrial cycles. My compliments to Hempelmann and Weber, their descriptions and their datasets were enough to replicate almost all of their results.
w.
DATA:
SIDC Sunspot Data here
ACRIM TSI Data, overview here, data here
Kiel Neutron Count Monthly here, link in H&W document is broken
BEST data here
Sethares paper on periodicity analysis of music is here.
Finally, I was unable to reproduce the H&W2011 results regarding MLO transmissivity. They have access to a daily dataset which is not on the web. I used the monthly MLO dataset, available here, and had no joy finding their claimed relationship with sunspots. Too bad, it’s one of the more interesting parts of the H&W2011 paper.
CODE: here’s the R function that does the heavy lifting. It’s called “periodicity” and it can be called with just the name of the dataset that you want to analyze, e.g. “periodicity(mydata)”. It has an option to produce a graph of the results. Everything after a “#” in a line is a comment. If you are running MatLab (I’m not), Sethares has provided programs and examples here. Enjoy.
# The periodicity function returns the power index showing the relative strength
# of the cycles of various lengths. The input variables are:
# tdata: the data to be analyzed
# runstart, runend: the interval to be analyzed. By default from a cycle length of 2 to the dataset length / 3
# doplot: a boolean to indicate whether a plot should be drawn.
# gridlines: interval between vertical gridlines, plot only
# timeint: intervals per year (e.g. monthly data = 12) for plot only
# maintitle: title for the plat
periodicity=function(tdata,runstart=2,runend=NA,doplot=FALSE,
gridlines=10,timeint=12,
maintitle="Periodicity Analysis"){
testdata=as.vector(tdata) # insure data is a vector
datalen=length(testdata) # get data length
if (is.na(runend)) { # if largest cycle is not specified
maxdata=floor(datalen/3) # set it to the data length over three
} else { # otherwise
maxdata=runend # set it to user's value
}
answerline=matrix(NA,nrow=maxdata,ncol=1) # make empty matrix for answers
for (i in runstart:maxdata) { # for each cycle
newdata=c(testdata,rep(NA,(ceiling(length(testdata)/i)*i-length(testdata)))) # pad with NA's
cyclemeans=colMeans(matrix(newdata,ncol=i,byrow=TRUE),na.rm=TRUE) # make matrix, take column means
answerline[i]=sd(cyclemeans,na.rm=TRUE)/sqrt(length(cyclemeans)) # calculate and store power index
}
if (doplot){ # if a plot is called for
par(mgp=c(2,1,0)) # set locations of labels
timeline=c(1:(length(answerline))/timeint) #calculate times in years
plot(answerline~timeline,type="o",cex=.5,xlim=c(0,maxdata/timeint),
xlab="Cycle Length (years)",ylab="Power Index") # draw plot
title(main=maintitle) # add title
abline(v=seq(0,100,gridlines),col="gray") # add gridlines
}
answerline # return periodicity data
}
They speculate that at small sunspot numbers, the TSI increases. However, when the number of sunspots gets very large, the size of the black spots on the surface of the sun rises faster than the radiance, so the net radiance drops.
Sunspots [‘black spots’] always decrease TSI. And big spots especially. The increase in TSI with the sunspot number is due to another effect: the presence of ‘faculae’ [magnetic debris (or sometimes precursors to spots) surrounding the spots]. More spots, especially the smaller decaying ones or the just born ones, mean more faculae. The effect of bright faculae on TSI is about twice that of the dark spots, so the faculae win and we end up with the well-known solar cycle variation of TSI.
The three peaks you see in the sunspot spectrum is a natural effect of a long-term [100yr] modulation of the sunspots. If the series is short the 100yr ‘cycle’ won’t show up as you noticed.
Leif, I’m glad you’re the first to comment, your input is always interesting. You say:
The sunspot data that I have is about 250 years in length. Let me extend the length of the analysis to 120 years. Unfortunately that just gives us two and a half cycles of data at 100 years, but needs must when the devil drives. If the effect is strong, it should show up …
…
OK, here’s the results … I’m not finding any hundred-year cycle in the sunspot data.
What are you basing the claim of the 100 year cycle on? I’m not seeing it in the data.
w.
I shall read this one with great interest, lot to get through, from casual look I can see it will be well worth while.
W,
Don’t assume a Fourier series necessarily indicates discrete underlying cycles, or that a Fourier series can’t find cycles with non- N/t periodicity, N being the total number of input data. An FT gives the spectrum of sines where the sum weighted sum and phases fit the data. Sines are not even required in a Fourier transform. Sines happen to be one any number of complete sets that can fit any well behaved curve. You may choose to filter spectrum and retransform to remove noise. Look into what you can do with a compose in a Fourier series. That trick has quite a bit of power to work magic. You definitely will have fun with it. It is one of the secrets behind Photoshop.
Willis Eschenbach says:
July 29, 2013 at 1:12 pm
What are you basing the claim of the 100 year cycle on? I’m not seeing it in the data
It is visible to the naked eye: http://sidc.be/sunspot-index-graphics/wolfaml.php
or even an FFT-analysis http://www.leif.org/research/FFT-Power-Spectrum-SSN-1700-2008.png
The many curves result from calculating the spectrum for 1700-2008, then 1701-2008, 1702-2008, etc to show the effect of choosing a different interval [that was an issue when I computed the spectrum – have forgotten why]. You can see the ~100-yr peak and its harmonics at 100/2, 100/3, etc.
I’ve been using a combination technique to fit data to known periods (daily, annual). Do a mulitple linear regression on a Fouier type harmonic series. The independent factors are cos(x), sin(x), cos(2x), sin(2x), cos(3x), sin(3x)…… where x = 2*PI*cycle length. The resulting coefficients give you the shape of the cycles and you can get an estimate of error for each harmonic. I find most curves fit well with three or less harmonics. Try it out on CO2 data.
Fascinating reading. It’s almost as if you just decided to look at the data as analytically as you could, and then tried to derive conclusions based solely upon the data instead of reinforcing your previously held beliefs. You’ve even incautiously suggested that you don’t know the full picture. I’m sorry, but this marks you out as a rank amateur in the rarefied field of climate science.
[I don’t really have to put /sarc, do I?]
Very interesting article Willis. Wish I had more time tonight to go into this. But a few comments:
Someone called Bart did a very interesting frequency analysis on SSN that comes up with several of the the same periods you found, and not to be impolite in any way, he does seem a lot more experienced than you with fourrier type techniques.
http://tallbloke.wordpress.com/2011/07/31/bart-modeling-the-historical-sunspot-record-from-planetary-periods/
Also you can try chirp transform instead of simple fourrier, this gets around the clunky quantisation of frequencies. You need to be careful of trends when doing this sort of thing and use a windowing function on the data before the F.T.
Your quick look around at various datasets is interesting. I did a much more detailed look at just SST, basin by basin:
http://climategrog.wordpress.com/2013/03/01/61/
One interesting thing to come out of this was that HadSST seems to mess up a prominent 9y cycle that is present in the original ICOADS data in nearly all basins and convertis to circa 7.5 years 🙁
Since this frequency also was recently reported by Judith Curry and BEST team from the land record I think this is a problem with Hadley processing.
PS. you’ll also note 3.7 is quite strong in a lot of the regions in SST. It is not a satellite defect.
3.8 years in trade wind data is probably the same thing:
http://climategrog.wordpress.com/?attachment_id=283
Also note SSN periods are clearly visible.
Thank you for the Sethares and Staley paper. I hope the approach becomes more widely known. I am sending copies to some friends who study biological rhythms. We have mostly gotten away from FFTs for reasons like those you cite.
This looks like a good application of the technique. The results are intriguing and I am sure that they will stimulate thought.
I am glad that you demonstrated the absence of stationarity. The non-stationarity of the climate system is one of the problems that makes it hard to analyze.
Figure 1 is interesting, I’ve not seen this done before. One feature I note is the vertical bands that seem to be spaced by about 15 years. Maybe this indicates that duration of each phase across the solar surface that Lief has refered to as the “real” length of the solar cycle ( as opposed to the repetition frequency of circa 11 years).
I’ll have a longer look at this tomorrow.
Matthew R Marler says: I am glad that you demonstrated the absence of stationarity. The non-stationarity of the climate system is one of the problems that makes it hard to analyze.
Working with rate of change is usually sufficient for most datasets. Thoughtful use of windowing functions is another common technique.
Leif Svalgaard says:
July 29, 2013 at 1:28 pm
Thanks, Leif. The FFT analysis is very interesting. Is there any chance you could send me a link to the data, so that I could do a more complete periodicity analysis?
In any case, your FFT link is excellent because it reveals a limitation of the Fourier analysis.
You’ve used 308 years of data from 1700-2008. This means that the only cycles that the Fourier analysis will reveal are 308/2 = 154 years, 308/3 = 103 years, 308/4 = 77 years, and so on. These individual points are clearly visible in the Fourier analysis.
As a result, you won’t find any evidence of say a 125-year cycle in the Fourier decomposition of a 308-year signal, even though such a cycle may certainly exist in the data.
This is one of the advantages of the periodicity analyses. It does not decompose the cycle into a series of theoretical sine waves at specified frequencies. Instead, it reveals the actual cycles at all the individual frequencies (periods).
As I said, your comments on this stuff are always welcome.
w.
Leif Svalgaard says:
July 29, 2013 at 1:28 pm
‘You can see the ~100-yr peak and its harmonics at 100/2, 100/3, etc.”
From your FFT spectrum, there is indeed a peak near 100 years and 50 years, but a strong “anti-peak” at 33 and nothing much at all at 25 years.
This should discomfort the ‘it’s the sun, stupid’ crowd.
Could the lack of correlation between sunspot numbers and temperatures be due to attempting to match (possibly accurate) sunspot numbers to highly adjusted, homogenised, UHI’d, in-filled, smoothed and distorted temperature ‘data’?
While we’re about it I suppose I should throw in Arctic sea ice:
http://climategrog.wordpress.com/?attachment_id=118
Strong 11.8 and its harmonic at 5.54 years.
Hoser says:
July 29, 2013 at 1:23 pm
Please don’t think I’m dissing the Fourier Transform. Joe Fourier was a genius. I remember when I first read of his work. I was stunned by the idea that any arbitrary wave could be represented as the superposition of nothing more than a set of sine waves … astonishing insight. And people have done dazzling things with it in a host of disciplines.
I’m just saying that for certain applications, periodicity analysis is more useful than Fourier analysis, for the reasons I list above. Someone commented that they wanted to look into it for biological rhythms. It’s superb for that, because those rhythms are often not sine-shaped waves. Periodicity analysis allows you to see the actual shape of a circadian or other cycle, as shown in Figure 2.
Thanks,
w.
Leif Svalgaard says:
July 29, 2013 at 12:50 pm
Interesting. Sounds like it would vary as something like the ratio of total spot area to total edge length. More smaller spots for a given area have more edge length, leading to exactly what’s shown in Figure 1. Thanks for the explanation.
w.
John Trigge (in Oz) says:
July 29, 2013 at 2:37 pm
Thanks, John, Even if that were the case, the 10-11 year cycle should show up strongly in the satellite data, but it doesn’t.
w.
Willis,
Nice article. As far as Berkeley vs. HadCRUT3 goes, its something of an apples-to-oranges comparison as one is land-only and the other is a land/ocean composite. Comparing Berkeley and CRUTEM4 (and NCDC land-only while you are at it) would be much more interesting. I suspect they would be quite similar, given how similar the overall anomalies are: http://i81.photobucket.com/albums/j237/hausfath/globallandtempcomps1850-2013_zps9d383290.png
Willis, did you get a chance to download the data I put up for you?
I think it would make a better source of data. 120 million samples, but I wouldn’t go before 1950, and don’t forget the limited samples of 1971-2. One of my charts is sample count it shows how lousy surface station data is, And what you get from GISS, BEST, CRU is all made up data. The spectrum you get from these data sets are lacking as you saw.
I have copies of a version of CRU and BEST, and they have lost all of the resolution of the data. When you homogenize and average the data into global averages of almost any kind, you loss so much.
A daily global average is a fabrication.
I also found when you start averaging areas north and south of the equator, it smears out the seasonal signals, even a wide range of either north or south causes loss of signal. Then tropics have 2 cycles instead of one. Yearly average, poof it’s all gone.
Philip Bradley says:
July 29, 2013 at 2:34 pm
“It’s the thermostat, stupid” …
w.
Lance Wallace says: From your FFT spectrum, there is indeed a peak near 100 years and 50 years
and the “100 years” look uncomfortably like half the dataset length !
The wikipedia entry on solar cycles has the comment
Hale’s observations revealed that the solar cycle is a magnetic cycle with an average duration of 22 years. However, because very nearly all manifestations of the solar cycle are insensitive to magnetic polarity, it remains common usage to speak of the “11-year solar cycle”.
I notice that the BEST and HADCRUT3 periodicity curves have peaks somewhat close to the 22-ish position. Maybe this is connected to the sun.
Apart from a millennial cycle such as MWP to LIA to date the sun is pretty irregular on shorter time scales of 3 or 4 solar cycles as witness the active sun and accompanying warmth of the late 17th century and early 18th century before the sun became less active and temperature dropped again in the late 18th and in the 19th century..
Furthermore the modulating effect of the oceans has an effect sometimes supplementing and sometimes offsetting any solar variations.
So I don’t think there are any clear cycles on time scales of less than a millennium or so and even that varies in length due to the ocean effect.
However, the absence of clear shorter term cycles does not imply that there is no link at all between solar activity and a temperature trend at specific levels of solar activity.
It just isn’t a neat, regularly repeating relationship. Just a tendency for temperatures to rise slowly when the sun is more active than a specific (currently uncertain) level and for temperatures to fall slowly when the sun is less active than that specific level.
I think someone did suggest a specific level of the Ap index at which the system would switch from cooling to warming or vice versa.
Suggesting that such an approach is ‘cyclomania’ is a straw man argument.
Much obliged for the essay, Willis Well done. I’m going to look into PT.
I urge all readers to take this observation to heart. It is the “Smoking Gun at Berkley Earth”.
My hobby horse has been that BEST’s scalpel process destroys low frequency data and the suture and regional homogenization creates counterfeit low frequency output — it looks real, but divorced from reality. Willis and I went several rounds on this topic in “Berkley Earth Finally Makes Peer Review…. ” Jan 19-23, 2013.
Willis ( 1/22/13 12:45 am
Willis, I submit that your observations and analysis that HadCRUT3 and BEST do not show the same long period results is confirmation that the scalpel and suture is destroying important signal and substituting false artifacts. If so, climatic results with a time scale beyond 6 years from BEST work should be highly suspect and probably discarded.
I made may argument from Fourier Theorems, not because I believe long term cycles have a cause, but from an Information Content argument. A Temp vs Time time series has a certain amount of information. The Fourier Transform of that series has exactly the same information content because there is a 1 to 1 correspondence between the two. The BEST scalpel, by shortening each time series, removes the lowest possible frequency in the Fourier spectrum. Information is lost. The lowest frequency information is lost — that which is most important to climate science. High frequency information cannot be used to recreate low frequency information. Regional homogenization will not save the day, for if you look at the regional homogenization in the Fourier Space, the low frequency component of the maps have gone to NULL.
http://wattsupwiththat.com/2009/12/08/the-smoking-gun-at-darwin-zero/
Greg Goodman says:
July 29, 2013 at 2:06 pm
Greg Goodman says:
July 29, 2013 at 2:10 pm
Thanks for both of those, I’ll take a look at that.
While you may certainly be correct, I caution again against interpreting Fourier results as real cycles. Read my comment above. If there is a strong 125-year cycle in Leif’s data, it will not show up in the Fourier analysis,despite the fact that it is a real cycle. The same is true of say a 90-year cycle. The Fourier analysis doesn’t show the actual cycles in the data. Instead, it shows a unique and non-overlapping (orthogonal) set of theoretical sine waves that add up to the data.
By the way, could I make a plea that graphs contain a link to the data used? It’s so frustrating to have to dig the data out, and often impossible.
w.
And of course whatever the sun does the effect is modulated by the thermostat effect which Willis has previously alluded to as a tropical phenomenon but which I would extend to the entire global air circulations.
My post at 2.55pm should have referred to the F10.7 Flux and not the Ap index.
Hi Willis
Some 10 years ago I produced this equation
http://www.vukcevic.talktalk.net/LFC4.htm
(on the graph numbers are rounded off) which has ~105 year period, except at the time of Maunder minimum when period halved.
I was flogging my ideas on another blog, and the well known solar expert declared it astrology, insisting that it is the well known Gleissberg cycle (70 years) that dominated the sunspot cycles.
My stance was that such cycle did not exist and that sunspot long term output is modulated by ‘Maunder equation’ as I call it, and challenge the good old doc to produce the FFT spectrum.
Instead of producing single spectral line, Dr. Svalgaard came with this:
http://www.vukcevic.talktalk.net/FFT-105y.htm
My comment reproduced below the graph clearly shows sentiments of the conversation at the time.
This particular topic may still be available on the old ‘SC24 blog’ but I couldn’t trace it, someone else may be more successful. I was banned for misbehaviour so I made record of many of my posts.
Dr. Svalgaarrd’s recollection may be different and if put forward I shall not challenge it.
As far as I know this is first time anyone defined 105 year cycle, but I could be wrong.
“It just isn’t a neat, regularly repeating relationship. Just a tendency for temperatures to rise slowly when the sun is more active than a specific (currently uncertain) level ”
First we need to understand the solar activity , then we need to understand the climate systems response to whatever part of solar activity is most important : TSI, UV, magnetism, solar wind.
Then we need to untangle anything else like a strong 9 year cycle that is probably lunar in original. Anyone expecting a nice 1 to 1 correlation to SSN is being foolishly simplistic. Equally those stating that solar is unimportant because of the lack of such a trivial relationship are being equally foolish.
There are clear signals but the relationship is not trivial.
Neither is bundling the whole global temperature data into a “global average” very helpful. Especially if we mix land and sea data. Tropics, temperate and polar regions get hit differently and have different responses and feedbacks.
Global average ‘mania’ has held back any real understanding of climate mechanisms for decades. Probably intentionally so by those wishing to analyse it as CO2 plus noise.
Brave of you to make so many basic errors on the capability of the Fourier Transform on a page frequented by many electrical / electronic engineers…
A simple DFT gives those numbers because that is all of the information that is in the data set. Any other data is, by definition, interpolation (which we know from the mathematical underpinnings of the transform). But if you want to perform the interpolation using the Fourier transform, it’s trivial – you just zero pad the time domain. Then you get any frequency you want.
Again, by definition, there is no additional information to give better resolution, and any method that gives better resolution must do so by interpolation – which is trivial and easy to do using the Fourier transform.
Cycle shapes can be trivially extracted from a Fourier transform from analysis of the harmonics.
Allow me to introduce you to window functions…
And allow me to introduce you to the Lomb-Scargle periodogram.
A 70,000 point FFT on my creaking, 8 year old lap top in MATLAB – including generating the random numbers to feed into it – took 0.01 seconds. Not forty. Not quite sure how you’re selling this as an advantage. (Command: tic; fft(randn(70000,1)); toc)
I’m sure periodicity analysis (just like all the other variants, such as singular spectrum analysis etc) has its merits, but I’d rather hear them from someone who actually understands Fourier analysis in the first place, if a comparison is to be made at all.
Global average ‘mania’ has held back any real understanding of climate mechanisms for decades.
Whole heartedly agree. I’d look for a solar effect in the desert zones. Australia has good data from a few arid desert locations. Although avoid the BoM’s Australian composite datasets. They have piled adjustment on adjustment.
Zeke Hausfather says:
July 29, 2013 at 2:46 pm
I would have thought the same, but curiously, not so. Here’s Best again for comparison:
and here’s CRUTEM4:
The most obvious difference is in the size of the peaks at around 52 years. Again, I suspect the result is because of the “scalpel” technique, but I have no way of demonstrating that.
Regards,
w.
I’m thinking that this shows more than anything else that a variety of tools used to examine data is best. I think to often people get locked into looking at things in only one way which is why an outside view can sometimes reveal amazing things. And yes other times multiple views don’t show anything, but that is data as well.
Spence_UK says:
July 29, 2013 at 3:19 pm
I understand that, Spence. The DFT returns a set of sine waves that perfectly reproduce the underlying signal. My point is that those may not be the actual waves that went together to make up the signal. Consider my example of a sine wave added to a sawtooth wave. The Fourier analysis will reconstitute the resultant wave perfectly out of a bunch of sine waves.
Butwhat Fourier analysis can’t do is give us back the original sine wave and triangle wave that were added together to make the resultant wave.
Spence, you have listed a number of ways to get around some of the limitations of the Fourier transforms. I’ve never denied that those ways exist. Nor, as I said above, am I dissing the DFT, it’s a work of art.
I’m just saying that there are advantages to the periodicity transform, some of which I’ve listed above. I’m comparing a simple DFT to a simple periodicity transform. Yes, there are wavelets, and ways to get around missing data. And yes, you can take a narrower window and sweep it across the data to improve your resolution. There’s lots of workarounds for the limitations of simple application of the DFT.
I’m just saying that the periodicity transform is a relatively simple transform that has certain advantages in certain situations. As Sethares and Staley point out with examples, there are things that it can do that the Fourier transform is not very good at, or can’t do at all. Different tools for different jobs is how I look at the two of them.
As to computational speed, neither the DFT nor the periodicity analysis are particularly fast, that’s why the Fast Fourier Transform was invented. Part of the difference is that the periodicity algorithm is calculating each individual frequency from low to high. So in a 70,000 point dataset, it’s analyzing some 23,000 individual cycles. If you wish to sweep a windowed DFT to that resolution across the same interval you’ll need to run it about 23,000 times plus overhead. (Although I’m sure there are clever algorithms for that as well.) My point is a homebuilt function with no attempt at optimization can run that in a reasonable time, always a plus on my planet.
Again, I’m not anti-Fourier. It’s just that sometimes instead of a saw I want a chisel …
w.
Greg Goodman says:
July 29, 2013 at 3:09 pm
/////////////////
An insightful comment.
I too have often commented upon the point you make in your final paragraph. The use of averages really hinders seeing what is going on and why.
Greg Goodman says:
July 29, 2013 at 2:01 pm
“Someone called Bart did a very interesting frequency analysis on SSN that comes up with several of the the same periods you found, and not to be impolite in any way, he does seem a lot more experienced than you with fourrier type techniques. “
That was I. The Sunspot data are the result of the rectification of primarily two processes with energy concentrated in frequency ranges centered at those associated with periods of about 20 and 23.6 years. When rectified, these produce the major observed peaks associated with about 10, 10.8, 11.8, and 131 years, in accordance with the Convolution Theorem.
I make the distinction of pointing out where the energy is concentrated because these are NOT periodic signals. If they were periodic, the energy would be distributed in lines, such as appear in atomic spectra. This type of behavior is fairly ubiquitous in a wide range of physical phenomena, owing to the fact that natural continuum processes can often be described by partial differential equations on a bounded domain, and such equations can often be solved as the expansion of a series of spatial eigenmodes associated with a time dependent amplitude function which is the output of a 2nd order ordinary differential equation. Dissipation of energy produces a broadening of the lines, and the processes manifest themselves as resonant phenomena driven by wideband random forcing.
I made such a model of the Sunspots here and showed how I could use it to generate qualitatively very similar outputs to the observed Sunspot behavior here and here. A Kalman Filter/Predictor could be formulated for this model which would produce optimal estimates of future behavior and associated error bars.
This kind of stuff is pretty old hat in control systems analysis and design. Someday, it will migrate over into the climate sciences.
One thing to perhaps consider: overall, the sun is going to generate a pretty constant amount of energy. Given that, the TSI is going to be fairly constant too, but as surface (at least) conditions change, the energy spectrum will change.
It might be interesting to explore how the Earth reacts to a constant TSI with a varying spectral density.
‘ The BEST scalpel, by shortening each time series, removes the lowest possible frequency in the Fourier spectrum. Information is lost. The lowest frequency information is lost — that which is most important to climate science. High frequency information cannot be used to recreate low frequency information. Regional homogenization will not save the day, for if you look at the regional homogenization in the Fourier Space, the low frequency component of the maps have gone to NULL.”
i suppose i could get a file done without scalpelling. However your description of it is wrong.
I feel strongly that one needs to work out what each individual solar output component does to the global temp,ETC, as TSI evens out individual components, and is thereby not that useful. We should be seeing how proton, electron, Ultra-Violet, X Rays, Ap, 10.7, etc, etc each individually have on earths various layers of atmosphere, jetstreams, temperature, magnetic effects, other reactions, the differing reactions at poles and differing latitudes, etc, etc. Using a broad brushed TSI is not going to achieve any real detailed meaningful results. And help us that much really in finding all the answers to Solar-Climate-Weather-etc interactions…There is so much out there we need to research and learn.
However, the problem lies in that most of this solar component data is only very recent!
Willis Eschenbach says:
July 29, 2013 at 3:47 pm
“But what Fourier analysis can’t do is give us back the original sine wave and triangle wave that were added together to make the resultant wave.”
Actually, the FFT is an isomorphism – it can give you back precisely what you put into it.
“Spence, you have listed a number of ways to get around some of the limitations of the Fourier transforms.”
It’s not a limitation of the Fourier Transform, but an aspect of the FFT implementation of it. The actual Fourier Transform is a continuous and dense function of frequency. The FFT is a sampled version of the Fourier Transform. Zero padding is merely a method to make it sample more points.
However, it really does not help with resolution in a strict sense. Fundamentally, resolution is limited by the length of the data set. E.g., you cannot generally isolate a 1000 year process in 10 years of data. But, you can use additional points of data to produce a plot which is more pleasing and recognizable to the human eye.
Anyway, the problem of looking for periodicities is that this is not really periodic data, but rather a random process with cyclical correlation, as I discussed above.
“…as I discussed above.”
Once it gets through the spam filter, I suppose.
Willis,
It’s an interesting analysis. But I think your list of disadvantages of the periodicity analysis is short but major.
The sunspot analysis was most interesting, so I ran your program for an exact sine of period 11 years, monthly data over 264 years. The full plot showed, rather like your Fig 4, a sequence of peaks at 22 yr, 33 yr etc diminishing quite slowly. These are spurious. The expansion about the 11 year also showed. like your Fig 5, side lobes at about 10.4 years and 11.7 years, though not as pronounced. It looks as though these are a sinc function relating to the 264 year window.
Willis, I appreciate what you’re saying, but methods you describe as “getting around limitations of the Fourier transform” apply equally to the periodicity transform – they are limits of the data, e.g. Nyquist sampling theorem.
As an example, I generated your figure 5 from using a Fourier Transform:
http://i42.tinypic.com/281wj0o.png
(Apologies for the cheesy hosting). As you can see, the Fourier Transform yields an almost identical plot to the periodicity transform. The limit of the information is dictated by the data, not the signal processing method. As someone quite familiar with frequency domain analysis, it is very difficult to get past the intro to the interesting part, without thinking “this is all wrong…”
Willis Eschenbach says:
July 29, 2013 at 2:32 pm
You’ve used 308 years of data from 1700-2008. This means that the only cycles that the Fourier analysis will reveal are 308/2 = 154 years, 308/3 = 103 years, 308/4 = 77 years, and so on. These individual points are clearly visible in the Fourier analysis.
As a result, you won’t find any evidence of say a 125-year cycle in the Fourier decomposition of a 308-year signal, even though such a cycle may certainly exist in the data.
It is very true that FFT is a blunt instrument for long-period cycles, as the time-resolution [really frequency] is poor for the longer period, but it is not completely useless: the peaks will still show up, but shifted somewhat. Here is an example: I added a 70-yr period to the SSN record [with same amplitude as average SSN] and in another plot I additionally added a 125-yr period: http://www.leif.org/FFT-SSN-70-125.png so the peaks are still there but not well resolved. Of course, if you have a very long series, the peaks will show up just fine as the third plot of 2562 ‘years’ of a 125-yr cycle.
I additionally added a 125-yr period: http://www.leif.org/research/FFT-SSN-70-125.png
This happens too often that I forget something on the long URL. I should make a really short one for this kind of stuff.
Willis Eschenbach says:
July 29, 2013 at 3:23 pm
“The most obvious difference is in the size of the peaks at around 52 years. Again, I suspect the result is because of the “scalpel” technique, but I have no way of demonstrating that.”
That looks indeed as if the BEST scalpel technique kills the low frequency periodicities. (Mosher’s defense “You’re wrong” doesn’t really cut it. I doubt he understands what he did.)
Does either of these methods point out cycles that are gradually changing in length ?
eg , in figure 7 you show 2 different major peaks in each half of the record (I don’t know which is which)
Suppose you were to use say 1/4 or 1/3 length records, or perhaps take a set length record and step it through the sequence, you may find a gradually increasing or decreasing period.
or even worse, a period length that is periodic.. 🙂
I will try it again.Just in passing, a naked FFT is a lousy tool for investigating noisy data. An experienced analyst would estimate a PSD.
Greg Goodman says:
July 29, 2013 at 2:01 pm
“Someone called Bart did a very interesting frequency analysis on SSN that comes up with several of the the same periods you found, and not to be impolite in any way, he does seem a lot more experienced than you with fourrier type techniques. “
That was I. The Sunspot data are the result of the rectification of primarily two processes with energy concentrated in frequency ranges centered at those associated with periods of about 20 and 23.6 years. When rectified, these produce the major observed peaks associated with about 10, 10.8, 11.8, and 131 years, in accordance with the Convolution Theorem.
I make the distinction of pointing out where the energy is concentrated because these are NOT periodic signals. If they were periodic, the energy would be distributed in lines, such as appear in atomic spectra. This type of behavior is fairly ubiquitous in a wide range of physical phenomena, owing to the fact that natural continuum processes can often be described by partial differential equations on a bounded domain, and such equations can often be solved as the expansion of a series of spatial eigenmodes associated with a time dependent amplitude function which is the output of a 2nd order ordinary differential equation. Dissipation of energy produces a broadening of the lines, and the processes manifest themselves as resonant phenomena driven by wideband random forcing.
I made such a model of the Sunspots here and showed how I could use it to generate qualitatively very similar outputs to the observed Sunspot behavior here and here. A Kalman Filter/Predictor could be formulated for this model which would produce optimal estimates of future behavior and associated error bars.
This kind of stuff is pretty old hat in control systems analysis and design. Someday, it will migrate over into the climate sciences.
“However, when the number of sunspots gets very large, the size of the black spots on the surface of the sun rises faster than the radiance, so the net radiance drops. Always more to learn”
Large black spots on the surface of the sun reduce total solar irradiance…you don’t say.
or even worse, a period length that is periodic.. 🙂
Like an FM radio signal of disco music?
For anyone interested, an Arxiv paper which sets out to rebut Abreu et al’s 2012 A&A paper ‘Is there a planetary influence on solar activity?’ is currently under discussion at the talkshop.
http://tallbloke.wordpress.com/2013/07/28/cameron-and-schussler-no-evidence-for-planetary-influence-on-solar-activity/
Abreu et al’s paper was favourably reviewed by Canadian Physicist Paul Charbonneau in Nature:
http://tallbloke.wordpress.com/2013/01/31/nature-print-edition-features-solar-planetary-theory/
He had previously written off the theory in 2003 but has since changed his mind in the light of strengthening evidence.
Stephen Rasey says:
or even worse, a period length that is periodic.. 🙂
Like an FM radio signal of disco music?
I guess what I’m saying.. why are we considering that solar periods stay constant.
They almost certainly don’t…….. UNLESS they are influenced by the motion of the big planets…
which Willis says there is no signal of.
The whole thing with Fourier and Periodic analysis is that they identify constant periods.
Question is, how does one go about identifying a periodic cycle, when the period is somewhat elastic.
Willis,
Are you comparing CRUTEM and Berkeley over the 1850-present period, or are you comparing 1750-present Berkeley to 1850-present CRUTEM? If the latter, I suspect that the early Berkeley record will skew the comparison a bit given the limited spatial coverage.
WRT TSI, does it include far UV effectively?
The issue isn’t the blackness, which I do not understand (Leif?) but the temperature of the penumbras of the black spots are getting cooler.
Why are sunspots black, rather than white?
Is it that the magnetic tubes open up the photo-surface with holes descending to the optically opaque regions?
Bart says:
July 29, 2013 at 4:07 pm
I’m sorry for the lack of clarity, Bart. What I meant was that a sawtooth wave of period 17 added to a sine wave of period 19 looks like this:
A Fourier analysis can decompose that combined waveform into a large number of superimposed sine waves.
What Fourier can’t do is to recover the sine wave and the sawtooth wave that created the waveform, while periodicity analysis can do that easily.
w.
Robert of Ottawa says:
July 29, 2013 at 6:03 pm
WRT TSI, does it include far UV effectively?
The ‘T’ is TSI means Total, so the answer is YES.
Robert of Ottawa says:
July 29, 2013 at 6:08 pm
Why are sunspots black, rather than white?
they are, in fact, blindingly white. Just look darker [black] against the even hotter background
Is it that the magnetic tubes open up the photo-surface with holes descending to the optically opaque regions?
No, the Sun gets hotter and brighter descending below the surface.
Zeke Hausfather says:
July 29, 2013 at 5:20 pm
Thanks, Zeke. The latter … here’s the more modern section of the CRU record
Regards,
w.
Willis:
Wonderful insightful research Willis. Sort of like finding the encyclopedia Britannica on one’s bookshelf; so much to learn.
Interesting insight about one cycle being close to Jupiters. Only, there is far too little knowledge to infer Jupiter’s orbit into the mix.
Consider the Earth’s ocean tidal cycle with two full high/low tides per day, and then some. Except, of course, for the Gulf of Mexico tides along America’s Gulf coast where there is approximately one high/low tidal cycle per day.
Proving beyond coincidence that a solar cycle matches Jupiter’s orbit needs actual correlation via research. Not something you are expected to do Willis, you’ve already opened portals to our minds.
One does wonder if some of the cycles you’ve identified might be reverberations from the sun’s nuclear furnace igniting into our sun.
Theoretically, Earth-Venus-Jupiter syzygy cycles should give peaks around 10.27 years and 11.86 years. On the short term they have to alternate between 6.5 and 7.5 Venus synodic periods (1.5987yrs).
Now people are actually starting to think about the physics that would be involved in astro cycles rather than endless curve fitting garbage and I congratulate you on getting that to happen Willis.
Photo looks a bit north of Santa Rosa, more like Windsor.
Hmm, so remind me again why inflections in the evolution of the WSO polar field strength data appear to centre fairly precisely on gas giant alignments or close pairs thereof? Yep, I think its coincidence (and therefore bullshit) too. But I guess we’ll have to wait and see what happens in 2015, 2017 and 2020 (a somewhat wider than normal separation and therefore a reasonable test).
One
Two
Three
Four
Sure looks like an n-body problem of some sort. The kind that’s going to produce fuzzy pseudo-cyclic behaviour that won’t cut up clean under Fourier’s surgical blade – hysteretic evolution of some underlying cycle attenuated by very small perturbations and a fair dose of randomness. No cycles precisely predictable over the longer term; Black Swans abound.
If there is a mechanism (which I seriously doubt) it’s tidal in nature. Jupiter is the driver whose influence is modified by the other three beyond; Sun too large and inner planets moving too fast to have any noticeable effect. Pseudoscience at its finest; needs maths. Precise, verifiable dates from 1580 to 2100 when gas giants cross(ed) the Sun’s equatorial plane gratefully received.
One thing I would like to ask is, are there really any cycles at all? Now when people analyse the temperature record, and say that it is warming/cooling/staying the same, it is sensible to ask whether that conclusion is statistically valid. Now some of the analysis, e.g Scafetta’s, quotes periods (of sine waves) with a precision of two decimal places – that is a precision of 3 days! I am absolutely certain that the accuracy of the underlying climate data is not sufficient to justify this degree of precision, and that it is just meaningless curve fitting. But I am wondering about the “cycle” everyone takes for granted namely the solar cycle, with “period” of about 11 years. Except that it is really somewhere between 8 and about 14 years with 11 only being the average. If that is the case why would one expect to see a “period” of 11 years in the climate? There is a real physical phenomenon in the sun – the reversal of the magnetic dipole – but why is this considered a “cycle” rather than a sequence of irregularly spaced independent events? To what extent is it statistically significant to represent this sequence by a single periodic function of length 11 years?
AJB says:
July 29, 2013 at 8:14 pm
Since you’re the first person on the thread to mention the “WSO polar field strength data”, I can’t remind you AGAIN of a dang thing. It’s not clear at all what your point is.
w.
jimmi_the_dalek says:
July 29, 2013 at 8:27 pm
There is a real physical phenomenon in the sun – the reversal of the magnetic dipole – but why is this considered a “cycle” rather than a sequence of irregularly spaced independent events?
The dalek makes a good point. The solar ‘cycle’ is, in fact, not a cycle, but a sequence of eruptions of magnetic flux which when they have run their cause leave behind a semi-random polar field as a seed for the next series of eruptions and so it goes. The Sun is not an ‘oscillator’ that runs a precise cycle.
jimmi_the_dalek says:
July 29, 2013 at 8:27 pm
There is a real physical phenomenon in the sun – the reversal of the magnetic dipole – but why is this considered a “cycle” rather than a sequence of irregularly spaced independent events?
I agree with Leif and your point it most likely is not a cycle.
Old faithful in Yellowstone park is probably closer to what is actually happening
(http://en.wikipedia.org/wiki/Old_Faithful)
Only lately it has become unfaithful but yet …
=> Over the years, the length of the interval has increased, which may be the result of earthquakes affecting subterranean water levels. These disruptions have made the earlier mathematical relationship inaccurate, but have in fact made Old Faithful more predictable. With a margin of error of 10 minutes, Old Faithful will erupt 65 minutes after an eruption lasting less than 2.5 minutes or 91 minutes after an eruption lasting more than 2.5 minutes. The reliability of Old Faithful can be attributed to the fact that it is not connected to any other thermal features of the Upper Geyser Basin.
There is an example of a thermal explosion cycle which has no link to anything other than how the pressure builds up in the basin itself.
I suspect that the Sun does not have a timing belt.
OK I worded that badly Willis. The point is that as Leif says, the Sun is not an ‘oscillator’ that runs a precise cycle. However, IMO there is (very scant!) evidence in the WSO data that the gas giants may attenuate/modify its evolution slightly giving rise to fuzzy cyclic behaviour close to synodic periods, etc. Sorry to be obtuse – cycles without the mania is pretty much spot on.
Going on from Lief’s reply above : If the earth’s climate is sensitive to minor variations in the sun’s output, and if the sun spot record is a good proxy for this, then maybe the record can be used a different way. The sun spot record records a sequence of events with varying magnitude and irregular spacing going back at least 250 years, in effect defining a wave packet. So why not use this wave packet and look for its signature, i.e. the 250 year long irregular shape, in the climate instead of looking for periodic approximations to this data. If that pattern can be found, possible with a bit of a time lag, then well and good, but if not then this whole “cycle” thing is probably BS from beginning to end.
Willis, As I mentioned before.
Take a 1/3 length of the record find the peak periodicy, then step that 1/3 length along the whole record, see if there is any pattern in the period change.
Actually thinking about “Old faithful” what would be interesting would be to ask one of the cycle maniacs what could be causing our 91 min eruption does it have a planetary cause?
Willis,
Thanks for that. Looking at them side-by-side they look much more similar now, though the slightly different y and x axis make it a tad difficult to eyeball. Not to put more work on your shoulders, but if you could put them side by side on the same graph (with different color), it would be greatly appreciated.
AndyG55 says:
July 29, 2013 at 8:53 pm
I suspect that the Sun does not have a timing belt.
###
Maybe its just a bit loose …
This puzzles me……
The TSI Monitor seems to be a heat sensor http://earthobservatory.nasa.gov/Features/SORCE/sorce_07.php
Wouldn’t it favour IR radiation ?
“Advantage: Computationally reasonably fast.”
Come on, periodicity analysis is no match to fourier analysis in terms of efficiency so you can’t put it as advantage when comparing the two. And efficiency of algorithm is not measured in “seconds on my computer”. Periodicity analysis runs in O(n^2) at best while FFT is O(n log n). Clearly FFT has an advantage here.
Keith Minto says:
July 29, 2013 at 10:36 pm
The TSI Monitor seems to be a heat sensor … Wouldn’t it favour IR radiation ?
No, TSI measures all there is. It does that by basically letting plain sunlight of all wavelengths fall on a black surface and measuring the resulting heating. IR is about half of the incoming radiation.
The PT is a great tool to add to box. It provides quite a bit of insight into the structure of the data being analyzed, with little effort, but like the FT it is most useful when applied appropriately. I suspect that unstable higher frequency components will get washed out at higher data lengths.
The 10 year cycle is actually 9,93yrs, half the synodic (i.e. the tidally effective period) of the two largest bodies in the solar system outside the Sun itself. Fig7 doesn’t find these periods in the two halves of the sunspot data because the cycle lengths tends to cluster at around 10.38 (VEJ) or 11,86 (J) and the longer term cycle of this bi-polar behaviour won”t be captured by half the sunspot data length. The 11.07 average is rarely the actual cycle length.
tallbloke says:
July 29, 2013 at 10:56 pm
The 10 year cycle is actually 9.93yrs, half the synodic (i.e. the tidally effective period) of the two largest bodies in the solar system outside the Sun itself.
In spite of Willis’s calling the the topic of his thread ‘cycles without the mania’, it seems that the manics have wormed their way in after all.
MiCro says:
July 29, 2013 at 2:46 pm
When you homogenize and average the data into global averages of almost any kind, you loss so much.
============
The process of reducing the data to anomalies further degrades the fidelity of the signal. Searching for cycles in the processed data is highly likely to lead to spurious results unless the processing has been shown to not introduce artifacts. Has any such certification been performed for any global temperature signal?
Off the stats discussion slightly, but this information of TSI movement through the atmosphere is interesting…..
“So if the sunspots (or anything associated with them, TSI or cosmic rays) has a significant effect on global temperatures, we would see it in the satellite temperatures.”
I can clearly see that temperatures go the cooler the higher the sunspot number is. With all other effects affecting the Earth temperature, it may be even significant. And that’s just the first order effect, without delays or without analysing rates of change.
Leif Svalgaard says:
July 29, 2013 at 11:05 pm
tallbloke says:
July 29, 2013 at 10:56 pm
The 10 year cycle is actually 9.93yrs, half the synodic (i.e. the tidally effective period) of the two largest bodies in the solar system outside the Sun itself.
In spite of Willis’s calling the the topic of his thread ‘cycles without the mania’, it seems that the manics have wormed their way in after all.
==================
may be coincidence, may not. neither possibility can be fully ruled out. thus the mania exists at both extremes. CAGW is the effect of mania. The belief that there is only one possible cause for observed events, to the exclusion of all others. Such a position is preposterous in the absence of absolute knowledge. No being with limited knowledge and a relative point of view can hope to lay claim to absolute truth.
Cause is the realm of philosophy. It is open to endless debate. If you can predict with accuracy, you are practicing science. If your prediction more closely matches the observations that alternative predictions, the debate is moot. regardless of cause, the truth is that your method is demonstrably more accurate.
Leif Svalgaard says:
July 29, 2013 at 8:34 pm
The solar ‘cycle’ is, in fact, not a cycle, but a sequence of eruptions of magnetic flux which when they have run their cause leave behind a semi-random polar field as a seed for the next series of eruptions and so it goes. The Sun is not an ‘oscillator’ that runs a precise cycle.
Correct, it’s an oscillator near boundary conditions that runs to an imprecise cycle. One that never goes completely out of phase with the cycle of conjunctions of the most tidally effective planets revolving around it. In fact it runs closely in phase with those conjunction cycles except for when it goes into one of it’s bicentennial sulks, like now.
Kasuha: And efficiency of algorithm is not measured in “seconds on my computer”. Periodicity analysis runs in O(n^2) at best while FFT is O(n log n). Clearly FFT has an advantage here.
What matters is the actual elapsed time in the data sets of interest. For some problems, this theoretical advantage of FFTs is trivial. It certainly does not outweigh the problems that arise with FFTs performed on irregularly spaced data of arbitrary length. Each year of extra climate data changes the frequencies estimated by the routine use of FFTs, even though the problem is the same problem. As computing machinery becomes faster and faster, the speed advantage of the FFT is restricted to longer and longer time series.
Willis: A Fourier analysis can decompose that combined waveform into a large number of superimposed sine waves.
What Fourier can’t do is to recover the sine wave and the sawtooth wave that created the waveform, while periodicity analysis can do that easily.
Just so. The Fourier analysis returns an extremely non-parsimonious representation of any signal that is not a pure summation of sine waves. A sawtooth or stair-step signal will be exactly represented by a Fourier analysis, but you would hardly guess from the large collection of non-zero coefficients that the function had a simpler representation as a saw tooth or stair step. A periodic step function (such as the high secretion rate/low secretion rate of melatonin secretion) can be simply represented by a 5 parameter function, but the Fourier representation has about 2 dozen non-zero coefficients. A PubMed search on my name will produce an example in the analysis of circadian rhythms of activity, in the Journal Statistics in Medicine. Emery Brown of Harvard has used a similar technique to model the circadian rhythm in melatonin secretion and other processes.
This is not a blanket critique of Fourier analysis or trigonometric polynomial regression (aka harmonic regression). It’s just that in some circumstances, and this may be one, there are advantages to the method that Willis has presented here.
Fred Berple says ” Cause is the realm of philosophy. It is open to endless debate. If you can predict with accuracy, you are practicing science.”
So with that in mind, I am going to ask Tallbloke how the solar cycle caused the orbit of Jupiter to be what it is?
tallbloke says:
July 29, 2013 at 11:41 pm
Correct, it’s an oscillator near boundary conditions that runs to an imprecise cycle.
It is not an oscillator at all. The solar ‘cycle’ has two parts: it convert poloidal field into toroidal field [creating sunspots] in the beginning of the cycle. The spots decay and the debris are moved to the poles by a random process to build up a new poloidal field, which eventually is advected into the sun to serve as a seed for the following ‘cycle’. The first part is rather deterministic, the second part very random. The two parts are different physical processes and not part of unified, single cycle.
One that never goes completely out of phase
Any cycle whatsoever will at various times be in phase with the sunspot ‘cycle’ [like a stopped clock being correct twice a day]. when your tidal cycle [running at a different rate] is 5 years off with respect to the sun it is ‘completely out of phase’. Then, as it goes even more and more out of phase it catches up with the sun and you hail that as success while it actually is a growing failure. As I said, the cyclomaniacs has polluted this nice thread [as they always do].
Kasuha says:
July 29, 2013 at 11:21 pm
“So if the sunspots (or anything associated with them, TSI or cosmic rays) has a significant effect on global temperatures, we would see it in the satellite temperatures.”
I can clearly see that temperatures go the cooler the higher the sunspot number is. With all other effects affecting the Earth temperature, it may be even significant. And that’s just the first order effect, without delays or without analysing rates of change.
There are roughly speaking, three el ninos per solar cycle. the big one occurs soon after solar minimum when the ocean goes into reverse and kicks heat out instead of absorbing it, unless there was a volcanic eruption in the previous cycle, in which case the PWP is already partially discharged. Consequently the following big la nina usually occurs near solar max. That’s why there’s often a dip in global T near solar max.
If you smooth the temperature data at 1/3 solar cycle or at the average frequency of the ENSO cycle (around 40 months) you get a good correlation between solar activity and global temperature. The amplitude isn’t that big, but this is due to the antiphase nature of ENSO surface temps and solar cycle described above. The Sun is having a large effect, but it’s hidden below the surface, most obviously in the Pacific Warm Pool.
http://woodfortrees.org/plot/hadsst3sh/from:1955/mean:37/detrend:0.6/plot/sidc-ssn/from:1955/scale:0.001/offset:-0.3/mean:12
Anyway, Leif and Willis have more time to spend around here convincing you the planets don’t affect the Sun and the Sun doesn’t affect climate than I have to convincing you they do and it does, so I’ll return you to your normal programming.
Do not adjust your mindset.
jimmi_the_dalek says:
July 29, 2013 at 11:53 pm
Fred Berple says ” Cause is the realm of philosophy. It is open to endless debate. If you can predict with accuracy, you are practicing science.”
So with that in mind, I am going to ask Tallbloke how the solar cycle caused the orbit of Jupiter to be what it is?
Intelligent question. By interacting with it in with the corpuscular force and the electromagnetic force and the gravitational force. All systems exhibiting cybernetic feedback have periodicities which oscillate around the mean (Think Watt Governor).
Eventually they settle into patterns which cause least perturbation (principle of least action and entropy) and minimum interference (lognormal distribution). That’s why all the planetary periodicities and the periodicity of the solar cycles fit the only lognormal distribution which reconciles linear and rotational motions (the fibonacci series)
http://tallbloke.wordpress.com/2013/02/20/a-remarkable-discovery-all-solar-system-periods-fit-the-fibonacci-series-and-the-golden-ratio/
Hmm. If the planets *do* affect the sun (and I have no opinion either way), what might mediate the effect?
http://tallbloke.wordpress.com/2011/08/05/jackpot-jupiter-and-saturn-solar-cycle-link-confirmed/
tallbloke says:
July 30, 2013 at 12:14 am
That’s why all the planetary periodicities and the periodicity of the solar cycles fit the only lognormal distribution which reconciles linear and rotational motions (the fibonacci series)
Since the Fibonacci series is universal this would imply that all planetary systems around any star whatsoever must follow that distribution, which we already now know that they don’t. Stellar planetary systems vary enormously and no two alike are known [although they should all be alike].
Leif Svalgaard says:
July 30, 2013 at 12:00 am
Any cycle whatsoever will at various times be in phase with the sunspot ‘cycle’ [like a stopped clock being correct twice a day]. when your tidal cycle [running at a different rate] is 5 years off with respect to the sun it is ‘completely out of phase’. Then, as it goes even more and more out of phase it catches up with the sun
When I said it never goes completely out of phase, I meant exactly what I said. Not that it ‘runs past and catches up again’. Both the Solar cycle and the conjunction cycle of Venus-Earth Jupiter (the most tidally effective planets in the system) vary in length. The fact they never go completely out of phase is due to the fact the Sun and the planets are part of the same system. The Solar system. The word system necessarily implies cybernetic feedback. The Solar system is a true system.
http://tallbloke.files.wordpress.com/2010/08/rotation-solar-windspeed-adjusted.png
tallbloke says:
July 30, 2013 at 12:30 am
When I said it never goes completely out of phase, I meant exactly what I said.
could be [but that does not make it right]. Right now and for the several years the two ‘cycles’ are going completely out of phase [as per your graph]….
Michael Larkin says:
July 30, 2013 at 12:16 am
Hmm. If the planets *do* affect the sun (and I have no opinion either way), what might mediate the effect?
The interplanetary magnetic field and the solar wind and the gravitational tidal force on the Sun are the most prominent candidates. Even Leif acknowledges that the planets and sun were in a spin-orbit coupling during the formation and early history of the solar system. The force has diminished greatly since, but never gone to zero. In a finely balanced system near boundary conditions, it doesn’t take much force to produce a cyclic solar variation of 0.1%
Check out Kelvin-Helmholtz and Raylaigh-Taylor instability
Leif Svalgaard says:
July 30, 2013 at 12:38 am
tallbloke says:
July 30, 2013 at 12:30 am
When I said it never goes completely out of phase, I meant exactly what I said.
could be [but that does not make it right]. Right now and for the several years the two ‘cycles’ are going completely out of phase [as per your graph]….
Remains to be seen. Not been many visible spots (as opposed to the imaginary ones SIDC count in SS24) over the last month at ‘solar max’ has there?, but last time the phase got this near breaking completely we had the Dalton minimum. Bit of a ‘coincidence’ eh?
Leif Svalgaard says:
July 30, 2013 at 12:29 am
Since the Fibonacci series is universal this would imply that all planetary systems around any star whatsoever must follow that distribution, which we already now know that they don’t. Stellar planetary systems vary enormously and no two alike are known [although they should all be alike].
So far we’ve found data from two exoplanetary systems around other stars which fit the pattern. Onging research. Got any good data for us where the periodicities of at least three planets are well enough resolved?
tallbloke says:
July 30, 2013 at 12:45 am
Not been many visible spots (as opposed to the imaginary ones SIDC count in SS24)
When your theory doesn’t work, blame the data [is the usual excuse]
but last time the phase got this near breaking completely we had the Dalton minimum. Bit of a ‘coincidence’ eh?
Make the coincidence even more striking by plotting all the data [back to 1610].
Anyway, there are circumstances in which the pattern would be broken around other stars if the planets were large and close, due to mutual perturbation of the planets. These are the system most readily observable at sufficient resolution at the moment. It may take time before we have the instrumentation to spot more systems with wider spread smaller planets.
tallbloke says:
July 30, 2013 at 12:48 am
So far we’ve found data from two exoplanetary systems around other stars which fit the pattern.
Which ones?
Got any good data for us where the periodicities of at least three planets are well enough resolved?
Slide 18 of http://www.leif.org/research/AGU%20Fall%202011%20SH34B-08.pdf shows a bunch.
Check publications by Poppenhäger…
tallbloke says:
July 30, 2013 at 12:54 am
if the planets were large and close, due to mutual perturbation of the planets.
Then the planetary effects would be millions of times large and easy to observe. None seen so far.
Leif Svalgaard says:
July 30, 2013 at 12:50 am
Make the coincidence even more striking by plotting all the data [back to 1610].
As you keep telling us about Penn and Livingstones theory, just because the spots become invisible doesn’t mean there isn’t still a strong magnetic cycle occurring. Auroral records show it too. So the phasing of the VEJ cycles and auroral records should hold back to 1610, because sunspots are a visible symptom, not he causative agency. I’m working on other stuff at the moment, but if I get around to it, I’ll try out the dataset back to 1610 of your choice.
Anthony published another piece of junk written by Willis.
It is evident to me that Anthony and Willis are behaving quite dishonestly by trying to defame my research.
I do not know why Anthony is behaving in this way, but I found his behavior highly unprofessional and misleading. Evidently my research is “disturbing” somebody who is benefitting of the current chaos.
My research is moving toward the solution of the problem, and I believe that a lot of people from both side of the debate prefer the actual scientific confusion.
About the new post by Willis, I do not have time to rebut it in details. Just two points that demonstrate how Willis is incompetent on these topics.
The truth is that Willis is trying to criticize my papers, and many times I have denounced that he is not even reading my papers. He is just jumping around without trying to understand thing. But Anthony does not get it.
1)
Let us start from the easy thing.
Look at Willis Figure 9. He claims that it represents the ACRIM composite. However, it is the PMOD composite.
It is easy to figure it out. See here the two composites
http://acrim.com/TSI%20Monitoring.htm
Not only the shape is different, but ACRIM averages around 1361 W/m^2, while PMOD averages around the outdated value of 1366 W/m^2 as in Willis graph.
Thus, it is evident that or Willis does not know much about these data or he is misleading people.
Curiously, Leif did not note the error too.
2)
Now, note his Figure 5. He fully confirmed my result that the 11-year sunspot record is made of three peaks close to 10, 11 and 11.86 year.
He comments, “Now, before anyone points out that 11 years 10 months is the orbital period of Jupiter, yes, it is. But then ten years, and eleven years, the other two peaks, are not the orbital period of anything I know of … so that may or may not be a coincidence. In any case, it doesn’t matter whether the 11 years 10 months is Jupiter or not, any more than it matters if 10 years is the orbital period of something else. Those are just the frequencies involved to the nearest month.”
So, he acknowledges that the 11 years 10 months is the orbital period of Jupiter. However, later he claims that he does not know about the planetary origin of the quasi 10-year cycle. However, as clearly stated in my papers many times, that is the spring tidal period between Jupiter and Saturn.
Also the major cycle at 11 year, can be found as combination of the tides from Venus, Earth and Jupiter.
So, Willis has not read my papers, but he criticized them!
His additional critique that by using short periods “the strength of these cycles waxes and wanes over time”
is not a mystery. The cycles waxes and wanes simply because they are interfering. Moreover, using short periods the error associated to the frequency evaluation increases and it becomes more difficult to separate the frequencies.
In general, to separate close sequences at 10, 11 and 12 years, I need to see at least two beats, which implies that I need to use more than 200 years of data. Because the sunspot record is 262 years long, if I divide it in half, as Willis does, I get 131 year long sequences which are too short to separate the cycles with periodogram like techniques.
The right way to proceed is how I did in my paper:
Scafetta N., 2012. Multi-scale harmonic model for solar and climate cyclical variation throughout the Holocene based on Jupiter-Saturn tidal frequencies plus the 11-year solar dynamo cycle. Journal of Atmospheric and Solar-Terrestrial Physics 80, 296-311.
http://people.duke.edu/~ns2002/pdf/ATP3581.pdf
where I build a model based of Jupiter and Saturn two tides and I see how the model performs in hindcasting past solar pattern before 1750. The hindcast tests are fully missed my Willis, but it is these tests that support the interpretation of the cycles.
***
So, tell me, how can Anthony and Willis be trusted further? They are really doing some dirty game to defame my research or they are simply incompetent.
Very likely by listening Willis and Leif, Anthony is making suicide. Somebody needs to inform Anthony before it is too late for him.
To the readers of this blog I say that only by reading my papers one may understand what I write.
**********************************************
By the way
Just today a new paper was published showing strong evidences supporting
the planetary theory of solar variation.
Despite the defamation attempts of Anthony and Willis who simply are trying
to deal with something bigger than themselves, I continue to publish on the
topic. This thing is getting big.
Scafetta N, Willson R.C. (2013). Empirical evidences for a planetary
modulation of total solar irradiance and the TSI signature of the 1.09-year
Earth-Jupiter conjunction cycle. Astrophysics and Space Science. DOI:
10.1007/s10509-013-1558-3
http://link.springer.com/article/10.1007%2Fs10509-013-1558-3
Abstract
The time series of total solar irradiance (TSI) satellite observations since
1978 provided by ACRIM and PMOD TSI composites are studied. We find
empirical evidence for planetary-induced forcing and modulation of solar
activity. Power spectra and direct data pattern analysis reveal a clear
signature of the 1.09-year Earth-Jupiter conjunction cycle, in particular
during solar cycle 23 maximum. This appears to suggest that the Jupiter side
of the Sun is slightly brighter during solar maxima. The effect is observed
when the Earth crosses the Sun-Jupiter conjunction line every 1.09 years.
Multiple spectral peaks are observed in the TSI records that are coherent
with known planetary harmonics such as the spring, orbital and synodic
periods among Mercury, Venus, Earth and Jupiter: the Mercury-Venus
spring-tidal cycle (0.20 year); the Mercury orbital cycle (0.24 year); the
Venus-Jupiter spring-tidal cycle (0.32 year); the Venus-Mercury synodic
cycle (0.40 year); the Venus-Jupiter synodic cycle (0.65 year); and the
Venus-Earth spring tidal cycle (0.80 year). Strong evidence is also found
for a 0.5-year TSI cycle that could be driven by the Earth’s crossing the
solar equatorial plane twice a year and may indicate a latitudinal
solar-luminosity asymmetry. Because both spring and synodic planetary cycles
appear to be present and the amplitudes of their TSI signatures appear
enhanced during sunspot cycle maxima, we conjecture that on annual and
sub-annual scales both gravitational and electro-magnetic planet-sun
interactions and internal non-linear feedbacks may be modulating solar
activity. Gravitational tidal forces should mostly stress spring cycles
while electro-magnetic forces could be linked to the solar wobbling
dynamics, and would mostly stress the synodic cycles. The observed
statistical coherence between the TSI records and the planetary harmonics is
confirmed by three alternative tests.
See my web-site for the pdf file
http://people.duke.edu/~ns2002/pdf/10.1007_s10509-013-1558-3.pdf
Leif Svalgaard says:
July 30, 2013 at 12:58 am
if the planets were large and close, due to mutual perturbation of the planets.
Then the planetary effects would be millions of times large and easy to observe. None seen so far.
In fact you’re wrong about that. You should keep up with the literature.
I’m putting up a post in a couple of hours about just that. NASA have just published. Get some sleep and then potter over to the talkshop. 🙂
tallbloke says:
July 30, 2013 at 1:00 am
As you keep telling us about Penn and Livingstones theory, just because the spots become invisible doesn’t mean there isn’t still a strong magnetic cycle occurring.
We know max and min for those cycle from cosmic ray data. so you cannot hide behind L&P.
but if I get around to it, I’ll try out the dataset back to 1610 of your choice.
Try the group sunspot number augmented with SIDC after 1994. Although the amplitudes before 1885 are wrong, the times of max and min are OK.
Leif Svalgaard says:
July 30, 2013 at 12:54 am
tallbloke says:
July 30, 2013 at 12:48 am
So far we’ve found data from two exoplanetary systems around other stars which fit the pattern.
Which ones?
All in good time Leif.
Got any good data for us where the periodicities of at least three planets are well enough resolved?
Slide 18 of http://www.leif.org/research/AGU%20Fall%202011%20SH34B-08.pdf shows a bunch.
Check publications by Poppenhäger…
Thanks, you are on the case after all. Maybe you just didn’t spot the implications yet.
tallbloke says:
July 30, 2013 at 1:06 am
I’m putting up a post in a couple of hours about just that. NASA have just published
so you are misunderstanding yet another paper…
Leif Svalgaard says:
July 30, 2013 at 1:07 am
tallbloke says:
July 30, 2013 at 1:00 am
As you keep telling us about Penn and Livingstones theory, just because the spots become invisible doesn’t mean there isn’t still a strong magnetic cycle occurring.
We know max and min for those cycle from cosmic ray data. so you cannot hide behind L&P.
Who’s trying to hide? We’re on a roll!
As for the bolded if</i, tell you what, you do it, so you can be sure I haven’t fiddled it.
I have bigger fish to fry at the moment.
tallbloke says:
July 30, 2013 at 1:12 am
I have bigger fish to fry at the moment.
“put up or shut up”
Leif Svalgaard says:
July 30, 2013 at 1:09 am
tallbloke says:
July 30, 2013 at 1:06 am
I’m putting up a post in a couple of hours about just that. NASA have just published
so you are misunderstanding yet another paper…
And you ‘know’ this, even before reading what I have to say.
That’s known as ‘Blind prejudice’.
Get some sleep.
🙂
Leif Svalgaard says:
July 30, 2013 at 1:14 am
tallbloke says:
July 30, 2013 at 1:12 am
I have bigger fish to fry at the moment.
“put up or shut up”
I already did. the data from 1840 is more than enough to prove the point. You are the one wanting a plot from 150 years before the advent of reliable sunspot counts. You “put up or shut up”
http://tallbloke.files.wordpress.com/2010/08/rotation-solar-windspeed-adjusted.png
🙂
There’s a guy Nicola Scafetta whose work doesn’t need a lot of explanation: those lines hitting
time after
time after
boring time,
explain everything you need to know about how ‘mystical’ this all is. It’s about as ‘mystical’ as at the 20:00 mark when he says ‘this..Black…Line’ and like all real mathematics that has a point, everything lines up without any questions.
He posted here and I went to his site because I saw his numbers weren’t fake on his paper. I took the dart board approach and found his presentation in English called
“Solar Activity and Climate – Nicola Scafetta, ACRIM & Duke University
from Kavli Frontiers of Science Plus 1 year ago /
Creative Commons License:
Solar Activity and Climate
Nicola Scafetta,
ACRIM & Duke University
The Cimate Oscillations: Analysis, Implications And Their Astronomical Origin.”
He makes his projections line up like it’s really supposed to happen when properly weighted physical effects are properly calculated. It’s very simple, and on being told – especially when you see him project out for you the various temperature movements in earth – it’s very clearly intuitive.
The sun’s internal is fluid. It’s plasma or whatever but the mechanics are those of fluids and when the combined gravitational weights of different entities around the solar system align and disalign, the internal gravitational center of the sun, moves within it. You can’t see it, but the classical effects of a gravity driven fusion furnace operate just like they always did and will and are.
It’s exTREMEly accurate and Scafetta’s simply not to be compared to most anyone else in his climatic framework, attributive parameters.
He can’t speak English well and for ‘oscillation’ he says ‘isolation’. For induced he says in-dew-said’ and toward the end when he says ‘heliosphere’ he says ‘AY-lee-ohs-fear’
If you’re ADHD and are tired of watching weather/climate projection scams cut to 20:00 and watch where he says “this..black…line” and if you wonder whether he can develop this in a way you can understand, – he can. Go back to the start and let him develop his story and it’s plain, unadorned: he decided to check up on the claims of Magic Gassers. He found what they said made no sense so started looking around and found some people claiming what he himself later mathematically isolated.
Fundamentally there are 1000, 200, *60* hence 30, and 11 hence 22 year, and also a 9/18 year cycle attributable to gravitational presentation of the moon.
This is all going to be extremely compact and really discounting his English, a nearly flawless presentation. You will be seeing after 28 minutes that anyone who claims they think there’s some big incalculable mystery, just hasn’t paid close enough attention to the REAL people who understand the climate being driven by sun-derived energy pools washing outward from that sun, and that energy influenced by: surprise, the only other really very significant element in the solar system: those gravitationally relevant planetary masses, lining up and spreading out, around the sun.
If you really wonder if it’s really all that complicated, you need to see Italian Nicola Scafetta, have just 28 minutes of your life. I promise you, you’re just not going to ask for that 28 minutes back: in fact you’re prolly gonna sit there another 10 asking yourself why it is, things like this have to sit on a back burner for years, while people try to proclaim they discovered a giant magic infrared light on in the sky overhead, influencing climate but too small to measure: a heating component supposedly derived from the refrigerant in a frigid, refrigerated gas bath, a warm rock’s spinning at the bottom of.
Nicola Scafetta has some answers for you. Once you see those answers you’ll realize just what ‘knowing what he’s talking about’ looks and feels like: instead of having to peer into hypothesis which are constantly admitting and being seen not able, to project which way an instrument is going to point.
Enjoy because I guarantee you, nobody else around the climate field is going to want a competent mathematician so, it’s not like this is going to get picked up and pasted everywhere; although it really should be.
iframe=true&width=80%25&height=80%25
tallbloke says:
July 30, 2013 at 1:16 am
And you ‘know’ this, even before reading what I have to say.
Judging from your previous ‘performance’…
Willis Eschenbach says:
“The vagaries of the sun, whatever else they might be doing, and whatever they might be related to, do not seem to affect the global surface temperature or the global lower atmospheric temperature in any meaningful way.”
The ~ 11 year solar cycle does not have a particularly blatant or large no-lag temperature effect at its peak or trough. Weather noise and influences like the ENSO mix into likely the “governed, lagged system” aspects you illustrated in another context in your May 25th 2013 volcano article here.* However, more broadly, actually solar/GCR and climate correlation is very blatant in the following, a recently much expanded version of what I’ve posted a few times before, looking directly at appropriate raw data, with source references added:
http://s18.postimg.org/l3973i6hk/moreadded.jpg
(Image enlarges on click).
John Trigge (in Oz) says:
July 29, 2013 at 2:37 pm
“highly adjusted, homogenised, UHI’d, in-filled, smoothed and distorted temperature ‘data’?”
See the above image link for what happens when looking at some of the right raw data ;), what basically nobody even on WUWT ever has seen.
——————–
*
http://wattsupwiththat.com/2013/05/25/stacked-volcanoes-falsify-models/
tallbloke says:
July 30, 2013 at 1:18 am
I already did. the data from 1840 is more than enough to prove the point.
That the match has broken down…
You are the one wanting a plot from 150 years before the advent of reliable sunspot counts
The existing record is good enough to show maxima and minima. You are just using the ‘bad data’ excuse again.
Get some sleep Leif.
tallbloke says:
July 30, 2013 at 1:26 am
Get some sleep Leif.
Hey, I’m on a roll!
Willis Eschenbach said:
“The vagaries of the sun, whatever else they might be doing, and whatever they might be related to, do not seem to affect the global surface temperature or the global lower atmospheric temperature in any meaningful way.”
The ~ 11 year solar cycle does not have a particularly blatant or large no-lag temperature effect at its peak or trough. Weather noise and influences like the ENSO mix into likely the “governed, lagged system” aspects you illustrated in another context in your May 25th 2013 volcano article here.* However, more broadly, actually solar/GCR and climate correlation is very blatant in the following, a recently much expanded version of what I’ve posted a few times before, looking directly at appropriate raw data, with source references added:
http://s18.postimg.org/l3973i6hk/moreadded.jpg
(Image enlarges on click).
John Trigge (in Oz) says:
July 29, 2013 at 2:37 pm
“highly adjusted, homogenised, UHI’d, in-filled, smoothed and distorted temperature ‘data’?”
See the above image link for what happens when looking at some of the right raw data ;), what basically nobody even on WUWT ever has seen.
——————–
*
http://wattsupwiththat.com/2013/05/25/stacked-volcanoes-falsify-models/
Thanks for you response, Tallbloke.
C’mon Willis, is it really worth continuing to say things that are wrong to try and make a point?
The Fourier transform has the advantage of being a linear operator, which allows me to exercise a neat trick:
FT(sine+sawtooth) = FT(sine) + FT(sawtooth)
You already know what FT(sine) is, right? And FT(sawtooth) is a simple harmonic series, which is readily identifiable to someone familiar with spectral analysis.
As for Matthew’s complaint that the harmonic series is a non-parsimonious representation of a simple function, well, Taylor series are also a non-parsimonious representation of a simple function. Yet in the right hands, the Taylor series are immediately recognisable and very powerful tools. Likewise harmonic series. But this is true of *any* spectrum analysis method, including the periodicity transform. Because the information in the data is the same.
Leif, firstly your Fourier transform plots are terrible. If you want more information, zero pad your dataset prior to performing the transform.
Secondly, as Bart notes, things like sun spots do not have strict period, but in practice are “noisy” cycles. It would be more appropriate to analyse them with a tool that estimate power spectral density (PSD), like Welch’s method, or the climacogram. The “cycles” you see around the peak are only cycles from a single realisation of the time series derived from the PSD function. They are not “real” or special beyond what the PSD function describes.
The PSD itself seems to be a spread peak centred around 11 years but then at lower frequencies follows a 1/f type noise pattern, which is what I would expect for a naturally varying complex system.
Willis: Regarding the “TSI vs. sunspot number” correlation: Is there a difference between “rising flank” and “falling flank” in the solar cycle? Just a thought.
Spence_UK says:
July 30, 2013 at 1:44 am
If you want more information, zero pad your dataset prior to performing the transform.
I always do, but I don’t think you can create information where none exists.
Secondly, as Bart notes, things like sun spots do not have strict period, but in practice are “noisy” cycles.
Agreed, but the planetary cycles are much more periodic and that is what people claim to see [e.g. Scafetta, Tallbloke a few comments above]
They are not “real” or special beyond what the PSD function describes.
Yet are touted as real by enthusiasts.
Anyway, a generally accepted and often used frequency analysis is the Lomb-Scargle Periodogram using the Peranso period analysis software (http://www.peranso.com/). Here is a recent paper by Nick Lomb http://www.leif.org/EOS/Lomb-Sunspot-Cycle-Revisited.pdf showing that the side peaks of the 11-yr peak are real and indicate modulation by a long cycle [102 years] just as my crude FFT analyses showed. He states: “the period around 100 years remains with the modulation by this period obvious in a visual examination”. We disagree whether these periods are truly stable and indicate a deep-seated internal clock, but that is a discussion going back many decades and may continue for some more to come.
One more thing Willis:
When doing Fourier analysis, the form of the window is important as well. *Any* window type you choose will introduce errors, as you multiply the signal with the window, the Fourier transform will be an combination of signal and window. Choosing what window to use is an science in itself, and needs care and attention.
And just in case this wasn’t clear: Using “no window” is equal to using an rectangular window, which is usually the worst choice of window for Fourier transforms.
@ Spence and Leif, regarding the “Zero Padding” discussion:
Zero padding does not create any information, but it allows you to run a longer FFT, to get a resulting Fourier spectrum with more bins (finer resolution) – this is akin to interpolating in-between values, but should be better than interpolation.
One application of zero padding is if you want to visually determine the location of a peak in a spectrum. E.g. when the peak lies “between” bins, and 1 or 2 neighbouring bins are almost as high, and you want resolution of higher than 1.0 bins. Or if you want to visually determine the location of peak of a short period signal and are therefore in a region where a unpadded Fourier transformation has relative low resolution. If on the other hand you use a function to estimate the locations of peaks in your spectrum, you don’t need to zero pad.
@Willis:
“What Fourier can’t do is to recover the sine wave and the sawtooth wave that created the waveform, while periodicity analysis can do that easily.”
Have you tried that with other window types besides rectangular windows?
What FFT does with a rectangular window, is “loop back” the signal. If the ratio of “period of the signal(s) under analysis” and “total signal length” is not a natural number, then the rectangular window will introduce discontinuity. You use non-rectangular windows to get rid of this discontinuity – but these other window types introduce other errors as a trade off.
By the way, if you want to do Signal -> Fourier -> Signal transforms: Don’t throw away the phase information from the Fourier analysis.
And regarding “Again, I’m not anti-Fourier. It’s just that sometimes instead of a saw I want a chisel …”
Willis, what you have basically done is picked up a saw, ignored the saw’s manual, used a wood blade to cut stone, proceeded to cut off one of your fingers, in order to then triumphantly declare the saw to be suppar to a chisel.
m(
Leif Svalgaard says:
“Any cycle whatsoever will at various times be in phase with the sunspot ‘cycle’ [like a stopped clock being correct twice a day]. when your tidal cycle [running at a different rate] is 5 years off with respect to the sun it is ‘completely out of phase’. Then, as it goes even more and more out of phase it catches up with the sun and you hail that as success while it actually is a growing failure.”
Not so for the Earth-Venus-Jupiter syzygy nodes, there is no such long term slip relative to the sunspot cycle.
“Make the coincidence even more striking by plotting all the data [back to 1610].”
According to the E-V-J configurations there should be cycle maximum at around 1605 with the same polarity as SC23 (odd numbered).
tallbloke says:
“..because the cycle lengths tends to cluster at around 10.38 (VEJ) or 11,86 (J)..”
The configurations alternate between Earth-Venus conjunctions in line with Jupiter in even numbered cycles to Earth-Venus oppositions in line with Jupiter in odd numbered cycles, Meaning it has to alternate from 6.5 Venus synodic cycles to 7.5 Venus synodic cycles from one cycle to the next. On the short term that would imply lengths of 10.39 and 11.99 yrs, while on the longer term it’s nearer 10.27 and 11.86 years as there is an occasional shorter step to keep the three in sync. If you look where Jupiter is at every even numbered cycle maximum, you’ll see it do a full circle in about seven Hale cycles, which makes it look like the Jupiter orbital period is completely irrelevant from cycle to cycle.
Anyone got any Excel formulae I can use to explore some other datasets?
Leif Svalgaard says: ” Here is a recent paper by Nick Lomb http://www.leif.org/EOS/Lomb-Sunspot-Cycle-Revisited.pdf showing that the side peaks of the 11-yr peak are real and indicate modulation by a long cycle [102 years] just as my crude FFT analyses showed. ”
Using separation of peaks and interpretting as amplitude modulation is one way that spectral analysis can detect frequencies much longer than the sample. Of course some caution is needed in order not to see every pair of peaks as frequency modulation when this could be a totally spurious conclusion.
It’s more convincing if there is some residual of the central “carrier” frequency visible too. The symmetry of the triplet being a further indication that it is a true physical amplitude modulation .
An nice example is Arctic sea area/extent:
http://climategrog.wordpress.com/?attachment_id=423
There is a nice symetrical (in frequency) triplet centred of 2.0 years.
The analysis was done on 2nd differenctial (acceleration) or ice area/extent to make the data more or less “stationary”. When the triplet alone is used to reconstruct a time series it captures a significant amount of the variation. Interestingly it reproduces almost perfectly the “catastrophic” melting of 2007 and also accounts for the reduced variability seen in the time series during the “accelerating melting” that happended between 1997 and 2007, and the larger variability before and after that period.
http://climategrog.wordpress.com/?attachment_id=216
Sorry Willis, I do try to have the habit of putting data sourse into all my graphs and thoroughly recommend that practice. I’m really short of time right now. I will try to tidy this up later.
I’m afraid that I agree with Spence_UK’s comments.
There are many other ways of extract shape of a signal from its frequency domain representation than looking at the power spectrum.
On a more general topic. The HADCRUT data is aliased. See:
http://judithcurry.com/2011/10/18/does-the-aliasing-beast-feed-the-uncertainty-monster/
This is a fundamental error that leads to completely unreliable signal processing and spectral analysis.
@Tony and Leif
Yes, my apologies, I was a bit frustrated this morning at the errors and misconceptions in this post, and wanted to reply but also needed to leave the house for work, and I was careless in what I typed. As such, I’ve managed to add an error of my own.
As I explained to Willis further up thread, zero padding performs a (sin x / x) interpolation and does not add information. The point I wanted to convey (but articulated poorly) was that the central frequency of the peak in Leif’s plots was poorly identified without padding, as in the charts he linked.
I doubt the 102 year cycle is “real”. It may be, but the evidence is pretty thin. The reason being the power spectral density of the solar cycle at that point is very likely 1/f, and people always tend to attribute causality to the lowest frequency peak (which is, in 1/f noise, almost always the largest magnitude). Overintepretation of low frequency peaks in 1/f noise is the number one most common error in climate science. Unfortunately it has become such standard practice most people are unaware they are even doing it.
Willis, maybe you could look for cycles in this data;
http://i35.tinypic.com/2db1d89.jpg
Just a quick clarification – by “real” in quotes I mean a linearly separable deterministic cycle, as opposed to simply a random component of the 1/f PSD function.
Actually I also agree with Tony Mach’s comments.
Several other points: One cannot take a “Fourier Transform” of real data – it doesn’t exist. One is calculating a discrete Fourier series which has some operations in common with an FT. If you doubt this, what is the Fourier Transform of a sine wave?
I am struck by the lack of orthogonality and non-uniqueness of the “periodicity transform”. I’m not sure how this affects the interpretation of data.
I wrote a guest post about signal processing some months back after being challenged by Eschenbach to put up or shut up.
http://wattsupwiththat.com/2013/04/09/correlation-filtering-systems-and-degrees-of-freedom/
I concluded:
Modern scripting programs such as “R” allow one to perform many signal processing calculations very easily. The use of these programs lies in not applying them blindly to data but in deciding how to use them appropriately. Speaking from bitter experience, it is very easy to make mistakes in signal processing and it is difficult to recognise them. These mistakes fall into three categories, programming errors, procedural errors in handling the signals and not understanding the theory as well as one should. While modern scripting languages are robust, and may largely eliminate straight programming errors, they most certainly do not protect one from making the others!
I still think that this is appropriate.
@RC Saumarez thanks for the link to your article at Judith Curry’s site, it was an enjoyable read after this article 🙂
And for the comments by many (Tony Mach, Bart, etc) who do have a good understanding of signal processing principles.
Willis
Did you look at the CET data . It goes back to 1659. There appears to be a 110 year climate cycle with start of major troughs at 2000, 1890,1780, 1670, 1560. All are around the start of major solar minimums
Willis
I was thinking of this graph for CET. TB has the raw data
http://wattsupwiththat.com/2013/05/08/the-curious-case-of-rising-co2-and-falling-temperatures/
I thought I throw in some general recommendations for doing Fourier transforms, to do something constructive:
Beware of artefacts introduced by choice of window. What you feed into a FFT is the multiplication of signal and window, and what you get out of the FFT is the convolution of signal and window.
Using “no window” is equal to using a rectangular window.
A rectangular window is good for finding a sinusoidal signal in white noise. (While solar cycles are somewhat periodic, they are definitely not sinusoidal – and not actually cyclic! – and I would furthermore speculate that the noise is probably not white. So choice of window might probably matter a lot.)
Re window choice: Maybe try e.g. a tapered cosine window as a initial choice (because it throws away the least amount of data). IMHO it is prudent to compare this with the results from e.g. a Dolph–Chebyshev window (because of its well formed side lobes).
A Fourier spectrum is complex. The real part is the magnitude, the imaginary part is the phase. The magnitude is therefore only half of the information from a Fourier analysis. Sometimes looking at only that half is enough. But sometimes it is stupid to throw away the other half of the information. (Analysis of the phase information is certainly more difficult and not as intuitive as magnitude.)
This is of the top of my head, and I had to translate it from my primary language to English. So this list is most likely incomplete and possibly misleading (due to my translation of words). Caveat user.
One suggestion I have for analysing solar cycles: calculate a spectrogram.
Calculating the spectrum for e.g. 40y periods in your dataset. This should ensure that at least two complete solar cycles are in each period, and that you “catch” that 11+/-something cycle length. (Maybe do a second spectrogram with longer/shorter periods than 40y – there is possibly a better choice than 40y for solar cycle data.)
Start with the most recent period (e.g. 1970 to 2010), then calculate the spectrum for the a preceding 40y period with a 30y overlap, by moving the period back 10y (e.g. 1960 to 2000). (Maybe it is prudent to try out other overlap choices. Good choices for overlap may be 75%, 50%, 25% or 0%. Might depend on choice of window…)
Repeat until you reach the end of your data set.
For visual inspection: Plot all 40y Fourier spectra in one spectrogram.
Estimate the solar cycle length from each 40y Fourier spectrum (possibly best to have a function to do that).
For visual inspection: Plot that estimate over time. Now you should be able see if/how the solar cycle length changes over time.
Sorry for reposting, the formatting got lost in my last comment:
I thought I throw in some general recommendations for doing Fourier transforms:
– Beware of artifacts introduced by choice of window. What you feed into a FFT is the multiplication of signal and window, and what you get out of the FFT is the convolution of signal and window.
– Using “no window” is equal to using a rectangular window.
– A rectangular window is good for finding a sinusoidal signal in white noise. (While solar cycles are somewhat periodic, they are definitely not sinusoidal – and not actually cyclic! – and I would furthermore speculate that the noise is probably not white. So choice of window might probably matter a lot.)
– Re window choice: Maybe try e.g. a tapered cosine window as a initial choice (because it throws away the least amount of data). IMHO it is prudent to compare this with the results from e.g. a Dolph–Chebyshev window (because of its well formed side lobes).
– A Fourier spectrum is complex. The real part is the magnitude, the imaginary part is the phase. The magnitude is therefore only half of the information from a Fourier analysis. Sometimes looking at only that half is enough. But sometimes it is stupid to throw away the other half of the information. (Analysis of the phase information is certainly more difficult and not as intuitive as magnitude.)
This is of the top of my head, and I had to translate it from my primary language to English. So this list is most likely incomplete and possibly misleading (due to my translation of words). Caveat user.
One suggestion I have for analysing solar cycles: calculate a spectrogram.
1. Calculating the spectrum for e.g. 40y periods in your dataset. This should ensure that at least two complete solar cycles are in each period, and that you “catch” that 11+/-something cycle length. (Maybe do a second spectrogram with longer/shorter periods than 40y – there is possibly a better choice than 40y for solar cycle data.)
2. Start with the most recent period (e.g. 1970 to 2010), then calculate the spectrum for the a preceding 40y period with a 30y overlap, by moving the period back 10y (e.g. 1960 to 2000). (Maybe it is prudent to try out other overlap choices. Good choices for overlap may be 75%, 50%, 25% or 0%. Might depend on choice of window…)
3. Repeat until you reach the end of your data set.
4. For visual inspection: Plot all 40y Fourier spectra in one spectrogram.
5. Estimate the solar cycle length from each 40y Fourier spectrum (possibly best to have a function to do that).
6. For visual inspection: Plot that estimate over time. Now you should be able see if/how the solar cycle length changes over time.
Hi Willis,
I am unsure of how and where TSI is measured.
1) Does TSI just incorporate magnitude or does it include total energy at each frequency?
2) Previous to the Satellites, TSI was measured on the ground. Is it still measured on the ground? I
3) I know that now it is measured in space. How and where and what new technique?
4) Is there a comparison of the ground verses space based measurements?
5) Why does the enormous increase in high energy UV not show up in the TSI? Is the averaging so sever that the UV signal is “washed out”?
Thanks,
Jerry
Tony, I think your list is good, the only change I would make is your definition of magnitude and phase.
Magnitude is the RMS of the real and imaginary components, the phase is the four-quadrant arctangent of the real and imaginary components.
Willis says:
Figure 1 ARIM TSI
That’s not the ACRIM TSI, That’s the bloody PMOD modeled up nonsense.
PMOD Data here: http://acrim.com/TSI/composite_d41_62_1204.txt (Willis munged his link)
ACRIM data here: http://acrim.com/RESULTS/data/composite/acrim_composite_130329_hdr.txt
Ulric Lyons says:
July 30, 2013 at 3:23 am
Thanks for your observation Ulric.
@Tony Mach.
Although you say that phase is “more difficult”, it is essential in determining signal properties. Consider a signal with a nominally flat amplitude spectrum. If the phase of each component is zero, the signal is an impulse. If the phase is random, the signal is random. Correlation, which is determining the similarity between signals, depends on the crossed phase spectrum.
Phase is key to understanding the structure of a signal and has been ignored here. You get non-sinusoidal cycles, eg: a square wave or triangular wave, because the phases of the various components are aligned.
Ian H Australia says:
“I feel strongly that one needs to work out what each individual solar output component does to the global temp,ETC, as TSI evens out individual components, and is thereby not that useful. We should be seeing how proton, electron, Ultra-Violet, X Rays, Ap, 10.7, etc, etc each individually have on earths various layers of atmosphere, jetstreams, temperature, magnetic effects, other reactions, the differing reactions at poles and differing latitudes, etc, etc. Using a broad brushed TSI is not going to achieve any real detailed meaningful results. And help us that much really in finding all the answers to Solar-Climate-Weather-etc interactions…There is so much out there we need to research and learn.
However, the problem lies in that most of this solar component data is only very recent!”
I’m with Ian on this, as Willis has attested there are no reliable cycles in the record to date. This could easily be due to “emergent phenomenon” (phase changes, energy conversions, convection currents, etc., (basically work), etc.); but it also could be partially due to a component of solar output that varies independently of the total. I’ve used this example before: an acid copper plating solution can be working fine or producing scrap at the exact same 500 ppm TOC because there’s some two dozen organic components and the relative quantities matter, too little carrier for the amount of dye (even though the total is the same as when they are balanced) drastically alters the results.
Willis
The relationship between sun spot count and global mean temperature is only in the long-term of about 94-years running average as shown:
http://www.woodfortrees.org/plot/sidc-ssn/mean:1128/normalise/plot/hadcrut4gl/mean:1128/normalise
The correlation is an extremely strong one.
tallbloke says:
July 30, 2013 at 6:12 am
“Thanks for your observation Ulric.”
In fact it is the shape of the orbit of Jupiter that also effects the timing of the J-E-V syzygy nodes. Jupiter moves from the north node of the ecliptic plane to the south node in close to 4 Venus synods, and from the south node to the north node in close to 3.5 Venus synods. It’s looking like solar cycles with a maximum having the J-E-V syzygies in line with the nodes are more likely the longer ones at around 7 Venus synods.
“DirkH says:
July 29, 2013 at 4:19 pm
Willis Eschenbach says:
July 29, 2013 at 3:23 pm
“The most obvious difference is in the size of the peaks at around 52 years. Again, I suspect the result is because of the “scalpel” technique, but I have no way of demonstrating that.”
That looks indeed as if the BEST scalpel technique kills the low frequency periodicities. (Mosher’s defense “You’re wrong” doesn’t really cut it. I doubt he understands what he did.)
1. your wrong in your description of it
2. you can test whether it impacts the spectral characteristics quite easily.
You assume that there is low frequency information there, however, the source of that assumption is a data series ( crutem) that is seriously flawed.
sorry I am a bit late to the discussion
and I don’t have the time now to read all comments
but what I am missing in the post is something like the planetary movements (per their mass?) versus the sunspot counts
We know SST can be correlated with SSN, but I hope you will see that SSN can be correlated with planetary movement.
This is what the paper from William Arnold was all about:
http://www.cyclesresearchinstitute.org/cycles-astronomy/arnold_theory_order.pdf
My best fit for the drop in maximum temps. shows that around 40 years ago counted from 2012, we reached the maximum speed of warming. All fits (with high correlation) show that in 1995 we turned from warming to cooling (as far as maxima is concerned). The difference between 1995 and 1972 is 23.
That is one node (of 2 “current” solar cyces) as William Arnold pointed out. William Arnold did not agree with the current use of the half solar cycles. OTOH, he was just out by 5 years but he did not have my data to work with so his beginning and ending could be a bit wrong, for the present. Now, he next node (quarter part of the a-c wave) could be between 22 and 29 years. I am trying to narrow this down from his notes on the planetary movements. Unfortunately I never studied astronomy. I have no clue as to what his dials there mean, Can anyone of you help me out with that? I am sure this is something like the Rosetta stone, but we must get the translations right (of the movements of the planets and their weight i.e. gravitational force, exerted on the sun?)
If it is 22, we could arrive there in 1995+22= 2017. That (the bottom) would be the beginning of the major drought times for the higher latitudes because there will be less moisture going >[40] and the speed of cooling is constant for a period of time around the bottom of the a-c wave – no acceleration. Already, there should be a noticeable decrease in precipitation on the Great Plains as we are curving down, no doubt currently still being blamed on “man made climate change”.
(Hoover dam water is lower than normal?).
So, truly there is and there will be climate change with a major impact on the food production in USA and Canada. It is just not man made…..It is God-made. The planetary system works like an elastic band. lf it was not there we would have runaway warming and /or runaway cooling.
We must get the date right for the bottom of my curve:
http://blogs.24.com/henryp/2012/10/02/best-sine-wave-fit-for-the-drop-in-global-maximum-temperatures/
Ulric Lyons says:
July 30, 2013 at 7:35 am
Typo.. at around 7.5 Venus synods.
Spence_UK says:
July 30, 2013 at 4:36 am
I doubt the 102 year cycle is “real”. It may be, but the evidence is pretty thin.
The strongest evidence is the data itself: http://sidc.be/sunspot-index-graphics/wolfaml.php with low cycles every ~100+ years. Now, I don’t think there is a real cycle in the sense that there is a periodic physical process that gives rise to the longer-term variation, but for the last three hundred years there is a clear 100-yr variation which could be just a random fluctuation, but it is in the data, so must produce side lobes on the 11-yr peak.
Dr. Lurtz says:
July 30, 2013 at 5:48 am
1) Does TSI just incorporate magnitude or does it include total energy at each frequency?
TSI is the Total of all energy at all frequencies
2) Previous to the Satellites, TSI was measured on the ground. Is it still measured on the ground?
Not in its usual form, although there are measurements of atmospheric attenuation.
3) I know that now it is measured in space. How and where and what new technique?
On satellites high above the atmosphere. One is a million miles away, another just a few thousand. To measure TSI you simply let raw sunlight fall upon a black surface and measure the resulting heating. In principle this is as simple as it can be. In practice it is a lot more complicated, but still straightforward: sunlight is let into the instrument through a small hole the area of which is known with high accuracy. Then the light is absorbed by a black surface in form of a cone. The cone is wound by an electrical wire through which a current flows. The current heats the cone to maintain a constant temperature. The amount of current necessary to keep the temperature constant is measured accurately and is a measure of the energy absorbed. The instrument is calibrated on the ground before launch so that we know what current to expect for a given incoming radiation flux.
4) Is there a comparison of the ground verses space based measurements?
No, the space based measurements are so much more precise.
5) Why does the enormous increase in high energy UV not show up in the TSI?
Because there is no ‘enormous increase’ in energy received. There is high variability of extreme UV, but the total energy at those wavelengths is very small.
tallbloke says:
July 30, 2013 at 6:10 am
That’s not the ACRIM TSI, That’s the bloody PMOD modeled up nonsense.
Which is fine as ACRIM has severe problems [e.g. a spurious yearly variation]. PMOD has problems too, but is generally better than ACRIM. For Willis plot it makes no real difference which is used.
Great post from Willis, it will take some time to get it in either from the article or the additional comments.
Lot of other stuff worth attention and some not so, where my comment will end up depends on, as always who is judging.
– Average solar output varies over centuries, so is the temperature, need to be compared over similar period (some temperature lag is likely)
– For some time now I have concluded that it is not so much sunspot cycle direct intensity or the TSI, but the energetic events (CMEs) that are the ones to have effect on the natural climate change
Geomagnetic measurements combine both solar and the Earth’s magnetic field variability. I have shown elsewhere that these measurement do give a good correlation with the natural temperature variability.
Now we have one more indicator of the geomagnetic activity (sun-Earth) in form of the Danish aurora records.
And what do they show?
You can see here:
http://www.vukcevic.talktalk.net/AuGTs.htm
Well, to me it looks pretty conclusive, no ‘cyclo-hypertension’ here, very little response at SC but lot more at les known Hale Cycle.
Dr. S. may dismiss the above, since he is only half correct (on 11yr) and half wrong (Hale Cycle), but then he insists on always being correct
Total number of hits at my web-graphs at this moment is 199196 ; at 200k would be good time to take a break, and give Dr. S and many others a bit of a rest.
Thanks all
RC Saumarez says: On a more general topic. The HADCRUT data is aliased. See:
http://judithcurry.com/2011/10/18/does-the-aliasing-beast-feed-the-uncertainty-monster/
Excellent demonstration of the effect. I had not seen that article.
Averaging is a valid means of reducing _random_ (gaussian distributed) noise but is NOT valid processing in the presense (or possible presence if you have not even thought to check) of periodic or pseudo periodic variations.
The ubiquetous practice in climate science of taking monthly averages without any attempt at anti-alias filtering has its rooting in two things:
1) ignorance of proper data processing techniques.
2) ignorance of proper data processing techniques.
3) the abritrary and spurious assumption that anything that is not due to AGW is “random” noise.
Well OK, that’s three: I forgot the second one.
Kasuha says:
July 29, 2013 at 11:21 pm
Ummmm … have you looked at the scale on Figure 12? Less than a tenth of a degree over the range of sunspot numbers, and it’s going (if anything) in the opposite direction to that claimed by the “it’s the sun, stupid” crowd.
As to your claim that “it may even be significant” in affecting the earth’s temperature, it’s not even significantly affecting just the troposphere. As I noted in the head post, the relationship is far from significant (from memory the p value is on the order of 0.3) …
w.
Leif Svalgaard says:
July 30, 2013 at 8:28 am
tallbloke says:
July 30, 2013 at 6:10 am
That’s not the ACRIM TSI, That’s the bloody PMOD modeled up nonsense.
Which is fine….
….it makes no real difference
It’s fine we’re told one dataset is being used when in fact a totally different dataset is being used? Ohhh, “it doesn’t matter”. Riiiight.
vukcevic says: “Total number of hits at my web-graphs at this moment is 199196 ; at 200k would be good time to take a break”
200k would be a good time to start documenting what you are trying to show and all your vaugely labelled graphs. That would give other the chance to check what you are putting forward and asses whether it has any merit.
The particular link you give is fine example it just says “global temperature”. It’s anyone’s guess what data you’re using.
A source for the “Danish Aurora Data” would be a plus.
You’ve been banging out this stuff for over ten years now. It’s well over due that you turn it into something reproducible and verifyable. ( IMHO )
tallbloke says:
July 30, 2013 at 9:20 am
It’s fine we’re told one dataset is being used when in fact a totally different dataset is being used? Ohhh, “it doesn’t matter”. Riiiight.
Right! As the graph would look very much the same [you couldn’t tell the difference – except ACRIM would have a bit more noise].
I’m sorry but I think that this post is complete rubbish because you have use a technique, which you don’t understand, improperly to test a hypothesis that is ameanable to straightforward analysis.
You use “periodicity transform” to look for cycles in sunspot data and then try to relate this to temperature by forming periodicty analysis of temperature records. You conclude that there is no relationship between sunspot cycles and temperature(s).
What you appear to be saying that there is no linear correlation between sunspots and temperature. In this case, why on Earth did you not use standard signal processing correlation techniques to establish this? At least people with a signal processing background might take the analysis seriously and have some confidence that it was correct.
Maybe I’m going to get slammed here, but I’m going to try anyway: since our observation point moves every day should the time series be adjusted for the difference?
Spence says: “Magnitude is the RMS of the real and imaginary components”
No it’s not. There’s no mean.
If you’re going to correct someone, it’s best to be correct.
http://tallbloke.wordpress.com/2013/07/30/poppenhaeger-hd-189733a-has-been-tidally-influenced-by-the-hot-jupiter/
Steven Mosher:
Actually, there is low frequency information in BEST too–it shows a peak around 60 years, and the amplitude is very similar to what is seem using CRUTEM3.
Leif Svalgaard says:
”Because there is no ‘enormous increase’ in energy received. There is high variability of extreme UV, but the total energy at those wavelengths is very small.
But that doesn’t take into account the varying quality of energy. An amount of UV can split O2, cause sunburn, power photosynthesis, etc. but no amount of IR can do any of those things. WUWT?
Ulric Lyons says:
July 30, 2013 at 8:22 am
Thanks again Ulric, food for thought. Ian Wilson sent me something very interesting today related. I’ll pass your obs on to him.
tallbloke says:
July 30, 2013 at 9:20 am
———————————————————————————————-
Here’s a solution to the solar planetary problem. Put your finger on a plasma ball. Orbit your finger around the glass. Stop thinking about gravity. Start thinking FTE
Venus, Earth, Jupiter? Try re-running the numbers for Mercury, Earth, Jupiter. These are the only planets for which flux transfer events have been observed.
The only way for solar tidal (gravity) hypothesis to work would be for the pioneer anomaly to have verified the push gravity hypothesis and the sun to have an iron core. Leif’s head could explode. Best to keep with the less messy options 😉
Mosh’ says: 2. you can test whether it impacts the spectral characteristics quite easily.
How so? This was the first thing I wanted to do when they did the first public release. But I gave up on BEST as unverifiable / untestable a long time ago since they only provided data in such a massive file that you needed a main-frame computer just to load it.
If it’s “quite easy” presumably someone within the team has already done it as part of basic QA. Where can I find the results?
Thanks.
RC Saumarez:
Why don’t you do this and tell us what you find?
I have a signal processing background, I took it seriously, but I’m more interested in the methodology than the conclusions . You are right though that I would expect a comparison of several different methods, not just one, before I took the results seriously. Of course Mann made the same mistake in relying on the re-statistic and not reporting other metrics such as r2.
Unlike with Mann, which was a peer reviewed publication and considered part of the permanent record—this is a blog post, expected to be exploratory in nature, and should not be assume to be “permanent” in any sense. (Blogs end, data bases get corrupted, etc.)
Greg:
???
What computer are you trying to load it on, that it fails?
I assume something more modern than an Apple ][.
Greg says:
July 30, 2013 at 9:32 am
….
Hi Greg
Thanks for your lucid observations. Well I had few years break in between. It is a hobby that I do not take too seriously, no one is particularly convinced anyway. If someone is interested to pursue in a further a bit of effort may be required. On two or three occasions I was asked by JC to write a post for her blog, but I declined her kind invitations.
BTW. Temps here
http://www.vukcevic.talktalk.net/AuGTs.htm
are from hadcru3. Dr.S is source of the aurora data.
John West says:
July 30, 2013 at 9:54 am
but no amount of IR can do any of those things. WUWT?
It is mainly the IR that heats the Earth…
tallbloke says:
July 30, 2013 at 9:51 am
http://tallbloke.wordpress.com/2013/07/30/poppenhaeger-hd-189733a-has-been-tidally-influenced-by-the-hot-jupiter/
As I predicted, you misunderstood this paper too. Very early in the life of a star there is a coupling [not tidal but magnetic] between its stellar wind and the newly forming protoplanets. It is that mechanism, Poppenhaeger is taking about being disturbed by tidal effects back then. Not planets controlling stellar magnetic activity today. As she points out: “We can therefore exclude a stellar activity cycle to be the cause for the disagreement in activity levels”. From another recent paper of hers: “We conclude that there is no detectable influence of planets on their host stars”.
@Carrick
You are a signal processer and you take this seriously :
• “I’ve used the Periodicity Transform to look at the sunspot record, both daily and monthly. In both cases we find the same cycles, at ~ 10 years, ~ 11 years, and ~ 11 years 10 months. Unfortunately when the data is split in half, those cycles disappear and other cycles appear in their stead. Nature wins again.” ?
The last “signal processing” post by Mr Eschenbach on filtering and degrees of freedom also lacked a certain amount of DSP and statistical skill. I remarked that this was the case and was given a similar challenge. I responded by writing a very basic guest post:
http://wattsupwiththat.com/2013/04/09/correlation-filtering-systems-and-degrees-of-freedom/
If someone presents himself as a guru on “one of the most influentual resources on global warming” (see top), I think that better than this is required. I have explained in the above post how correlation works – it seems a pity that Mr Eschenbach did not read the post that he provoked.
I commend the last paragraph of this post to you.
It is mainly the IR that heats the Earth… said Leif
I don’t think so:
the data
http://blogs.24.com/henryp/2013/02/21/henrys-pool-tables-on-global-warmingcooling/
shows that most heat is coming into earth via the SH oceans
(if you would actually take the time to study them)
and IR does not heat the water, that much…
It is the SW sunlight that heats the oceans. How else do you explain the existence of clouds and weather?
The point is that although UV seems cold to the skin, it still burns after exposure, does it not?
So also with the water. The water has to absorb the UV and in the end it has to covert that UV to heat,
since water absorbs in the UV region/.
You can actually observe this phenomena if you swim in a pool of water unstirred by convection or wind and heated by the sun for a day. There are several layers of warm water…. why?
Hmm. It appears that I made another silly typo. Should have been “RSS” instead of “RMS”. Thank you for picking that up.
I will have to bow to your apparent infallibility that means you never make such mistakes when making corrections.
RC Saumarez, thanks for the link. I’ll parse it, may get a chance to respond, may not, depends on RL conditions.
I was ‘passing through’ today and often miss posts…. As to lagged-correlational studies, you might be interested in this.
I don’t think Willis would claim to be “guru”, but I’ll let him speak for himself. as I’m not particularly interested in food throwing exercises myself.
So then is the lack of Little Ice Age sunspots merely a coincidence? (apologies if this has already been covered)
HenryP says:
July 30, 2013 at 10:53 am
“It is mainly the IR that heats the Earth… ”
I don’t think so
Half of the incoming radiation is IR. That energy does not just disappear, hence contributes greatly to heating the surface and the air [land and water].
Leif says:
Yes, when I said “real” I was referring to a linearly separable deterministic (physical) process. As you note, I agree the fluctuation is present in the data, but it looks impossible to distinguish from a 1/f noise process, which is likely to be present judging by other low frequency peaks.
Nicola Scafetta: It is evident to me that Anthony and Willis are behaving quite dishonestly by trying to defame my research.
Please! That is absurd. Willis is critiquing your research and presenting an alternative method for exploring possible periodicities in the data.
Spence_UK says:
July 30, 2013 at 11:13 am
As you note, I agree the fluctuation is present in the data, but it looks impossible to distinguish from a 1/f noise process
The issue was [still is] whether such a fluctuation is present in the data [it is]. If the fluctuation would generate side lobes on the 11-yr sunspot cycle [it does]. If simple FFT could detect that [it can]. If the ‘side lobes’ are due to planets [they aren’t].
Leif Svalgaard: It is mainly the IR that heats the Earth…
Do you have a reference for that? Are you distinguishing between the relatively long wave IR and the relatively short wave IR? All my other references say that the earth is warmed by the full visible spectrum of incoming radiation.
What Matthew R. Marler said. Exploring an alternate method isn’t defamation.
Konrad says:
July 30, 2013 at 9:56 am
Try re-running the numbers for Mercury, Earth, Jupiter. These are the only planets for which flux transfer events have been observed.
Saturn exhibit’s aurorae and has a hexagonal shaped torus of cloud near the poles…
The only way for solar tidal (gravity) hypothesis to work would be for the pioneer anomaly to have verified the push gravity hypothesis and the sun to have an iron core. Leif’s head could explode. Best to keep with the less messy options 😉
Ray Tomes relativistic matter/energy conversion theory is still in play. That would cause differential gravitational pull from the gas giants in the z-axis over many years rather than cancelling per solar rotation as the x-y plane does.
http://tallbloke.files.wordpress.com/2012/11/ssbz-ssbr-ssn.png
And getting back on topic, the FFT of the z-axis data better matches the sunspot FFT
Tony Mach: Zero padding does not create any information, but it allows you to run a longer FFT, to get a resulting Fourier spectrum with more bins (finer resolution) – this is akin to interpolating in-between values, but should be better than interpolation.
Zero Padding does not create “information”. Zero padding adds bias to the estimates. Zero padding creates 0s in the parts of the series that, had they been measured, would almost surely have been non-zero. It is done for no other reason than to use a technique which would have good properties had there been no need for the 0 padding.
Leif
but it is in the data, so must produce side lobes on the 11-yr peak.
The ~100 yrs ‘cycle’ is a result of rectifying the Hale cycle, it’s not real. What we are observing is mainly the beat of 10 and 11 yrs (and 9*11.8yrs =106 yrs). http://virakkraft.com/SunspotFFT.jpg
Anthony Watts says:
July 30, 2013 at 11:27 am
What Matthew R. Marler said. Exploring an alternate method isn’t defamation.
If someone searches for a needle in a haystack with a pitchfork and fails to find it, and then declares the needle doesn’t exist, and calls the people who found it using a magnet cyclomaniacs, it might not be defamatory, but it does make them look stupid.
Leif, your focus is on the sidelobes, which is not the same thing I am interested in (I already understand where they come from, thanks to your explanation). The 1/f noise is more interesting in itself. It has many consequences, many counter-intuitive, and it behaves quite differently to “conventional” noise processes (white/iid, autoregressive). The physics of what causes such processes to arise is an interesting area.
Spence_UK says:
July 30, 2013 at 4:36 am
“I doubt the 102 year cycle is “real”.”
You and I are sympatico in most regards, however here, you are missing the fact that the 131 year beat modulation is evident in the time series plot.
At the root, there are two fundamental processes with central periods near T1 = 20 and T2 = 23.6 years. Let
x(t) = A*sin((2*pi/20)*t) + B*sin((2*pi/23.6)*t+phi)
Squaring this, get
x(t)^2 = A^2*sin((2*pi/20)*t)^2 + B^2*sin((2*pi/23.6)*t+phi)^2 + 2*A*B*sin((2*pi/20)*t)*sin((2*pi/23.6)*t+phi)
Use your trig identities to get
x(t)^2 = (A^2+B^2)/2 – A^2/2 * cos((2*pi/(20/2))*t+phi/2) – B^2/2 * cos((2*pi/(23.6/2))*t) + A*B*cos((2*pi*(1/20+1/23.6))*t+phi) – A*B*cos((2*pi*(1/20-1/23.6))*t-phi)
so, we see resulting periods of 20/2 = 10 years, 23.8/2 = 11.8 years, 1/(1/20+1/23.6) = 10.8 years and (1/20-1/23.6) = 131 years.
The Sunspot data is a magnitude function, basically the square root of this. That operation produces other harmonics, but these are the major ones. As I have shown, the Sunspot behavior can be qualitatively replicated with this type of model.
The important thing is, this all comes about because of fundamental processes at about 20 and 23.6 years. This is basically how long it takes for the Sun to return to a recurring state of magnetic polarity. So, besides the weak coupling I would expect, that makes me additionally leery of making a connection with astronomical phenomena having periods of about 11 years. It is periods of more like 20 and 23.6 years which would need to be matched.
RC Saumarez says:
July 30, 2013 at 4:26 am
“This is a fundamental error that leads to completely unreliable signal processing and spectral analysis.”
Not necessarily. It depends on what components are in the data. One of the reasons that simple “boxcar” filtering (averaging with full-width, back-to-back decimation) is extensively used for data collection and compression is that the zeros of the transfer function line up such that there is no aliasing to dc, and frequencies which would alias to near dc are severely attenuated. Some analysts at the top labs like to use a triangular weighting with half-width decimation, because there, also, the zeros fall precisely such that there is no aliasing to dc (because a triangular weighting is the convolution of two averages – for that reason, the envelope of the response falls off at a more rapid -40 dB/decade instead of the -20 dB/decade of the single average). The frequencies near dc are generally those with which we are most concerned.
Matthew R Marler says:
July 30, 2013 at 11:26 am
Leif Svalgaard: It is mainly the IR that heats the Earth…
Do you have a reference for that?
Does one need a reference?
All my other references say that the earth is warmed by the full visible spectrum of incoming radiation.
If you omit ‘visible’ I would agree. The ‘invisible’ part is half of all the energy. That energy does not just disappear. My comment was actually related to whether it was UV or IR that heat the Earth.
Leif
Another problem with your modulation pseudoscience. In AM the side bands are identical. The 10, 11, 11.8 are not.
I have another post in the queue – probably because of equations and multiple links.
Matthew R Marler says:
July 30, 2013 at 11:39 am
No. Zero padding does nothing more than more frequently sample the continuous-in-frequency Fourier Transform. It is not intuitive. You need to do the math.
Spence_UK says:
July 30, 2013 at 11:42 am
Leif, your focus is on the sidelobes, which is not the same thing I am interested in (I already understand where they come from, thanks to your explanation).
Contrast that with the following nonsense:
lgl says:
July 30, 2013 at 11:40 am
The ~100 yrs ‘cycle’ is a result of rectifying the Hale cycle, it’s not real. What we are observing is mainly the beat of 10 and 11 yrs (and 9*11.8yrs =106 yrs).
lgl says:
July 30, 2013 at 11:47 am
The reason should become apparent when my queued post appears.
Leif Svalgaard says:
July 30, 2013 at 11:49 am
“Contrast that with the following nonsense:”
He is correct to an extent. It is a result of rectifying the Hale cycle. However, it is quite real.
Spence_UK: As for Matthew’s complaint that the harmonic series is a non-parsimonious representation of a simple function,
Is that a complaint? I’d say it’s just a fact that needs to be taken into account when choosing between alternative representations (and explorations) of a time series. You can certainly do an FFT on a time series that has a sawtooth curve, but it’s hard to compute from the coefficients what the starting, ending, and inflection points are, as well as the slopes of the teeth. You’d be better off, if you suspect a sawtooth to be the true shape of the signal, to perform piece-wise linear regression with non-linear estimation of the change-points. Certainly not as fast as an FFT, but at least you’d know the shape of the curve from the coefficients, which is sometimes what you want to know.
Leif Svalgaard says:
July 30, 2013 at 10:11 am
tallbloke says:
July 30, 2013 at 9:51 am
http://tallbloke.wordpress.com/2013/07/30/poppenhaeger-hd-189733a-has-been-tidally-influenced-by-the-hot-jupiter/
Thanks for reposting the link Leif. Rather than argue with you here on Willis’ stats thread, I’ll let others follow the link, download the paper, read for themselves, and decide for themselves. Or they can take your word for it, their call.
tallbloke says:
July 30, 2013 at 11:54 am
I’ll let others follow the link, download the paper, read for themselves
They [and you] should also read:
http://hea-www.harvard.edu/~kpoppen/papers/Poppenhaeger_Correlation.pdf
“We conclude that there is no detectable influence of planets on their host stars”
http://tallbloke.wordpress.com/2013/07/30/poppenhaeger-hd-189733a-has-been-tidally-influenced-by-the-hot-jupiter/
though it is worth quoting the author:
This star is not acting its age, and having a big planet as a companion may be the explanation. It’s possible this hot Jupiter is keeping the star’s rotation and magnetic activity high because of tidal forces, making it behave in some ways like a much younger star.
See that use of the present tense there Leif?
Leif Svalgaard says:
July 30, 2013 at 11:59 am
tallbloke says:
July 30, 2013 at 11:54 am
http://tallbloke.wordpress.com/2013/07/30/poppenhaeger-hd-189733a-has-been-tidally-influenced-by-the-hot-jupiter/
I’ll let others follow the link, download the paper, read for themselves
They [and you] should also read:
http://hea-www.harvard.edu/~kpoppen/papers/Poppenhaeger_Correlation.pdf
“We conclude that there is no detectable influence of planets on their host stars”
That’s an earlier paper Leif.
Some scientists have the integrity to admit they may have been wrong and change their minds.
e.g.
Charbonneau 2003 “The planetary theory is dead”
Charbonneau 2013 “This is not astrology, it is science”
tallbloke says:
July 30, 2013 at 12:06 pm
It’s possible this hot Jupiter is keeping the star’s rotation and magnetic activity high because of tidal forces
Strong tidal forces can brake rotation [Moon on Earth] or even lead to tidal locking [many examples of that] and since rotation is implicated in generating stellar activity there could be a link there in a general sense. Nobody disputes that. The issue is whether the stellar cycles are synchronized with the orbital period of the planet and that does not seem to be the case, at least no examples have been found. But it seems you think you have found a straw to grasp at.
tallbloke says:
July 30, 2013 at 12:11 pm
That’s an earlier paper Leif. Some scientists have the integrity to admit they may have been wrong and change their minds.
The data from the paper stands. This is not about to changing her mind. Or admitting to be wrong.
Charbonneau 2013 “This is not astrology, it is science”
Even Charbonneau has not changed his mind: “It may all turn out to be wrong in the end, but this is definitely not astrology”. He says that the theory is testable and therefore qualifies as science, is all.
Leif Svalgaard says:
July 30, 2013 at 12:15 pm
Strong tidal forces can brake rotation [Moon on Earth] or even lead to tidal locking [many examples of that] and since rotation is implicated in generating stellar activity there could be a link there in a general sense. Nobody disputes that. The issue is whether the stellar cycles are synchronized with the orbital period of the planet and that does not seem to be the case, at least no examples have been found. But it seems you think you have found a straw to grasp at.
The interplay of several big and several close planets in the solar system produces exactly the frequencies found in properly done spectrographic analyses of the sunspot record. There is some straw grasping (and mud flinging) going on, but it’s not me that’s doing it.
Leif Svalgaard says: July 30, 2013 at 1:22 am
tallbloke says: July 30, 2013 at 1:18 am
I already did. the data from 1840 is more than enough to prove the point.
That the match has broken down…
________________________________
Leif,
Tallbloke posted a graph of his ‘planetary index’ closely following the SSN for 150 years, but you have not explained why you disagree with this graph. Are you saying this close match is coincidence? Do you disagree with the data? Why would there be such a close match, if there was no link between the two?
and…
Tallbloke,
Can you explain how your planetary index is calculated (from the orbits of the main planets). In what way does the PI effect the Sun – is this the business of swinging the Sun around its barycenter?
Thanks,
Ralph
tallbloke says:
July 30, 2013 at 12:06 pm
though it is worth quoting the author
What she actually said was:
“it a more likely possibility that the stellar angular momentum of HD 189733A has been tidally influenced by the Hot Jupiter, which has inhibited the stellar spin-down enough to enable the star to maintain the relatively high magnetic activity we observe today”
This is a has been scenario in the distant past [as the stellar spin-downs take place at the beginning of the life of the star]. One can trust you to misinterpret this.
tallbloke says:
July 30, 2013 at 12:23 pm
There is some straw grasping (and mud flinging) going on, but it’s not me that’s doing it.
You mean there are other poor souls out there grasping at other sun-planet straws? Perhaps some have already posted here.
Leif Svalgaard says:
July 30, 2013 at 12:19 pm
tallbloke says:
July 30, 2013 at 12:11 pm
That’s an earlier paper Leif. Some scientists have the integrity to admit they may have been wrong and change their minds.
The data from the paper stands. This is not about to changing her mind. Or admitting to be wrong.
You can’t have your cake and eat it Leif, if you want to claim data from the earlier paper on a different star system falsifies the later paper, then I can equally validly claim the conclusion from the later paper supercedes the earlier.
Charbonneau 2013 “This is not astrology, it is science”
Even Charbonneau has not changed his mind: “It may all turn out to be wrong in the end, but this is definitely not astrology”. He says that the theory is testable and therefore qualifies as science, is all.
“Is all”
Heh. After all the years of mud flinging, namecalling and denial of the possibility of planetary feedbacks to the Sun from you, that’s a good one.
Of course he has changed his mind. In 2003 he declared the theory dead. Now he’s excited.
ralfellis says:
July 30, 2013 at 12:33 pm
Tallbloke posted a graph of his ‘planetary index’ closely following the SSN for 150 years, but you have not explained why you disagree with this graph
I pointed out that the ‘close’ match [which is not that close to begin with, IMHO] has broken down for the last cycle. This often happens for spurious correlations. I asked him to extend the graph on the left [we have 400 years of SSN] but he claims to be too busy and doesn’t have time for that…
bart says
It is periods of more like 20 and 23.6 years which would need to be matched.
henry says
we agree!
so do you know which is the exact time for the period starting 1995?http://wattsupwiththat.com/2013/07/29/cycles-without-the-mania/#comment-1375260
@Bart,
The “boxcar” averaging and decimation does cause significant aliasing when taken in the context of the variability of a temperature signal, simply because the decimation reflects daily data around the Nyquist frequency of 1/(2 weeks). Simulations show that this produces spurious trends that are significant in climatic terms i,.e.: 0.3 degrees over a 30 year period.
See the link.
ralfellis says:
July 30, 2013 at 12:33 pm
Tallbloke,
Can you explain how your planetary index is calculated (from the orbits of the main planets). In what way does the PI effect the Sun – is this the business of swinging the Sun around its barycenter?
Thanks,
Ralph
Hi Ralph and thanks for your interest.The planetary index is calculated as specified by NASA scientist Ching Cheh Hung in his 2007 paper ‘Apparent Relations Between Solar Activity and Solar Tides’
ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20070025111_2007025207.pdf
The title of which informs you that it is tidally based.
However, I took a novel approach on the hunch that the relationship may have more to do with electromagnetism. which is about a million-zillion times stronger than gravity. So I recalculated the alignments along the Parker spiral, and factored in the changing speed of the solar wind using Leif’s admirable reconstruction, supplemented by another study which went back to 1840.
This is why I’m reluctant to be bullied by Leif into trying to use a solar activity dataset back to 1610. Partly because although the auroral records give a reasonably good idea of when the peak of the cycles occurred, the minima are much vaguer. And partly because we have no idea of the solar wind speed at this earlier epoch. I’ve done the best I can to work faithfully to the scientific method, and I think it’s unfair of Leif to demand more.
Leif Svalgaard says:
July 30, 2013 at 12:47 pm
ralfellis says:
July 30, 2013 at 12:33 pm
Tallbloke posted a graph of his ‘planetary index’ closely following the SSN for 150 years, but you have not explained why you disagree with this graph
I pointed out that the ‘close’ match [which is not that close to begin with, IMHO] has broken down for the last cycle.
Actually it hasn’t. Solar cycle 24 has (for the time being at least) died right on the planetary index cue. It may start up again in a couple of years and give us a ‘double peak’ cycle. Or there may be an odd ‘mini cycle’. It depends how the bigwigs choose to define it. Either way, that would be the icing on the cake for me.
Here’s the link to the plot again, for anyone who wants to keep and eye on its progress
http://tallbloke.files.wordpress.com/2010/08/rotation-solar-windspeed-adjusted.png
lgl says “…(and 9*11.8yrs =106 yrs).”
The presence of a strong 9 year cycle confounding attempts to identify a simplistic correlation between SSN and global temps was one of the main points of my article here:
http://climategrog.wordpress.com/2013/03/01/61/
This is the same 9 year cycle N. Scafetta identified as being due to the influence of the moon on Earth’s orbit around the sun and the same same one Judith Curry and BEST team identified in the land record recently.
Dr. S’ recent conversion to 100 year cycle is somewhat ‘disturbing’.
Let me help out to good old doc : there is no 100 fundamental as you can see here it is simply cross modulation of two other periodicity.
http://www.vukcevic.talktalk.net/LFC4.htm
We can safely assume that since it casts back with sudden reduction in length into the Maunder Minimum.
Average length since 1700 between two zero crossings is 52,5 years, giving an average length of 105 years.
Where two periodicity come from, one could produce at least half dozen planetary combination for each, or it may be something to do with solar internal workings.
It is well known fact that solar magnetic field has a pronounced bulge (discovered by young Svalgaard and his colleagues) written about by Dr. J. Feynman, which slowely drifts longintudinaly
http://www.vukcevic.talktalk.net/LFC7.htm
I would expect Dr. S to concentrate on the proper solar science rather than straining in the spurious harmonics of irregular centennial oscillation proclaiming it to be a fundamental. The lobes that Dr. S is so keen may be just irrelevant noise.
Year is strictly a local metric and 100 is a human invention, I don’t think sun cares much about either.
Leif Svalgaard says:
July 30, 2013 at 12:47 pm
I pointed out that the ‘close’ match [which is not that close to begin with, IMHO]…
Well it’s your solar wind speed reconstruction, not mine. 😉
Bart
Thanks for the details, but where is the ~100 yr cycle?
It is periods of more like 20 and 23.6 years which would need to be matched.
Right, like Earth and Venus being accelerated a little extra every 22 yrs on average,
http://virakkraft.com/EMB-AM.png (from Semi)
The Sun and the inner planets together must ‘counter’ Jupiters motion, they are one object in this regard, and since one part of that object, the inner planets, are accelerated the other part, the Sun, must counter their ‘extra’ motion.
from NASA
http://www.nasa.gov/content/goddard/large-coronal-hole-near-sun-north-pole/
vukcevic says:
July 30, 2013 at 1:20 pm
Dr. S’ recent conversion to 100 year cycle is somewhat