What Slow Fourier Transforms can tell us.
Guest essay by Stan Robertson, Ph.D., P.E.
On May 3, 2014, an article on WUWT by Willis Eschenbach entitled, The Slow Fourier Transform (SFT) was posted. As he noted, the amplitude of the Slow Fourier Transform components are in the same units as the fitted data, intervals of arbitrary length and irregular data can be used and periodicities rather than frequencies are automatically extracted. In addition to rediscovering a very useful mathematical tool, Willis went on to show that there were apparently no variations of temperature associated with solar cycle variations for several long term temperature records. Now my normal inclination would be to say that if Willis didn’t find any there probably aren’t many to be found. But, on the other hand, as I showed in an October 10, 2013 WUWT article entitled The Sun Does It: Now Go Figure Out How!, it does not take much of a temperature variation to represent a very significant solar contribution to ocean surface temperatures and heat content.
Several researchers, including Nir Shaviv (2008), Roy Spencer (see http://www.drroyspencer.com/2010/06/low-climate-sensitivity-estimated-from-the-11-year-cycle-in-total-solar-irradiance/) and Zhou & Tung (2010) have found that ocean surface temperatures oscillate with an amplitude of about 0.04 – 0.05 oC during a solar cycle. Using 150 years of sea surface temperature data, Zhou & Tung found 0.085 oC warming for each watt/m2 of increase of TSI over a solar cycle.
In my previous article, I showed that the changes of Total Solar Irradiance (TSI) over a solar cyle were too small, by at least a factor of 3.6, to cause temperature oscillations with an amplitude of 0.04 C. Since the variations of temperature considered were clearly associated with solar cycles, it seemed to me that the sun does something more to change ocean surface temperatures than just vary its TSI. But the whole idea would fall apart if there really are no significant variations of ocean temperature correlated with solar cycles. That motivated me to look in places where Willis had not and, in particular, to look at shorter and more recent temperature records that might be both more accurate and with better distribution over the ocean surfaces.
I downloaded the HADSST3 global sea surface temperature raw data (http://woodfortrees.org/plot/hadsst3gl ) and took a look at the data since 1954. This covers 60 years of data and about five and one half solar cycles. To get an idea of what sort of noise would be in these data, I fitted the sea surface temperatures to a cubic polynomial just to get rid of most of the systematic variations. The figure below shows a plot of the residuals for the last 60 years.
Figure 1 HADSST3GL residuals for the last 60 years
If we are looking for variations of about 0.04 C amplitude over the 5.5 solar cycles in the time period shown, then with apparently random variations of about 0.3 C amplitude in the record, the signal to noise ratio would be about 0.04 / 0.3 = 0.13. This would be a signal a long way down in the noise. So the question is, can we extract such a signal with a Slow Fourier Transform? To answer this question, I adopted Willis’ lovely SFT technique. I generated some test monthly data for a 60 year interval consisting of sine waves with a 10 year period plus monthly random noise in the range of +/-0.5 C. The slow FT results for waves with amplitude of 0.15 C, 0.1 C and 0.05 C would have signal to noise ratios of 0.3, 0.2 and 0.1, respectively. The results are shown in Figure 2.
Figure 2. Slow FT for test sine waves with 10 year period for a sixty year interval; 6 cycles.
As one might expect, the random variations would have both short period and long period apparent periodicities as shown in Figure 2. At a signal to noise ratio of 0.2 (blue line), or larger, the signal buried in the noise can be nicely extracted by the Slow FT. At a signal to noise ratio of 0.1, and none of the other curves to aid the eye, you might just have to believe that there might be a signal with a 10 year period. It is hardly bigger than the spurious noise peaks. Of course, there are much more sophisticated signal extraction processes than the Slow Fourier Transform. From comments that I have seen here on WUWT, there are some sharp readers around who could surely teach us some lessons. It might be expecting too much to see such a small signal in the noisy sea surface temperature data with an SFT method. But it is worth noting that in each of the test cases, the Slow FT peaks at 10 yr are smaller than the amplitudes that generated the test data by about ten to twenty percent with worse results at lower signal to noise ratios.
Since it is pretty clear that we will be looking for a small signal in a lot of noise, we probably ought to see where to look. A slow FT of the SIDC sunspot numbers for the years since 1954 shows a peak at 10.8 years as shown in Figure 3.
Figure 3. Slow FT for SIDC sunspot numbers 1954 – 2014
Now let’s have a look at the Slow FT for the sea surface temperature data. The average was subtracted to help suppress spurious long periods, but no smoothing was applied.
![]()
Figure 4 Slow FT for HADSST3gl sea surface temperatures
I leave it to the readers to decide whether or not there is a solar cycle signal in the HADSST3gl sea surface temperature record. Considering that the slow FT tends to understate the actual signal amplitude at low signal to noise ratios, I think that this might be a credible detection of a solar cycle driven temperature variation at a 10.4 year period with a signal to noise ratio of at least 0.065 C/ 0.3 C = 0.22.
For the remainder of this essay, I would like to extend and recapitulate some of my previous findings. The prevailing view in climate science is that the sun has contributed very little, if anything, to the warming of the last century. Finding that ocean temperatures are affected during solar cycles to a much larger degree than can be explained by the small changes of solar irradiance that reach the sea surfaces is a huge challenge to the prevailing view, but it rests on some bedrock physics. A detailed accounting for energy exchanges, including thermal energies is as fundamental as it gets.
I was able to account for the long term secular trends of both the sea surface temperature changes AND the ocean heat content since 1965 with a linearly increasing rate of surface heating. This involved numerically solving some heat transfer equations, including the absorption of solar energy, but it provided a simple, two parameter simultaneous fit to the sea surface temperature record AND the ocean heat content record. The two parameters found were a rate of increase of surface heat input of 0.31 watt/m2 per decade and an average thermal diffusivity of the upper oceans of 1 cm2/s. A fairly good fit to both trends was obtained as shown in Figure 5.
![]()
Figure 5. Measured and Calculated Sea Surface Temperature and Ocean Heat Content
A good fit was obtainable only for very narrow ranges of parameters. If the thermal diffusivity is taken to be too large, too much heat would be calculated for the ocean depths and surface temperatures would rise too little as the heat moves on to greater depths. If too small, the reverse occurs. If the input heating rate is too large, both rise too rapidly and if too small, both rise too little. The point of this exercise was to obtain a thermal diffusivity that could then be used to tell us how much surface temperature change could be produced by the changes of solar irradiance that occur during solar cycles. The answer is that the small variations of solar irradiance that reach the sea surfaces are far too small to produce temperature oscillations of even 0.04 C amplitude, much less the 0.065 watt/m2 amplitude suggested by Figure 4.
By the same computer program that I had used for my previous WUWT article, I have found that the amplitude of oscillating heat flux entering the ocean that would be required to produce surface temperature oscillations with the Figure 4 amplitude of 0.065 C would be 0.47 watt/m2 for thermal diffusivity of 1 cm2/s. How does this compare to the oscillating flux of solar radiation that reaches the sea surface? Let’s have a look at the solar irradiance changes over solar cycles. Figure 6 shows that TSI varies approximately sinusoidally over recent solar cycles with an amplitude of about 0.5 watt/m2 . (Thanks to Leif Svalgaard for TSI data.)
Figure 6 TSI variations for a few recent solar cyles.
As explained in my previous WUWT post, about 70% of one fourth of this amplitude, or 0.0875 watt/m2 enters the troposphere averaged over the earth area and day-night cycles. About
(160 watt/m2 /1365 watt/m^2) X 0.5 watt/m^2 = 0.0586 Watt/m2 is absorbed at the surface at wavelengths below 2 micron. About half the difference between the 0.0875 and 0.0586 watt/m2 reaches the surface at longer wavelengths and after scattering in the atmosphere. This give a solar TSI amplitude of 0.073 watt/m2 that is absorbed at the sea surface. This is about 6.4 times smaller than the 0.47 watt/m2 amplitude needed to drive surface temperature oscillations of 0.065 C. This result is in better agreement with the larger factors of 5 – 7 found by Shaviv (2008) ( see http://www.sciencebits.com/files/articles/CalorimeterFinal.pdf)
It is of some interest that my results were obtained without assuming any particular depth of an ocean mixing layer. For a thermal diffusivity of 1 cm2/s, the contribution to thermal gradients that vary with the solar cycle below the first ten meters would be much less than 0.001 C/m anyway. I saw no need to introduce a mixing zone with zero gradients and an arbitrary depth boundary.
This leaves us with a clear result that the TSI variations during solar cycles are not the direct drivers of the associated ocean temperature oscillations. Something else that varies with the solar cycles affects the amount of heat flux that penetrates the ocean surfaces. In my opinion, the most likely candidate would be cyclical variations of global cloud cover, but the mechanism that would control it is presently a research topic. Whatever the mechanism of the larger heating variations, it seems quite possible that it might be capable of producing long term secular trends under the control of the sun in addition to variations over solar cycles.
To examine this point, go back to the result shown in Figure 5. The heat flux required to account for the trends of increasing sea surface temperature and ocean heat content had to increase by 0.31 watt/m2/decade. Could this be due to greenhouse gases? CO2 is supposed to produce heating at a rate of about 3.7 watt/m2 per doubling period of its concentration. With concentration increasing at a rate of about 5% per decade, the doubling time would be about 14 decades. Since the heating effect is a logarithmic function of concentration, this would produce a linear heating at a rate of 3.7/14 = 0.26 watt/m2 per decade. This is certainly in the right ballpark to be part of the explanation of the apparent surface heating of the last few decades, however, when we recall that sulfate aerosols with negating effects would partially counter the CO2, it seems to me unlikely that CO2 is the entire explanation. Considering the similar period of rapid warming in the first half of the last century and the presently expanding and embarrassing pause of temperature increases, it seems to me that there is ample room for a significant solar contribution to the longer term warming periods. So I still think that the sun does a lot of it and I would still like to know how. Climate scientists would be well advised to spend some time trying to find out.
M Simon , you talk of “gun decking” whatever that means, and making the data up if no one was watching. What do you think is the likelihood of an engine-room reading being logged as bucket and vice-versa?
Pamela Gray says: July 26, 2014 at 11:39 am
Schrodinger’s Cat, you say in your proposal that subtle trends only show up after a long period of time, but one individual strong difference does not show up. How is that possible?
I’m saying that one or two very active cycles may not show up but a whole string of them do. Think of each very active cycle as a heating pulse. A string of them would cause heating.
When the solar activity is low, the heating pulses are very weak with a longer gap between peaks. Also, they allow GCRs and more albedo. A string of them causes cooling.
Bart says:
July 26, 2014 at 11:50 am
Some info on the “SFT“.
———————————-
Thank you!
” How is that possible?”
Low-pass filter
Typical low-pass filters affect earlier data as well as later data which can be acceptable but is not physically possible
A relaxation response, as a weighted integral, has low pass properties but also introduces a lag as can be seen here compared to low-pass and lagged SSN.
http://climategrog.wordpress.com/?attachment_id=998
I think this possibly what David Evans’ has picked and incorrectly interpreted as “notch-delay”. Finding a notch filter in nature is not easy. IMO he in misinterpreting his FT and is really seeing the presence of low-pass filtering, not a notch filter. The lag he is then having to invent to align the data is the lag that is created by a relaxation response.
The above plot shows a 20y relaxation is quite similar to an 11y low-pass with an 11y lag.
It is unlikely that the system can be accurately modelled with a single time constant response due to multiple ocean depths and other surface effects, each with a different response time. Some faster feedbacks are likely to be further attenuating the 11y periods.
However, for such a simplistic model it seems to catch general form of SST.
Here I show that tropical SST is very insensitive to changes in radiative forcing , this also means there will be little 11y signal visible. The cumulative effect may be mainly from SW component that penetrates deeper and bypasses the surface feedback mechanisms like evaporation and convection.
http://climategrog.wordpress.com/?attachment_id=884
Schrodinger’s Cat says:
July 26, 2014 at 1:14 pm
Think of each very active cycle as a heating pulse. A string of them would cause heating.
========================================================================
My thoughts have been similar in that regard.
No need to torture data to extract solar input.
I used NOAA global Land&Ocean data (see the other thread) and calculated its spectrum.
It is clear that the 11 years Sunspot cycle isn’t there, but the Hale cycle shows as the most prominent .
But before the ‘mechanism’ questions comes up,
Miss Gray, I forgot to bring my homework notebook, but it does say something about interaction between the Arctic summer atmospheric pressure and the Himalayan monsoons.
Thanks for a very interesting analysis, Dr. Stan. I just took a look at the dataset in question, and I fear I can’t reproduce your results. I find no significant signal of the type that you show. Here is my result:


I suspect the problem may be that you have subtracted a cubic polynomial from the data, which is a technique fraught with problems. If not, I’m not sure what you did … my analysis shows, for example that there is a 5-year cycle nearly as large as the 9-year cycle, and a 3.8 year cycle that is larger than the nine year cycle. In addition, your graph is much smoother and less detailed than mine. Again, I’m not sure why.
As a test, I ran a cross-correlation between sea surface temperature and sunspot numbers for the same period. Here’s that result:
Lag is in years, with positive lag meaning SST lags sunspots. Unless you believe that somehow the warming from the excess sunlight during the stronger half of the sunspot cycles is magically delayed between ten and twelve years, I’m not seeing anything there but random results. It’s a recurring problem with looking at something with a strong signal such as the sunspot data—you get an ~11-year cycle with almost any random red noise dataset you choose to compare it against.
Next, as a further check on the results, I looked at the period 1900-1950. There is no strong cycle of anything around 11 years in that earlier HadSST3 data, but there is a cycle at 14 years, which is obviously not solar driven, and no cycle at 9 years as in my results above … and the cross correlation is no better (although different) from the one shown above.
Overall, I’d say you are looking at artifacts, but I’m willing to be convinced otherwise …
Finally, the HadSST3 data is the average of 100 realizations of a computer model of the sea surface temperatures using slightly different assumptions. Such a procedure always makes me a bit nervous, particularly when the cycles are quite small. Details on the realizations are here.
Best regards, and thanks again for the work. I do love to see someone run the numbers themselves.
w.
So Willis, are you connecting with the internet on the ship or are you still landlocked?
My bad. I forgot you are attending a physicians’ conference related to emergency stuff.
bones says:
July 26, 2014 at 1:15 pm
See “The Slow Fourier Transform“. In a subsequent post, tamino identified what I’d done as:
tamino says:
May 26, 2014 at 7:57 pm
and I see no reason to doubt him.
It’s a great technique, because it ignores missing data and can accept totally non-regular data.
I’ve previously posted the R code for the functions, it’s here.
w.
Pamela Gray says:
July 26, 2014 at 1:57 pm
Indeed, I’m in Knoxville, Tennessee.
w.
The period 1950-2000 had a series of cycles which were high in SSN and highest in the duration of SSN>75 compared with all SSN cycles that were logged since 1750. The average SSN over that period was about 52.
It is probably fair to use SSN as a proxy for TSI since the turbulence caused by the magnetic field generates the higher energy radiation.
Long periods of low solar activity are associated with cooling, as you know. All I am claiming is that the solar effect on climate is multi-cycle and therefore multi-decadal and related to cumulative heating or the lack of it. A mixture does not register apart from possibly a small net effect.
Figure 3 and figure 4 should have same size and indent.
Willis Eschenbach says:
July 26, 2014 at 1:51 pm
Thanks for a very interesting analysis, Dr. Stan. I just took a look at the dataset in question, and I fear I can’t reproduce your results. I find no significant signal of the type that you show. Here is my result:
I suspect the problem may be that you have subtracted a cubic polynomial from the data, which is a technique fraught with problems. If not, I’m not sure what you did … my analysis shows, for example that there is a 5-year cycle nearly as large as the 9-year cycle, and a 3.8 year cycle that is larger than the nine year cycle. In addition, your graph is much smoother and less detailed than mine. Again, I’m not sure why.
—————————————————–
Willis, I just computed an arithmetic average SST from the monthly data, then subtracted it from the monthly values. I did not fit the residuals from the cubic fit. I only looked at the cubic fit residuals to get an idea of the size of the random variations. I suspect that the reason that my graph looks different might be that my least squares fits to sine curves with arbitrary phase were done for 0.2 year period increments. In addition, I did not use a log scale for periods on the graph. What I did was least squares fits to monthly data that I downloaded from woodfortrees.
sst = avg + A sin(2 pi t/P) + B cos(2 pi t/P)
and took the amplitude to be sqrt(A*A + B*B)
I used the entire data interval from Jan 1954 – May 2014 for each tested period, P, and I have just reported what I got. I agree that there is a significant 5 year cycle. It showed up clearly a while back when I tried to use Roy Spencer’s technique to look for a signal for the last four solar cycles.
It would not surprise me to find a delayed correlation such as the one found by Solheim et al., in which temperature trends correlated with solar activity and length of solar cycle delayed by a cycle period. On that basis, they predicted cooling for Norway during solar cycle 24 due to the short cycle 23.
It may be that I am finding artifacts. I will examine this point in some future calculations with your SFT. It is a pretty neat tool. I will probably use some 3 year averaging to suppress short period noise and some apodization to suppress side lobes and see just how small s/n ratios can still yield valid detections. Will also look at some of the long temperature records. It beats being outdoors in the heat here.
” I will probably use some 3 year averaging to suppress short period noise”
I do hope you are not contemplating using a running mean filter 😉
http://climategrog.wordpress.com/2013/05/19/triple-running-mean-filters/
You may find some useful alternatives here, as well as a script to calculate a relaxation response.
http://climategrog.wordpress.com/category/scripts/
Thanks for the links, Greg. I was thinking of using something more like a gaussian filter. Having taught optics many times, I am well aware of ways that the sinc function can bite you.
“It would not surprise me to find a delayed correlation such as the one found by Solheim et al., in which temperature trends correlated with solar activity and length of solar cycle delayed by a cycle period. On that basis, they predicted cooling for Norway during solar cycle 24 due to the short cycle 23.”
This kind of approach always seems rather contrived to me and success rather intermittent. Perhaps they are picking up a real effect of which the length is an indicator.
The minimum point is the cross-over between the tail of the last cycle and the rise of the new one. If the new ( current ) cycle is weak, it will take longer to pick up and the min date will be later.
I have already posted the lag-correlation showing about one cycle lag and proposed an obvious relaxation response as being the cause of the lag.
Greg Goodman says:
July 26, 2014 at 3:12 pm
. . . You may find some useful alternatives here, as well as a script to calculate a relaxation response.
http://climategrog.wordpress.com/category/scripts/
————————————–
Right on! An awk script! I used to use awk a lot and I still have a functional DOS version. I may be the last dinosaur, but I will take good tools where I can find them. Thanks.
I rather thought I’d debunked the SFT by running a simple sine wave through it and noticing sidelobes and other anomalies. Please just use a transform that’s been tested and validated for nearly a century – the discrete fourier transform, or its optimized version, the FFT.
http://wattsupwiththat.com/2014/05/26/well-color-me-gobsmacked/
Any signal transform should have:
(1) sine waves run through it
(2) Impulse function run through it
(3) step function run through it
(4) white noise run through it.
And verify the results are in agreement with the well-known spectral properites of those types of signals.
The SFT does not produce the expected output for a sine wave. This invalidates most of the analysis above.
Notes:
which comes from a file designed to be read by Willis’s SFT code:
https://www.dropbox.com/s/ieglk36hnxa430f/test-sin.csv
Source code: https://www.dropbox.com/sh/5wh9dbja6x37nfa/AADhfZFr6JXWF2vCDAj407Hoa
“…they predicted cooling for Norway during solar cycle 24 due to the short cycle 23.”
If there is a long term solar effect, it should have been dropping since before 1990 according to the relaxation idea but has been propped up by the volcanic effect. However, that current cycle is so weak ( as picked up by the late minimum ) that I think it’s going to start biting soon, Unless AGW saves us.
If there is not some clear cooling in the next 5 years, I may well be convinced that there is significant AGW.
“Right on! An awk script! I used to use awk a lot and I still have a functional DOS version. I may be the last dinosaur, but I will take good tools where I can find them. Thanks.”
Second last 😉
There’s a gaussian in there too. The Lanczos has a nice short transition band but has rather a long kernel so you loose quite a bit at each end. If that is not a problem its a very good filter.
Peter Sable:
In what way is that unexpected? What does FFT produce on that same sample?
Cross-correlation of tropical and ex-tropical Indian Ocean shows similar 9.3y ( anti-phase ) and 22y in-phase, with weaker ~11y. Similarly very strong 3.8y.
http://climategrog.wordpress.com/?attachment_id=779
Most SST records seem to show ~9y lunar being stronger than the 11y solar, and comparable to 22y solar. Failing to acknowledge the significant lunar signal has been the cause of many problems for those trying to detect 11y solar.
Schrodinger’s Cat says:
July 26, 2014 at 10:02 am
“I think the missing link could be duration of heating (or cooling) over several solar cycles.”
That idea is the basis for my solar model, based on HADSST3, where solar input vs HadSST3 reveals the idea you presented, as you explained well too. You and everyone else talking about this, the author of this post, many others, are converging on the truth. As SC24 goes by with more and more low SSN days, when the solar “mini-max peak” is finally over and we’re really into a long solar min, the OHC will give it up and SSTs will drop, as HadSST3 depicts every solar cycle min, and we will bear witness together as it happens again. This time the warmists will have nowhere to hide, nowhere to run, no way out except to abandon their ship of foolishness.
I really appreciate Stan Robertson for going to such great lengths to prove this relationship exists.