Guest Post by Willis Eschenbach
[UPDATE] In the comments, Nick Stokes pointed out that although I thought that Dr. Shaviv’s harmonic solar component was a 12.6 year sine wave with a standard deviation of 1.7 centimetres, it is actually a 12.6 year sine wave with a standard deviation of 1.7 millimetres (5 mm peak to peak) … I got 1.7 cm into my head and never questioned it because mm seemed way too small … but there it is. My thanks to Nick for pointing out my error.
So … in answer to the question, is a sine wave with a standard deviation of just under 2 mm detectable by Fourier analysis of the tidal station data … the answer is no. I’ve struck out the incorrect conclusions below.
========================================================================
I got to thinking about whether the Fourier analysis I used in my most recent post was sensitive enough to reveal the putative “harmonic solar component” which Dr. Shaviv claims to have measured. He said that he’d found a sine wave signal with a standard deviation of 1.7 cm mm in the satellite sea level record. So I added a solar signal with a standard deviation of 1.7 cm to the same 199 long-term climate records. [Ten times the size of Dr. Shaviv’s signal.] Note that unlike Dr. Shaviv’s so-called “harmonic solar component”, which was actually just a sine wave, I have used the actual sunspot record, and I scaled it to give it the same standard deviation (signal strength) as Dr. Shaviv’s sine wave. Figure 1 shows the “before” graph of the 199 tide station records from my last post.
Figure 1. The average of the station by station periodograms of the tide station data without the solar signal. All stations detrended before periodogram is calculated.
Figure 1 shows the actual tide station data. Notice that there is no signal at around 11 years. And here’s what it looks like with an added solar (sunspot) signal with a standard deviation of 1.7 cm, a mere 3/4 of an inch, the size of Dr. Shaviv’s claimed signal.
Figure 2. Average of the periodograms of all tidal stations with records longer than 60 years, to each of which have been added a copy of the sunspot signal scaled to a standard deviation of 1.7 cm. This gives a signal (sunspot data) to noise (tidal data) ratio of one part signal, seven parts noise.
So in answer to the question, can my method detect a signal of the strength claimed by Dr. Shaviv mixed into the maelstrom of the individual tide station records, the answer is clearly yes, no problem. It is quite visible.
Ah, but Willis, I hear you say … surely all of these tide stations wouldn’t be affected by the solar changes at the same time. And that is true, there might be lags that differ on the order of months, seasons, or perhaps even years between the forcing change and the response in a given location. But that is the beauty of my method of averaging the periodograms. The periodogram finds the signal regardless of the phase. The phase of the signal doesn’t matter in the slightest—if the signal is there, the Fourier analysis will reveal it. As a result, the lag at any individual tidal station is immaterial.
Let’t push it further. Let’s see if we can do twice as well, say a signal to noise ratio of one part signal to fifteen parts of tidal noise. That would mean a tiny signal with a standard deviation of 0.8 cm … bear in mind what I’m doing. I’m adding a tiny duplicate of the solar signal to the monthly tide data, with a standard deviation of only eight freakin’ millimetres, less than half an inch. None of the tidal records cover exactly the same time span, and many have gaps. So each record gets a different chunk of the sunspot data. So the question is … can the periodogram find a solar signal at fifteen parts noise to one part signal?
OK, here’s that graph.
Figure 3. Average of the periodograms of all tidal stations with records longer than 60 years, to each of which have been added a copy of the sunspot signal scaled to a standard deviation of 0.78 cm.
Yes, I can still see the signal. It’s clearest at the lower edge of the black error bar lines behind the gold graph line. But I’d say we’ve reached the detection limit for this size of signal in this size of dataset … one part signal to 15 parts noise, a detectability limit of a signal strength of 0.8 cm. Not bad.
Conclusions? Well, I’d say that if there is a solar signal in the sea levels, it is vanishingly small. And I’d also say that Dr. Shaviv’s claimed signal with a 1.7 cm standard deviation is large enough to be found if it actually existed … see Figure 1 and 2 to decide if you think it exists.
And finally, I hope that this puts an end to the claim that Fourier analysis can’t find solar signals because they have different periods from nine to thirteen years. It not only can do so, it can do so in the face of stacks of noise and with the solar data covering different periods and often broken by gaps in individual tide station records. Consider that some of the records look like this …
Figure 4. Detrended monthly tidal data, Sheerness.
Despite the gaps, we can find a signal with a standard deviation of 8 mm in the midst of tidal data like that if we have enough tide stations … not bad.
Regards to all,
w.
For Clarity: If you disagree with someone, please quote the exact words that you disagree with. That way we can all understand what you are objecting to.
I cant believe its not the sun, therefore you must have done something wrong.
because I cant change my mind, you must have made a mistake.
Yes…. Obviously…. It’s the SUN Stupid 🙂
Nope Seriously… Well done and well ferreted Willis, don’t let the buggers grind you down
I can’t believe it is the sun, therefore I’ll keep trying to prove a negative.
Because I can’t change my mind, it’s uncomfortable sitting on a fence looking for better data and methods.
What you think is frankly immaterial.
Fixed it for you:
I cant believe its not the CO2, therefore you must have done something wrong.
because I cant change my mind, you must have made a mistake.
Fixed it for you:
I can’t believe the CO2 matters much or its the Sun, therefore you must be selling something.
Because I can change my mind I’ll smell your ready meals but likely eat elsewhere.
It’s not just the co2. It’s methane volcanoes natural cycles the sun land use changes hfc and co2
I can’t believe it’s not CO2, therefore I change the data. No matter that my CO2’s central q channel, 50% of its radiative and absorptive capacity, is saturated. No matter that human emissions are a mere 3% of the carbon cycle. No matter if assuming ALL of the atmospheric increase is human, the human increment is less than 4% of the planetary greenhouse gasses. No matter that fossil fuel combustion produces twice as much WATER as CO2. No matter that there is no rational signal of CO2 in historic temperature nor any proxy evidence that CO2 EVER caused any warming on the planet. I can’t belive it’s not CO2!
Check the stratosphere. Where it matters co2 is not saturated. We learned this in the 50s
Outgoing radiation in the q channel does not get to the stratosphere. It does not even get to aircraft with sprctrophotometers.
Where it matters co2 is not saturated.
Then does that not falsify the assumption that CO2 causes any measurable global warming?
And where exactly does it matter?
Then does that not falsify the assumption that CO2 causes any measurable global warming?
NO.
1. There is no such ASSUMPTION.
2 The Hypothesis put forward as early as 1896 is this:
a) if you increase c02 and hold everything else constant the temperature will increase.
b) The effect is Logrithmic
c) The estimates of how much warming vary between 1.5C per doubling and 6C per doubling.
d) Saturation has not been reached
d1. In the early 20th century come scientists argued that saturation Had been reached.
d2. In the 1950s the US AIR FORCE flew missions in the stratosphere to Settled the matter.
d3. Settling this issue was of VITAL national interest.. read the cold war.
IF you want to have IMPACT and not be ignored read nic lewis. Your best argument is about sensitivity.
Notice that there is zero transmittance in the center of the CO2 spectrum. This corresponds to the Q channel shown below at wave numbers in the 660’s and it is about half or the energy of the CO2 spectrum. The “wings” of increased absorption at higher CO2 concentrations are very real in the higher and lower wave numbers to the left and right but they occur over a narrower range and at lower intensity than the central channel.
Steven Mosher says:
c) The estimates of how much warming vary between 1.5C per doubling and 6C per doubling.
Ok then, using the log effect, please extrapolate from the chart below. Show us how much warming would occur if CO2 was doubled from 400 ppm to 800 ppm:
As anyone can see, there would not be a 1.5ºC rise in global T, or anything close. It is very doubtful that any such warming could even be measured.
wrong again db.
1. you need to show your work,
2. Pre Industrial (1750) is taken to be 280ppm.
3. Double 280 is 560.
4. at 560ppm, you would see a 1.5C to 6C warming from the 1750’s value. PROVIDED nothing
else had changed
5. We are at 400pmm
6. Temperatures are roughly 1.5C higher than in 1750.
Steven M says:
1. you need to show your work
Did you miss the graph I posted? That’s my ‘work’. Please use the graph to extrapolate from the current 400 ppm to 800 ppm. Can you see how little warming that would add? It is far too minuscule to be measurable. Certainly, CO2 rising from 400 ppm to 800 ppm would not raise global T by another 1.5ºC.
The real world is verifying that: despite the rise in (harmless, beneficial) CO2, global warming stopped a long time ago.
And:
6. Temperatures are roughly 1.5C higher than in 1750.
Non-sequitur. You need to show your work.☺
In 1750 we were at the tail end of the Little Ice Age — one of the coldest episodes of the entire Holocene. The assumption that the 1.5º warming was due to rising CO2 is a conjecture. An opinion. Which you’re entitled to.
But if that 1.5º global warming was due to CO2, then we are still in the LIA. That blows Occam’s Razor to smithereens, doesn’t it?
Ockham says the simplest explanation is almost always the correct explanation. And the simplest explanation, corroborated by the graph, is that almost all of the warming effect of CO2 happened in the first few dozen ppm. That’s why no one has been able to produce a measurement quantifying AGW. Carbon dioxide has just too small of a warming effect at current concentrations.
Billy Ockham also said we should dispense with extraneous variables whenever possible. CO2 is an extraneous variable that only confuses matters. The planet is naturally recovering from the LIA. So forget CO2, it only plays an unimportant bit part. Simples, no?
Mr. Mosher appears unaware that the logarithm he refers to is a roughly roughly logarithmic DECREASE in radiative forcing with increased concentration. This is mostly due to saturation. He also appears unaware that the data on saturation comes from the HITRAN efforts he refers to by the air force for national security.
He clearly needs to go to the HITRAN website and download their little program.Mandatory curriculum for anyone in his position.
“Everything should be made as simple as possible, but not simpler.”
Albert Einstein
Comparing forcing to temperature is too simple. There are interesting questions being asked all the time. Here’s one: does solar control ocean heat transport?
http://link.springer.com/article/10.1007/s00343-015-3343-3
Phase shift will matter in the case of perfectly regular sine waves. If perfectly aligned with the sampling, your sine coefficient could be zero when cosine is maximum or vice versa. FFT outputs are mirrored with one side being the sine coefficent and the other side being the cosine coefficient if I remember right (more or less; but they call one side real and the other side imaginary but apparently means the same:)
http://www.mathworks.com/matlabcentral/answers/7475-fft-result-does-not-jive-with-theory-for-basic-sine-and-cosine
Apply euler’s equation.
I was thinking about this today. So many post math-cad people do their fft’s and if they even know the difference, put the result in amplitude format then throw away the phase information. Like it doesn’t matter. Half the information they just throw away. That’s how ignorant the fast in fft has made us.
Note that there was no reply to the question posted in your mathworks link. That’s because the question is ill posed. The sines and cosines merely represent projections of a complex signal ‘vector’ (complex numbers are isomorphic with 2D vectors) onto the X and Y axes of the plot. They are not properties of the data itself, just part of a 2D to 1D transformation of a rectilinear representation.
Not sure what you mean by ‘regular’ sine waves (‘regular’ and ‘normal’ are the two most overused adjectives in mathematics). But recall that any signal can be represented as a sum sinusoidal components (“Fourier series”). So it’s sinusoids all the way down.
Furthermore, the Fourier transform is shift invariant. Which means the absolute spectrum of frequencies does not change due time shift of the input signal. This is a very useful property of FT’s and does not hold in general for other transforms, such as wavelet transforms.
But recall that any periodic signal can be represented as a sum sinusoidal components (“Fourier series”).
But what makes sunspot data periodic? [Not a trick question and not because there are 11 year cycles.]
Michael 2 August 19, 2015 at 9:04 pm Edit
There are two ways of looking at Fourier. One is using imaginary numbers. The other is using the sum of sine and cosine waves, which was the method used by Fourier. I’m a practical guy, and although I can do the former, the latter makes more sense to me.
In either case, what we get from the analysis is both amplitude and phase information at each given period (or frequency). In one case the two variables (amplitude and phase) are imaginary and real components of an imaginary number.
In the case I prefer, the two variables are the amplitudes of the sine and cosine waves with the given period. A sinusoidal wave of arbitrary amplitude and phase can be described as the sum of properly scaled sine and cosine waves. So the two variables are the amplitudes of the sine and cosine waves, and from these we can calculate the resulting amplitude and phase angle.
In either case, the periodogram is unconcerned with the period. It throws away the phase information, and simply shows the amplitude of the sinusoidal wave with that period.
I use a brute-force Fourier method that I developed myself, and later found out (thanks to Tamino) was first described in the literature in the 1970s. To determine the amplitude and the phase of a Fourier component of a given period, simply do an iterative or other fit of a sine wave to the data. This has the enormous advantage in climate studies of being tolerant of missing data.
At first I used an iterative fit … sloooow. Far too slow. My big insight into doing it was to note that I could use a linear model of the form
data = a * sin(2*pi*t/P) + b * cos(2*pi*t/P)
where P is the period.
Using any linear model solver, say LINEST in Excel, or lm in the computer language R, set up sin(t) and cos(t) as the variables, and solve for a and b using your data. Then calculate a * sin(t) + b * cos(t), and calculate the amplitude. Done.
Of course, you’ll want to use a Hanning or Hamming or other filter on the data first, but there you are.
w.
Willis Eschenbach writes “My big insight into doing it was to note that I could use a linear model of the form
data = a * sin(2*pi*t/P) + b * cos(2*pi*t/P) where P is the period.”
Yes, that is how I do it (more or less) particularly when I have some idea what I am looking for since, as you point out, your datapoints don’t have to be regular and neither do your frequency bins. Also, the number of datapoints does not have to be a power-of-two as it is for FFT if I remember right.
But I think merely adding sine and cosine could result in cancellation of a signal in some circumstances.
I assume datapoints are distributed over y=0, half negative and half positive. If not, the algorithm raises the “floor” since it all looks like signal to varying degree.
Adding sines and cosines produces a peak (all points between 0 and about +1.41) when the signal being discovered is 45 degree phase shift relative to the sine term (pi/4). At 7pi/4 (135 degrees phase shift) the resulting addition of sine and cosine produces a sine centered on y=0 thus the sum will also be zero and you have cancellation.
Plot: sin(x) sin(x+7*pi/4) + cos(x) sin(x+7*pi/4)
The term “sin(x+y*pi/4)” produces the “signal” and sin(x) + cos(x) is the sampler or correlator sine and cosine wave.
As it happens, you can correlate with pretty much anything and that’s how GPS works. Each satellite talks on the same frequency but each has a digital pattern unique to that satellite. By multiplying the incoming stream of datapoints by a copy of the expected pattern you will get a peak when the pattern is phase matched to the very noisy signal. Phase is everything to GPS since it derives timing from phase, and from timing it derives location on Earth.
Similarly, if you find a signal at a period or frequency, you then start phase matching for a peak if you really want to know the amplitude of the signal. Vector addition will somewhat give you absolute amplitude but phase matching with just a sine wave is ideal for the purpose and substantially more sensitive. To actually do that you run multiple passes over the data with the same “P” but tweak “t” slightly each pass, effectively phase shifting the sine wave used for correlation.
Thanks for going the extra mile–er, centimeter, Willis. 🙂
Willis Eschenbach:
In your article you make repeated reference to a “signal” including: “is the signal detectable” and “despite the gaps, we can find a signal with a standard deviation of 8 mm…” You make reference to “noise” in stating that”…this gives a signal (sunspot data) to noise (tidal data) ratio of one part signal, seven parts noise.”
“Signal” and “noise” refer to concepts that are appropriate in the context of telecommunication but inappropriate in the context of control. Control (of Earth’s climate) is the objective of global warming research. Hence use of “signal” or “noise” is inappropriate in this context.
That use of either term is appropriate is a consequence of relativity theory. For telecommunication the signal must travel from the past to the future; this can occur at less than or equal to the speed of light satisfying a requirement of relativity theory. For control the signal would have to travel from the future to the past at greater than the speed of light in a vacuum but this would be impossible under relativity theory.
Terry,
Is this a joke? It sure reads like it. The problem being that you’re using two completely different meanings for “control.” I hope you don’t actually think political control (via a dubious appeal to controlling some aspects of the weather) is the same as control theory as a scientific enterprise.
Terry,
applications for the post of Director of Communications for the IPCC closed 2 days ago, which only 3 days ago was in the future by 1 complete diurnal sine wave function, but due to a transient time wave function , at a rate approximately equivalent to 1 day per 1/365 of a year (ignoring the minor 0.25 `noise` signal, now means the forward date has changed function, or possibly phase chnaged, to a backward function.
Good luck with the job search.
Love it.
Excellent. Don’t tell me you worked on Hijri date conversion too. Visibility is everything 🙂
Oldberg, you are completely wrong about the use of the terms ‘signal’ and ‘noise’. The term ‘signal’ is used by practitioners of data processing to denote any useful or informative waveforms found in a dataset. ‘Noise’ are the components (temporal or spectral) that are deemed to be not of interest, useless, or errors of process and observation.
Terry, what terms would you prefer were used in the context.
“He said that he’d found a sine wave signal with a standard deviation of 1.7 cm in the satellite sea level record.”
Is there a link or citation? It sounds like a lot, relative to the model results that were plotted, eg in Fig 4 in your original post.
Thanks, Nick. Sadly, you are right. I remembered the scaling parameter of his purported signal as being 2.5 cm, and never questioned it, because 2.5 mm seemed ridiculous … but now I’ve gone back to take a look and sure enough, it’s 2.5 mm.
This means that he claims that that satellite data are accurate enough to detect a sinusoidal signal with a standard deviation of 1.7 mm, not 1.7 cm as I’d believed.
Hmmm … well, time to edit the head post and point out that you found my error … then I’ll add some data and see if it is detectable in the satellite data.
Thanks for finding the error,
w.
Willis, from your article: ” bear in mind what I’m doing. I’m adding a tiny duplicate of the solar signal to the monthly tide data, with a standard deviation of only eight freakin’ millimetres, less than half an inch.”
8 mm’s is actually even less than 1/2 inch, it is even less than 1/3 of an inch ( one inch = 25 mm, so half would be 12.5 mms and 8 mms is even less than a third but I guess I am nitpicking here LOL) .
Thanks, asybot. Yeah, 8 mm is about 5/16 of an inch, but “less than half an inch” scanned better and was equally true.
Numbers are always trouble in a post. I figure for every number I put in I lose one reader … so I’d rather use something like “less than half an inch” rather than “about 5/16 of an inch”.
Plus which “less than half an inch” has more impact and is more memorable, while being just as true.
As you can see, I do put thought into these small details, particularly with numbers …
All the best.
w.
On global sea level :
its yearly rate of change looks like an irregular periodic swing as can be seen on this diagram obtained from a running average over 3 months, thus eliminating the yearly periods :
http://blog.mr-int.ch/wp-content/uploads/2015/07/gmsl_rise_rate.png
Source: John A. Church, Neil J. White, “Sea-Level Rise from the Late 19th to the Early 21st Century”, Surveys in Geophysics, Volume 32, Issue 4-5 , pp 585-602
and Jevrejeva, S., J. C. Moore, A. Grinsted, and P. L. Woodworth, “Recent global sea level acceleration started over 200 years ago?” (2008), Geophys. Res. Lett., 35
Satellites : Combined TOPEX/Poseidon, Jason-1 and Jason-2/OSTM
The coefficient of correlation R2 for the Jevrejeva series is 0.0555, which indicates that nothing is statistically significant to determine an acceleration of the rate of rise over a long period of time.
Sorry: over 13 (thirteen), not 3 months
And sorry again: for the Jevrejeva set the running average was calculated over 7 years (it contains only yearly data points)
Would be interesting to see a frequency spectrum of the Jevrejeva data.
Heads up…
http://www.bloomberg.com/news/articles/2015-08-19/weather-channel-said-to-hire-morgan-stanley-pjt-to-seek-sale
http://www.weather.com/science/environment/news/global-warming-weather-channel-position-statement-20141029
Could it be Jupiter? Heaviest of the planets. Orbital period 11.86 years.
Earth overtakes Jupiter roughly every 399 days. Seems to me that would have a larger impact than Jupiter’s own orbit.
Agreed but that effect could ‘hidden’ in the peak at 1 year.
I’m just floating idea not necessarily arguing that it is true.
Distance varies between 4 and 6 AU every 399 days, but gravity pull is very small. Both, the Earth’s and Jupiter’s magnetospheres are regularly wrapped by the solar ‘magnetic ropes’
http://i.space.com/images/i/000/043/034/original/solar-eruption-model.jpg
The Earth’s 399 day transit is picked up by the N. Atlantic SST oscillations.
Vuk, citation please.
The Earth’s 399 day transit is picked up by the N. Atlantic SST oscillations.
====================
why not post to WUWT showing the data and analysis? it would throw the large body of science that believes there is no effect on its ear. seriously.
No, unless the data is very messy FT would resolve 12mo from 13mo. However, such a signal would be totally removed by the ever popular 13mo runny mean.
The tidal effect of Jupiter is almost certainly negligible, so I would not expect to see anything at around 396d from Jupiter. However J does affect the perihelion distance of Earth from the sun, and that is related to the jovan year of 11.85y. It is also implicated in the complex lunar “months” of 27.323, 27,545 and 29.5 days and the moon is the primary mover of tides.
Jupiter’s relationship to those periods is fascinating but far too long for an elevator speak presentation.
Mr. FB
I have been looking at the CET and and the Earth’s magnetic field for number of years now, data such as they are show indisputable nonstationary correlation, (two data sets timings are ‘intertwined’)
http://www.vukcevic.talktalk.net/HmL.gif
which is hard to explain. Detailed analysis of the N. Atlantic SST – Arctic link reveled strong ~400 days factor. I know of only one reason for it, and it is unlikely to be tidal gravity pull, since it is almost negligible compared to the lunar one.
I will write full description hopefully get it published, then anyone would be able to reproduce the results.
“it would throw the large body of science that believes there is no effect on its ear.”
I doubt it; it will be declared just another coincidence and duly ignored.
Vuk’, interesting as ever but not much use until you either do publish or provide enough information about the data source for someone else to look at it.
How significant is your R^2 value with such heavy filtering? Does it look any better with a proper non distorting filter. You are using a crappy running mean that will invert part of the data and lets a lot of LF noise through. Try better filter ( eg. gaussian ) with shorter period like 10y.
Your delta(Bz) is basically a 30 running mean on the rate of change as well.
What is the R^2 without the filter, that is the first thing to report , with an estimation of significance for the number degrees of freedom.
The number of degrees of freedom with a 30y filter will be quite low, so you need to estimate significance there too. 0.95 may not be that impressive. It’s not much use quoting r^2 values on their own.
You presumably know that but I don’t recall you ever showing a significance value with your results.
This is interesting, please do it more rigorously.
a quick look at CET with a 120mo triple running mean ( 120, 104, 74 mo ) has a peak around 1730 that better matches you Bz ( wherever that comes from ) and is far “smoother” than your 30y ma.
Hi Mike
Thanks for the observations, the above graph is the reason I decided to look further. Delta Bz comes from the Arctic, and since the CET is highly responsive to the N. Atlantic SST, the above correlation however crude, was the trigger for the further research, which revealed strong ~ 400 day factor in the AMO – Arctic link (data from NOAA), in which neither the CET or Bz feature as such.
p.s. for filtering I often use Butterworth low pass filter.
Vuk.
Please explain what your N Atlantic hydro magnetic loop is.
Tx
Well I wish I could.
Ryskin’s paper claims that ocean circulation in the N. Atlantic modulates the Earth’s magnetic field.
http://iopscience.iop.org/1367-2630/11/6/063015/pdf/1367-2630_11_6_063015.pdf
On the other hand ocean currents are saline electric conductor, aby change in the magnetic field could affect velocity of the circulation. If so one could speculate about existence of a feedback loop between saline water circulation and magnetic field change, hence: hydro-magnetic loop.
Just the idle mind’s speculation.
The linear regression dots distribution for the Loehle’s global temperatures (right hand portion) clearly suggest some kind of a feedback in action
http://www.vukcevic.talktalk.net/LGt-Bz
http://www.vukcevic.talktalk.net/LGt-Bz.gif
For time series analysis Butterworth is not a good choice. Butterworth filters are used for frequency domain analysis. For time domain analysis Butterworth filters have undesirable effects such as impulse responses, long-distance correlations, and so on.
Also you have edge effects you need to deal with. Do you use reflection, extension, zero padding or something else?
Good discussion here: http://climateaudit.org/2008/10/07/are-butterworth-filters-a-good-idea-for-climate-series/
If you post the source code we will stop asking you questions and provide you constructive feedback to improve your analysis.
Peter
Mr. Sable
Thank you for your remarks. There is nothing there I am not aware of; Butterworth filter’s +s and –s are well known, so are methods and length of padding necessary (I use ‘trend function’ for both back and front),
Having all that in mind it is easy to use, fast and satisfactory for the most of practical purposes including time data series.
Here is what crutem4 end looks like
http://www.vukcevic.talktalk.net/BF.gif
Now you can write your own code if so inclined
I google “trend function” and I get Excel’s linear trend function that returns a line. Your padding certainly does not look like a line.
So once again, you leave us guessing, instead of just publishing source+data+tests. It just means we don’t believe you for the most part. It’s bad rhetoric style in the 21st century.
Also, you typically test a filter function with known signals such as step, impulse, ramp, pink noise, sine wave, etc, not crutem4. crutem4 may or may not contain the signal components that those test signals have that validate edge conditions, step responses, etc.
Peter.
It is very simple; for each data point that data is extended, trend function is the linear trend for the previous 10% of the total data points in the data set. Since Crutem4 has ~160 data points it is the linear trend as determined by the previous 16 data points; it eventually ends in a strait line. For the data start it ‘TF’ works in reverse.
Jupiter makes the Sun wobble:
From memory, looks like the Sun’s position changes by one Sun diameter, varying the Earth to Sun distance by about 1%. It seems reasonable that this distance change could effect the Earth/Moon system.
The sun and the planets all orbit around the center of mass of the solar system (mass centroid). And this centroid wobbles due to the changing distribution of planetary mass. Jupiter of course is the primary player. The planets cause tidal waves to develop within the sun, just as the sun causes tidal effects on the planets. These tidal perturbations are thought to be the underlying cause of the 11 yr solar cycles. (From an Isaac Asimov book, The Stars In their Courses, that I read decades ago… )
-Brian in central Nevada
This is a good example of a Lorenz oscillation.
Since radiation drops off as 1/r^2, does this mean that the incident radiation from the sun drops by (1/86M – 1/(86M*0.99)^2 = 2%? At 340W/m^2 that’s a fluctuation of 6.8 W/M^2, which is pretty significant. It should show up in some measurement somewhere..
Peter
Willis, this isn’t quite a kosher way to test whether your method works. You need to add the cycle to data you know for sure doesn’t have one, then see if your method picks in up. If you add it to the tidal gauge data you are assuming the tidal gauge data doesn’t have a cyclical signal in it. You may say “yes, but I already showed that it doesn’t” with your previous analysis, which ordinarily would be fine except you are testing the method you used in your previous analysis.
Here’s a suggestion for the right way to do it: create synthetic data with similar characteristics to the tide gauge data, then add a cycle to the tide gauge data. If you’re right, this shouldn’t change the result, but since whether or not you’re right is the very thing you want to test, that’s what you need to do.
You could always use the NOAA methodology. If something is not there that you want to be there, such as warming oceans, you just change the data so it is there (in the data as least), then say -“oh look what we’ve found”.
I don’t understand why you think Willis’ method isn’t kosher. If there is an underlying signal, the addition merely adds to it. If there isn’t, it just adds itself. It’s pretty clear from what Willis did that there’s no signal of the type and amplitude that Dr. Shaviv claims. There should be a bump. There’s no bump.
D.J. If there’s no signal there isn’t a problem, except that you can’t test something by first assuming that it’s true, except by contradiction.
The reason it isn’t kosher is that if the signal is present, but not detectable which is the question we want to test whether or not we can reliably answer adding a signal on top of a signal artificially lowers the threshold below which we conclude a signal isn’t detectable.
Look at it this way. Willis wants to know “how large could a solar cycle signal be and my method give a false negative result as to it’s presence?” To answer that we need to take data we know doesn’t have any such signal at all, and add the cycle we want to detect in varying magnitudes.
We have N + C where N has no cycle in it, and C is a cyclical component added to it.
But the test Willis actually ran was U + C, a series which may or may not have a cyclical component. It therefore consists of a series N which doesn’t have a cycle in it and a series kC where k is a number between zero and one inclusive.
Let’s say for the sake of argument that the actual data do contain a signal that’s, say, 30% the size of the signal Willis artificially adds the data. Willis concludes when it detects a cycle that it can detect a cycle of magnitude C, but actually, he’s found that it can detect a cycle of magnitude 1.3C! And say he finds that the signal has to be at least of magnitude .5C to be detected-no, what he’s actually found is that it can detect a signal as long as it’s at least .8C in magnitude!
These numbers are arguendo. If k happens to be zero, there’s no problem! But that k is zero is the proposition Willis wishes to know whether his test can successfully…test!
Willis has made a mistake that exaggerates the power of his statistical test to avoid false negatives.
I’ve multi-linear fitted the Brest data to a detrended Fourier type series and do not find a statistically significant cycle around eleven or twelve years. Cycles at one year, half year, 65 years, and 3.1 years, were statistically significant, in descending order.
I can hear the data screaming … UNCLE
I like this kind of trial-and-error approach to obtaining detection thresholds, equations exist to give quick and easy estimates, but my ex-employer is likely to get burned shortly by promising something based on equations, but that in reality cannot be achieved.
I don’t think anyone said Fourier methods would not work, just that they are a bit sub-optimal due to the varying period, which spreads the signal energy over multiple Fourier cells, i.e. the peak cell amplitude will be less than for a pure sinewave, a loss that can be avoided.
Good point with respect to varying periods and FT. I would do a scatter plot of sea level rise versus SSN and see if there are any hints of correlation.
I like the use of tide data for this analysis. Some have complained that there is a seasonal signal that is creating too much noise, some complained about using the rate of change of sea level but what this shows is how insignificant the signal must truly be compared to everything else that the tide gauge is measuring.
Thanks to everyone on both sides of this ongoing thread for an interesting exercise.
Tide gauges measure Relative Sea Level. That is, they measure the local sea level relative to the nearby land surface. That relative measurement is affected by things like the rebound of the Earth’s crust after then end of the last ice age (Glacial Isostatic Adjustment), tectonic uplift, self attraction, loading, subsidence, etc. If you just average together a collection of tide gauge data, then the result is not the Global Mean Sea Level (GMSL). See the University of Colorado’s Sea Level Research Group FAQ: Why is the GMSL different than local tide gauge measurements?
When different research groups use the tide gauge data to estimate GMSL, they come up with significantly different answers. The University of Colorado’s Tide Gauge Sea Level page lists eleven different result for estimates of the GMSL. They range from 1.2 mm/yr to 2.8 mm/yr. The differences result from different ways of considering vertical crustal movement, the number of tide gauges used, and the length of the tide gauge records. Basically, estimating GSML from tide gauge data is a very involved and very tricky business.
The tide gauge data presented in this post are not an estimate of Global Mean Sea Level. Therefore, the tide gauge data presented in this post cannot address the question of whether or not there is an 11 year cycle in the GMSL.
Willis,
The steric plus eustatic effect claimed by Howard, Shaviv and Svensmark amounts to a total amplitude of 2.4mm, repeat MILLIMETERS, over the solar cycle. The total peak to trough change is less than 5mm. If you convert that to a standard deviation I would estimate something close to sd = 1.2 millimeters. I cannot therefore see how your sd = 1.7 centimeters is derived or where it fits in. It looks way too big for a detection test of the solar cycle.
As I explained in the previous thread, you cannot find the solar cycle in the data unless you apply a reasonable methodology to eliminate the massive seasonal variation. Once you have filtered out the very high frequency variation, the solar cycle (and other multiannual cycles) does then become visible with a variety of methods of spectral analysis.
You should perhaps be asking yourself why the well-documented 18.6 year lunar-tidal cycle is not visible in your data (either) – especially in individual local records where it should show up as a notable feature with higher amplitude than solar cycle effects.
Indeed, 18.6y is one of the first things I was expecting to see. It was certainly pretty visible in the Brest data from the last post.
Paul, can you remind me what the freq response is for the annual monthly difference you suggested? I seem to recall it is some odd notch filter. #
I agree with using a difference to detrend the data and remove autocorrelation, but I’d be wary of this annual difference. More inclined to use a low pass after the diff. ( or a diff-of-gauss to do both at once ).
Just had a quick look at the power spectrum of rate of change at PMSL data for Honolulu, that is I did a year to two ago.
There is a clear , though small, peak at 10.8 years and another similar one at 20.4y a much stronger one at 5.38y. Obviously the diff will be attenuating the longer periods.
Clear 10.8 in PMSL Ijmuiden ( Netherlands ) too.
I think Willis needs to look at why he is not finding what should be there.
very clear 10.7 and 18.1y in Hoek van Holland too.
These results were obtained by using a low pass filter to remove the annual and sub-annual variations from the monthly rate of change as I suggested above.
@Mike,
Paul, can you remind me what the freq response is for the annual monthly difference you suggested? I seem to recall it is some odd notch filter. #
Yes. It applies a crude band-stop or notch filter to the time-differentiated series to reduce the obscuring effect of the very high amplitude intra-annual change in tide levels. It also has the less desirable effect of frequency-weighting the amplitudes of the original series which causes relative attenuation of the higher wavelengths in the subsequent Fourier analysis, but the quasi 11 year cycle is still visible. (cf your Honolulu analysis)
A low pass filter is probably better. Additionally there are empirical spectral analysis methods which allow for some variation of periodicities within a bandwidth. But Willis is a do-it-yourself kind of guy and I did not know whether he had ready access to signal processing software. I wanted to give him something which he could very easily test for himself with tools to hand.
I hope that your results prompt Willis to re-think what he is doing here.
Mike,
My first response to you vanished into the ether.
Yes, it is a crude notch filter or bandstop filter applied to the differentiated series. It has the unfortunate effect of attenuating the longer wavelength amplitudes in relative terms, but it is still sufficient to render the solar cycle visible.
A low-pass filter is probably better. Additionally, simple Fourier analysis is not necessarily the most sophisticated way to isolate the quasi-11-year signal when some variations in periodicity and amplitude are expected. But Willis is a do-it-yourself kind of guy and I did not know whether he has access to more sophisticated signal processing software.
Willis –
You make good points. You did exactly the right test. It is a misconception to suppose that the Fourier Transform is trying to “detect” a frequency component. It is displaying a signal in an dual domain. Often this is useful. But it is not going to “dig out” a tiny signal. If you don’t “almost see” it in the time domain itself, don’t expect anything overwhelming in the FT. Noise is noise.
Further, given that most “Fourier Transforms” are achieved with an FFT and this involve sampling, end effects (“windowing”) and many issues with resolution and so on, it is easy to get fooled. That’s why most of us who use FFTs run and rerun “toy” examples before even loading any real data.
Bernie
I’m assuming that Willis is using his “slow FT” , FFT requires continuous evenly spaced data and that is not what is being used. He should be more explicit about what “FT” means.
Mike –
I think Willis developed wnat he called a “slow Fourier transform” as a kine of etemplate matching to a sinusoid. But there is no traditional “slow Fourier transform” This is all there is:
http://electronotes.netfirms.com/AN410.pdf
Bernie
Yes that’s it. I did say “his” SFT. It seemed to give results compatible with FFT but just took a lot longer. The huge advantage being that it does not require continuous, equally spaced data like FFT does.
Also since it does not effectively wrap the end back onto the start, it does not require the usual window functions, which are themselves distorting.
I’m only guessing that’s what he did, he really should have explained what he was doing in the article.
Willis fixed his lack of windowing a while back after I and a couple of others nagged him… AFAICT his “slow FT” is quite robust at this point. I independently developed a matlab/octave model and my test data agrees mostly with his test data as well as agreeing with my experience in signal processing. (sine wave, step function, ramps, impulse, pink noise, gaussian noise… am I missing a test?)
Proper software engineering methods would require posting the source every time for every submission, and the test results should be available in that post of course. Willis didn’t do that, so I can’t tell if his code is still robust for this analysis (it was a bit messy thus easy to introduce new bugs )
I continue to be amazed that the signal processing folks and stats folks don’t talk more. Linear interpolation and filtering can easily fix continuous data “problems” with FFT. (Heck if you’re careful zero padding can do it as well if you are only interested in low frequency components, it’s the default in matlab/octave). Equally spaced data is a “depends” but is also addressable with conventional signal processing techniques. FFT is used by signal processing folks, and stats folks use oddball (to me) tools like the SFT, loess filters, etc.
Peter
Peter Sable writes “I continue to be amazed that the signal processing folks and stats folks don’t talk more.”
I get quite a bit out of this conversation.
Peter said:
“Linear interpolation and filtering can easily fix continuous data “problems” with FFT.”
Really! Interpolation adds NO actual information. You must mean something else.
Please do elaborate.
Bernie
Bernie:
You say, “But there is no traditional ‘slow Fourier transform'”, but even the link you give shows the “discrete Fourier transform” as the standard transform for a sampled data series, with the FFT as just a more computationally efficient version of this.
Compared to the FFT, the DFT is exactly a traditional ‘slow Fourier transform’.
Curt – Not at all !
The DFT is just N-equations in N-unknowns (often way too slow in practice). Please read the notes that follow my “map”. The FFT is a Fast algorithm. They both compute the EXACT same RESULT – one is just obtained faster.
And I think the DFT is unrelated to the procedure Willis calls a SFT.
Bernie
Precisely!. There is no information to be added, it’s missing, and you don’t want to add any, that’s called “drylabbing”. I hate drylabbing, it’s one of the reasons I think the CAGW folks are wrong, they do it all too often.
However, FFT/DFT does require a continuously sampled function, it’s the nature of how it works.
So to fill in empty points to satisfy the requirements of FFT, you generally have the following options. Keep in mind the requirement is to add no information to the PERIODOGRAM that wasn’t there already, while also not removing any useful information. You can modify the time domain signal in a manner where it doesn’t affect the periodogram in a significant manner. Tricks generally derived from FT properties:
https://en.wikipedia.org/wiki/Fourier_transform#Properties_of_the_Fourier_transform
(1) just assume the points are zero, then low-pass filter to remove the resulting signal/energy spike. This has problems because it introduces significant anomalies that are proportional to how far the signal is from zero. You can move the filter to a lower frequency to compensate, but that can start destroying useful information elsewhere in the signal. Unfortunately this is the default in matlab/octave for e.g. its upsample functions.
(2) linear interpolate. This introduces a fairly small high frequency blip that is easily low pass filtered with a filter that doesn’t destroy information in the rest of the signal. I tend to use this method, it’s fairly fast.
(3) spline or polynomial interpolate, likely don’t need a low pass filter. I have found I get almost no different result between this and linear interpolate+filter.
For those of you who might have noticed, I insist on source code to back claims. I’m working on cleaning up the above observations, I’ll publish one of these days… trying to get the same data through R and octave is driving me nuts (I’m very new to R), and a lot of the stats functions aren’t available in octave, only in R.
Also Willis stop publishing his source code and he writes in a style where it’s hard to test. So just testing his code and comparing to other related methods has its own headaches. That’s how I found his (lack of) windowing problem which he subsequently addressed. (though to my eye I can see a sync smeared pretty easily just by looking at the graph).
Peter
Peter – thanks for that. I think we would agree completely if standing around a chalk board coffee cup in hand. I like the “how is the FFT trying to fool me this time school”! Of course, “subtle but not malicious” applies. Best wishes.
Bernie
Not in the 11 year(no signal will be apparent) so called sunspot cycle because it is obscured by noise and many other climatic items that run counter to the solar activity.
In addition the solar activity ‘s duration of time when in a typical 11 year cycle is way to short and cancels itself out.
Besides typical 11 year sunspot cycles are going to act to keep the climate stable not unstable.
Willis is entitled to write about any solar study he wishes but this particular one is a big waste of time.
Simple analyses disprove consensus assertions.
Proof that CO2 has no effect on climate and identification of the two factors that do cause reported average global temperature change (sunspot number is the only independent variable) are at http://agwunveiled.blogspot.com (new update with 5-year running-average smoothing of measured average global temperature (AGT), results in a near-perfect explanation of AGT since before 1900; R^2 = 0.97+).
Willis,
Thanks for continuing on this topic with another post. I think the most valuable part of this is the discussions that have sprung redarding the meaning (or lack thereof) of this signal if it is there.
I am a bit confused though. I don’t know where you came up the 1.7cm number for standard deviation??? Are you still talking about the paper entitled “The solar and Southern Oscillation components in the satellite altimetry data” by Howard, Shaviv and Svensmark?
I’ve searched this paper carefully, and the number of 1.7cm is nowhere to be found. I agree with you that a sine wave (or your suspot data) with a standard deviation of 1.7 cm would be easily detectable. Did you make a mistake on units — perhaps changing mm to cm?
In the paper’s equation (1), table (1) and also in figure (2) the claimed solar component is clearly stated to be a 2.5 MM peak sine wave so it will have an RMS value (roughly the same as standard deviation) of about 1.76 MM. Your signal with a 1.7 CM standard deviation is equivalent to a sine wave with a 2.4 CM (or 24 MM) peak value.
I do agree that a signal this large (24 mm peak sine wave, or equivalent sunspot data) would be easily detectable. But that’s not what the quoted paper is claiming is there — they are only claiming a 2.5 mm peak sine wave (equivalent to 1.76mm standard deviation).
So unless I’ve made my own mistakes here (that’s not impossible either), the bottom line is this:
It is not possilbe to determine the presence of the solar signal in the above quoted paper using the indicated tide gauge data. It might be there, it might not but you cannot tell by looking at tide gauge data.
In an article that I posted last year, I did a Fourier transform of HADSST3gl sea surface temperatures http://wattsupwiththat.com/2014/07/26/solar-cycle-driven-ocean-temperature-variations/, Fig. 4
?w=762&h=360 and found a variation of about 0.13C peak to trough over the last four solar cycles. I also found that it would take about 0.62 Watt/m^2 peak to trough on a global average to produce that level of temperature variation. That level of solar heating variation is very considerably more than the averaged variation of solar irradiance at the earth surface during a solar cycle. Nevertheless, the thermosteric contribution to sea level rise would only be about 1.7 mm peak to trough. ( http://wattsupwiththat.files.wordpress.com/2013/10/clip_image005_thumb1.jpg?w=936&h=669 integrated ) This would likely be lost in the noise of individual tide guage records, but might show up in carefully averaged data, as in Holgate’s work http://i1244.photobucket.com/albums/gg580/stanrobertson/sealevel3_zpsc79e6748.jpg
To be honest I would say that NIr Shaviv has a good case for libellous content, I hope Willis Eschenbach has a good lawyer.
OK, I can see it now: “Yer honor, he bludgeoned my hypothesis with a blunt transform.”
The detectability of any signal of a given variance depends upon the signal-to-noise ratio. With tide gauge records, where the least-count-unit round-off noise is typically more than a centimeter RMS, the prospect of detecting a stochastic signal of a few millimeters RMS is slim to none.
Agreed, your post (which is more concise) crossed with my comment below.
Anyone who has been involved in shipping and has first hand experience in obtaining/assessing accurate draft information for ports/port approaches/river estuaries etc, the ballast/laden draft of vessels, would readily appreciate the difficulty (I would say absurdity) of hoping to ascertain a small signal pertaining to a claimed variation of 1 to 3 mm annually from what is noisy data.
Leaving aside the tectonic movement and isostatic rebound problem, how does one know that there has not been minor settlement of the structure on which the tide guage is attached (how good is the foundation into the seabed?), or silting in and around the relevant datum point. How many tide gauges are scaled in millimetres. Certainly not those put in place 50 years ago!!
How do you know that the measurements are taken at precisely the same point in the tide cycle (not simply daily, but also lunar & seasonal changes of exceptionally strong neap tide or spring tide. What TOB adjustments are being done ? After all TOB is claimed to be extremely material to the proper assessment of temperatures and the global anomaly assessment.
How can a satellite which measures a continuously moving surface (which movement can amount to 10s of metres depending upon swell and storm conditions over the oceans) realistically measure to a millimetre accuracy? Even measurement to a fixed land point to millimetre accuracy is difficult, but the surface of the ocean; come on, someone is having a laugh when they claim to be able to measure change to within 1 to 2 millimetres.
Further, what is the annual variation in rainfall? How much is annual variation in rainfall a reflection of the amount of solar insolation being absorbed by the oceans and/or minor differences in the amount of evaporation from the oceans?
Before embarking on an analysis of this kind of data, one should first conduct a quality audit of the data, to see whether anything useful could be found from the data given its quality issues.
Willis
You often rhetorically ask whether something passes the ‘smell/sniff test.;
Like most data sets in Climate Science, the tidal data sets are extremely noisy and have huge error margins, and when coupled with tectonic movement and changes in the ocean bed render it impossible to eek out a small signal.
Given the extremely modest variation of TSI over the solar cycle, why anyone would expect to be able to eek out the signal from the noise is beyond me. Quite simply looking for this solar cycle signal in the sea level data, does not pass the smell/sniff test.
Further, as far as I understand matters, those who consider it probable that the sun is a major player in natural variation on a long term basis (ie., one measured on say a centennial basis if not longer), do not argue that the small changes in TSI over the solar cycle are the drivers for climate change on a decadal basis, It is the prolonged and cumulative effect of a ‘quiet’ or ‘strong’ sun which is claimed to drive temperature/climate change over a multi-decadal basis (aided by ocean cycles and the amount of energy that solar has imparted into the oceans over the long term).
What signals are detectable in CruTem4:
http://www.vukcevic.talktalk.net/CT4-Spectrum.gif
4-5 years: ENSO
9.1 years: N. Atlantic SST (AMO)
21.5 years: Solar magnetic field (Hale cycle)
26.6 years: Asian monsoon
60.2 years: N. Atlantic SST (AMO), caution there are 164 years of data; 60 years is on the border of acceptable certainty.
Willis
You ask : “Is The Signal Detectable?”
Any answer to that question will depend upon the quality of the data that is being used. If you wanted to answer that question, you should first have carried out a quality audit of the data that you intended using. Poor quality data cannot produce a silk purse, one is left only with a sow’s ear. A quality audit of the data, would have quickly have led to the conclusion that the data cannot reveal anything of significance, because the real error bounds are larger than the signal being sought.
The reason why no one knows the sensitivity to CO2 is because of poor quality data. Nearly all the data sets in climate science are not fit for purpose such that no proper science can be carried out on these data sets.
There is probably only one data set that is acceptable and that is the Mauna Loa CO2 set. However, that does not clarify what the CO2 levels were prior to the start of the series, nor whether CO2 is a well mixed gas such that Mauna Loa data truly reflects global concentrations. The recent OCO 2 data (and the earlier Japanese satellite data) suggests that CO2 is not as well mixed as climate scientists would have one believe. Indeed, in essence the reason for rejecting the past chemical analysis of CO2 data (ie., the MBL estimation 1826- 1960 from directly measured data (Beck 2009)) rests upon CO2 not being well mixed.
Provided that the satellite temp data is properly and accurately calibrated against radiosonde balloon data, that data set is acceptable, but the duration is rather short. That said it is clear that there is no first order correlation between CO2 and temperature in that data set.
ARGO, has the potential to be good, but it is of too short a duration, and lacks spatial coverage such that the margins of error when assessing sea tem data globally is far larger than the 1/1000ths to which they claim that they can measure data.
The rest of the data sets, for want of a better word are garbage. This extends to sea level data.
The need for a proper quality audit of the data being used is the reason behind the surface station review. Unfortunately so many papers in climate science are being published on extrapolation and interpretations of p*ss poor data such that the conclusions of the paper are not worth a damn!
Here is the power spectrum for Honolulu tide guage as archived by UK’s PMSL service
http://www.psmsl.org/data/obtaining/
Data goes back to 1905
rate of change of sea level was used to remove the general upward trend and it should be noted that this attenuates the longer periods. A low-pass fitler was used to remove the annual and sub-annual signals.
Looks to be a fairly clear solar signal there. 5.36y is the first harmonic and 2.57y may also be the 2nd. These harmonics will be present since solar cycle is not a nice smooth sine wave, they are a result of it’s non symmetric shape.
It would seem that Dr Shaviv may have got a stronger result had he used the actual solar data instead of a simple sine wave approximation.
Willis,
I feel like throwing my hands up and screaming “aaaaagh!”
Your mini-series on this subject together with your last correction/update has now left a bunch of people with the entirely erroneous conclusion that a quasi 11-year cycle cannot be detected convincingly in tidal records.
Have a look again at the short series of posts from Mike August 20, 2015 at 9:53 am.
A low-pass filter applied to the level series renders the 11-year cycle visible in several of your chosen records..
Alternatively, generate the annual differences from the uncorrected monthly data and apply a Fourier analysis to the differentiated series. The 11-year cycle again becomes visible.
In fact, apply any reasonable filter to the high frequency (largely gravity-induced) annual variation any darn way you please and then tell us that the cycle is “not detectable by Fourier analysis”.
(Douglas 1997 “Global Sea-rise: a Redetermination” applied a more rigorous screening of tide-gauge data than just continuity. You might also consider testing his subset before declaring that the cycle is “not detectable by Fourier analysis”.)
Just to clarify my early comment on the 12mo diff as a filter, it is quite nice above 12mo, so suitable for what you suggested. It is basically the 1/f attenuation of the diff but falling to a zero at 12mo and with a max at 24 mo.
However, below 12mo there is also a zero at 6mo, 3mo, 1.5mo etc. That is the repetitive notch behaviour that I was recalling. I’d wouldn’t mind betting that every second lobe in that region is inverting the data. So it would probably be OK to remove a strong annual cycle prior to doing FT but would seriously corrupts the data if used as a time series filter.
Mike,
Agree with your comments, and thanks for posting the Honolulu results. I was offering Willis an adequate rather than a best methodology.
I just confirmed your Honolulu results, using a 12 month difference series followed by a 13 month box filter applied to the derivative series. Ran an FFT. A very crude 5 minute job, but out pops a cycle at 10.7 years, with an amplitude of 7.95mm/year.
This translates (by integration) into an amplitude of the level cycle of MSL of around 14mm if I have done my sums correctly – highly visible via a Fourier analysis.
Not all local records produce such a clear signal – because of latitude and geography dependence.
Thanks for the confirmation Paul. Good to have two different heads using different tools.
The Ijmuiden data from the same source has a strong 10.8y too. It’s on the dutch coast in the North Sea so makes a good alternative to Hawai’i
( pronounced : “Ay-mow-den” )
Paul_K “…. a cycle at 10.7 years, with an amplitude of 7.95mm/year.
This translates (by integration) into an amplitude of the level cycle of MSL of around 14mm if I have done my sums correctly ”
7.95*10.5 / ( 2* pi ) = 13.5 , looks right.
Anyone who knows –
Hitherto I had assumed at an increment of extra irradiance from sun to earth would cause ocean expansion or contraction almost immediately, as in instantaneous less any delay caused by slower progress through the atmosphere. My thought was that an alcohol thermometer seems to respond almost at once, given it takes time to change the temperature of the enclosing glass.
So, can we expect a change irradiance large enough to be detected to change ocean temperature at the surface immediately? As in soon enough to respond accurately to spectral analysis of frequencies as Willis has done here?
If the change is not instantaneous, where does the incoming energy reside in the lag period?
No Geoff, a change in radiative flux will cause rate of change of temperature. Then negative feedbacks in climate will eventually counteract it and a new settled level at a slightly higher ( lower ) temperature will result. It depends up on the thermal inertia of the system and how all the feedback interact as to how long that will take.
This is why the vast amount of climatology that tries to asses climate sensitivity is physically wrong they are regressing quantities that are out of phase. They say that climate takes decades to settle yet regress dRad and dTemp as though they were in-phase. This leads to all sorts of false negatives and false attribution problems.
The only thing that may just about work is a very slow and fairly constantly rising quantity like GHG forcing and all other factors are likely to be rejected and/or falsely contribute to AGW. If that’s what someone expects to see and they have little knowledge of the physical sciences, they will publish and become an expert on climate.
It’s a bit like the SST vs CO2 discussion. You need to look at d/dt(CO2) to see the correlation with SST. It is then in phase and can be correctly correlated.
whoops, I’d better close that strong tag or else every thing with follows will be bold 😕
This question was touched upon by Roy Spence ( Spencer & Baswell 2010 IIRC )
The temp record is a mix of in-phase and orthogonal data which makes assessing sensitivity to the driving radiation changes very difficult.
There is a detailed discussion about this here:https://climategrog.wordpress.com/2015/01/17/on-determination-of-tropical-feedbacks/
https://climategrog.wordpress.com/2015/01/17/on-determination-of-tropical-feedbacks/
PS, the phase information is lost when just plotting the magnitude of the FT which is what Willis, I and Paul_K are doing. So that is not relevant here.
Geoff,
Further to Mike’s comment, you need to distinguish clearly between energy and flux, which is energy per unit time.
If you turn the gas on a pan of water, then you have added to the heat flux (forcing). This causes the water to gain temperature in accordance with its calorific properties or heat capacity. As the water gains temperature, it also loses more heat to the atmosphere (temperature-dependent feedback). The net heat flux into the pan is the difference between the heat flux added by the gas and the heat flux lost by the water cooling to the atmosphere. The integral of the net heat flux over time gives you the total energy gained by the pan of water. If the calorific value of the water is Cw, expressed here in joules per degree centigrade, and we measure the change in temperature as T deg C, then the total energy gained by the water is then
E = Cw * T
If you differentiate this w.r.t. time, then you obtain the net heat flux going into the pan, hence:-
dE/dt = net heat flux = Cw* dT/dt. This is the rate of gain of heat energy of the water.
The heat flux into the pan from the gas is given by F, say. For small temperature changes, we can use a linear approximation for the cooling of the water:in the pan:- heat flux from the water = λT
The difference between the two must also equal the net heat flux going into the pan.
Hence,
The net heat flux going into the pan = F – λT
Equating the net heat fluxes, we obtain for our pan of water:-
Cw*dT/dt = F – λT
It is no coincidence that this equation looks identical to the single-body linear feedback equation much loved by many climate analysts. For the climate version, Cw represents the heat capacity of the oceans (or just the mixed layer), F represents a flux forcing and T represents global surface temperature (or just the sea-surface temperature). Please note that there is no missing energy implied by this system. If the energy arrived, then It is either in the form of heat in the water or it has been lost via cooling to space.
We can see immediately from this equation that we should not expect a simple relationship between forcing and temperature – Mike’s main point above.
If we apply an oscillatory forcing to this theoretical system, we also find (by solving the equation) that the temperature response is oscillatory and has the same periodicity as the input forcing. However, it is phase-shifted relative to the input forcing.
The net flux response is also oscillatory and is phase-shifted by exactly 90 degrees from the temperature response, (since dT/dt is phase shifted from T by 90 degrees). So, we find then that the peak in net flux leads the peak in forcing which leads the peak in temperature, with a phase shift between net flux and temperature of 90 degrees.
This theoretical system is however somewhat defective, because of (amongst other things) oversimplification of the ocean model. The sea surface temperature is responding to the mixed layer temperature. However, it is a reasonable assumption that the warmer the mixed layer, the faster its rate of heat loss to deeper ocean. If the simple mixed layer model is replaced by a more sophisticated ocean model, then the phase separation between net flux and temperature ceases to be 90 degrees. Instead, the phase separation has an upper bound of 90 degrees.
I hope this helps rather than confuses.
since the forcing is driving the flux, I’m not clear on how you see the flux changing first. Is this what you intended to write?
I’m inclined to defer to your expertise on this subject but wouldn’t the eddy diffusion also just represent a negative feedback and simple modify the value of lambda? Could you elaborate? Thanks.
Mike,
Although it may seem counterintuitive, yes the peak in net flux leads the peak in forcing for an assumed oscillatory forcing input for this model. The forcing in this instance is an unfettered independent input by assumption. It immediately induces a change in both temperature and in net flux. The forcing goes on to reach its first peak – unfettered – while the net flux finds itself increasingly constrained by the restorative cooling feedback. In this race, the net flux tails over before the forcing hits its peak. You can see this by solving the equation analytically for a sinusoidal forcing input, which can be done very simply with an integrating factor.
The loss of heat from the mixed layer is normally captured either as an upwelling diffusion term – with heat loss proportional to the temperature gradient – or, in a multiple body model, as something linearly proportional to the temperature difference between the mixed layer and the (next) deeper layer. In either event, this does not effect the feedback term – which is intended to reflect the radiative response from the surface upwards. In effect, for a two-body ocean model, the single-body governing equation which I described above would become :-
Cw *dTs/dt + K*(Ts-Td) = F – λTs
where the subscripts s and d refer to the surface and the deep ocean layer respectively.
You can see, I hope that this has not changed the radiative feedback term ( λTs) at all. Nor should it change the total net flux described by the RHS of this equation. What has happened is that the flux forcing F is now heating both the mixed layer and the assumed deeper layer.
The same thinking applies for an upwelling diffusion formulation or an n-body model where n>2 – they only affect the LHS of the energy (or actually flux) balance equation.
Thanks for the equation Paul, always helps to have some concrete to refer to. I’ll do some algebraic rearrangments:
Cw *d/dt(Ts) + K*(Ts-Td) = F – λTs
Cw *d/dt(Ts) = F – λTs- K*(Ts-Td)
Cw *d/dt(Ts) = F+ K*(Td) – Ts(λ+K)
So if Td is considered a quasi-constant deep ocean temperature we have a slightly modified forcing term and a new “effective” lambda. This is what I meant by it acting as an additional negative feedback.
Within the assumption that the temp of the bulk of the deep ocean does not change measurably, this equation seems to be essentially of the same form as the original, unless I’m missing something.
.
I think the substitution solution you are referring too is what you provided at Lucia’s for a sinusoidal forcing:
http://rankexploits.com/musings/2013/estimating-the-underlying-trend-in-recent-warming/#comment-116290
T = Asin(ωt) – Aωτcos(ωt) + Aωτexp(-t/τ)
where A = S/(1+(ωτ)^2)
The exp is a transient term that can be ignored in the settled response to a constant oscillation. Sketching out sin(ωt) – cos(ωt) it peaks between pi/2 and pi, after the forcing term of sin(ωt) and crosses zero ( max rate of change ) somewhere between 0 and pi/2 , ie. leading as you correctly said .If the time-const is small the cos term becomes less important and the phase of temperature is close to that of the forcing.
Cw *d/dt(Ts) is the flux term so it is the long time-constant ( small neg. feedback ) case where the flux is nearly in phase with the forcing.
Which all goes to show that the trivial linear regressions that are the main stay of attempts to assess climate sensitivity from observational data are fundamentally flawed and can be assured to give the wrong result.
The presence of both the in-phase and the orthogonal terms will dilute the regression slope. If dRad is used as the ordinate ( x-axis ), as is almost universal practice in climatology, the slope will be too low and the deduced climate sensitivity ( reciprocal of the slope ) will be too high.
Mike,
Your rearrangement is fine, but now to get an approximate solution for the two body problem, you can use the known solution of the first equation – the single body equation – with the deeper temperature fixed at a constant value and appropriate modification of the parameters, to solve for Ts and for dTs/dt. The net flux term however is not Cw*dTs/dt – which would be 90 deg out of phase with T, but is CwdT/dt + kT -kTd. This is a linear combination of a sin and cos function of the same phase plus an approximate constant, kTd. Hence net flux moves to being less than 90 deg out of phase with T. Alternatively we can say that 90 deg represents an upper bound on the phase difference.
This conversation about relative phasing of net flux, forcing and temperature response is very pertinent to Shaviv’s work – and why it was never appropriate to assume that the solar forcing should be in phase with MSL or with rate of change of MSL. The fact that MSL is also responding to additional changes (notably mass) adds a further complication, but overall, he was IMO perfectly correct to leave phase as a free variable – something that Willis seemed to think was wrong in some way.
Corrigendum: I wrote “This is a linear combination of a sin and cos function of the same phase …” when I should have written “This is a linear combination of a sin and cos function of the same periodicity…”
Sorry.
Thanks for the confirmation that this can still be treated as a simple linear feedback, with slightly modified parameters. You say that the eddy diffusion is usually treated as upwards +ve presumably following the same logic as +ve downwards for atmospheric fluxes. Thus K would be a negative constant.
This phase issue is a major problem for climatology and is at the heart of much of over-estimation of CS. As recently as Santer et al 2014 we find them still not getting it.
Original caption is “ENSO and volcanoes removed” yet we can see a major disturbance remains after Mt P. The initial dip shows that they are under-scaling volcanic effects and the following bump is the out of phase climate reaction.
Credit goes to Santer et al for clearly documenting this residual error in the paper, though it is a poor state of affairs that after 30 years of concentrated effort and resources they still have not got beyond high school level physics. This is first year undergraduate work in most hard sciences.
Marotzke & Forster 2015 make similar mistakes by repeatedly shunting the out of phase components into a “random error” term then applying regression as if the response was in phase with the forcing. These guys hold chair positions at major universities. Astounding.
Paul,
Thanks for the reply. My email is sherro1 at optusnet.com.au
In about 1972 I did some experiments with radon on the fringes of the Ranger (undeveloped then) ore bodies, looking at Radon movement.
A colleague who lived on site had a baby delivered by his wife and was up and down in the night. So, I asked him if he would duck out and take Rn and temperature measurements at various depths in the soil.
Here is a graph that seems to show peaks and troughs through the day rather where one would expect with a fast response system
http://www.geoffstuff.com/jabsoilt.jpg
There is the other observation that swimmers note that the top few cm of calm sea is warmer at noon than at midnight.
Could it be that you are modelling starting conditions while I’m observing near-equilibrium conditions?
The other problem I have is – if there is a lag as you note, 90 deg out of phase, where does the energy go while it awaits the right phase?
Why does an alcohol thermometer reswpond near instantaneously and the shallow sea not?
Cheers Geoff.
Sorry for the late reply – severe health problems in the family.
Willis
Re: Literature sea level periodograms showing 11-13 yr cycles vs Eschenbach
You said:
Further to my comment raising the periodogram of New York City by Scafetta (2013) showing an apparent solar cycle, see:
Kenigson, J. S., and W. Han (2014), Detecting and understanding the accelerated sea level rise along the east coast of the United States during recent decades, J. Geophys. Res. Oceans, 119, 8749–8766, doi:10.1002/2014JC010305
Note especially Fig. 1B .
These power periodograms clearly show an apparent solar cycle around 11 – 13 years for ten ports: Eastport, Portland (Maine), Boston, New York (The Battery), Philadelphia (Pier 9N), Atlantic City, Baltimore, Annapolis (Naval Academy), Sewells Point (Hampton Roads) and Charleston. These appear to have a similar order of magnitude to the multi-decadal oscillation of ~ 55-65 years. (Conversely I do not see the annual cycle shown in this figure).
So why do Kenigson & Han (2014) show 11-13 year cycles (solar cycle?) in 10 sea level periodograms, similar to Scafetta (2013) Fig. 3, yet you do not find such in your sea level periodogram analyses?
Scafetta, N. Common errors in analyzing sea level accelerations, solar trends and temperature records. Pattern Recognition in Physics 1, 37-58, doi:10.5194/prp-1-37-2013, 2013
PS Thanks for the correction in this post.
David,
I hope that Willis is still listening. Another reference which is worth looking at (also already mentioned in previous threads) is Jevrejeva 2008. Although his focus was on the quasi-60 year periodicity in the MSL dataset, his results based on tideguage data back to 1700 also clearly show evidence of something that looks remarkably like the solar cycle.
See Figure 2 from here:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.178.7972&rep=rep1&type=pdf
Yes, Willis seems to regard absence of a solar signal as proof of his tropical governor hypothesis and is a bit too invested in trying to prove a negative. I think he is basically correct that tropics are very insensitive to radiative change, however, this is less true for non tropical zones.
I have not seen him comment on this explicitly but I get the implied idea that he considers the topical ‘governor’ also means that other zones are insensitive too, which is less the case. Oceanic and atmospheric circulation ensure the tropics have a stabilising effect but mid latitudes are notably more sensitive to radiative forcing.
If there is some expansion in extra-tropical regions due to solar, the additional volume has plenty of time in 11 years to find its way around and be visible in all regions.
Reported average global temperatures fluctuate in a non-physical manner. Effective thermal capacitance prevents true energy level of the planet from changing so quickly. The effectively random fluctuation results because temperature distribution is not smooth, temperature measuring points are discreet and, for satellite based measurements, local weather causes variations.
Reported average global temperatures fluctuate in a non-physical manner because so-called scientists insist on averaging the temperature of two physically incompatible media: water and air, which have specific head capacities that differ by about three orders of magnitude.
The result is physically meaningless before you even work out what the numbers are.
But because air warms quicker it boosts the rate of “global warming” and that is the required result. Science be damned, this is politics.
What does any of that have to do with the impossibly rapid fluctuations in reported temperatures which are dominated by (approximately 71%) ocean surface temperature measurements?
I’ve been playing around with MODTRAN and the Ice Core Data and it is pretty hard to make the case that CO2 causes warming.
1) According to the Vostok data the ice age bottomed 18600 years ago, with CO2 at 185ppm, temps -8.73&Deg;C
2) Over the next 1,400 years CO2 remains basically unchanged, yet temperatures increased a full 1.26°C.
3)CO2 doesn’t start increasing until 1,200 years after temperatures began increasing.
4) CO2 bottomed at 182ppm, that puts the outgoing radiation at 292.993W/M^2 at its bottom. Assuming clear skys and 70m altitude.
5) Temperatures increase a full 9.34°C over the next 7400 years, and CO2 increased by 67ppm. That puts outgoing radiation at 291.455W/M^2. 1.5W/M^2 by CO2 corresponded with a 9.34°C increase.
6) Temperatures the drop from 0.616°C 11200 years ago, to -1.114°C 3000 year ago. CO2 went from 252 to 278, putting the outgoing radiation at 290.984W/^M2 . A 1.72°C drop corresponded with an increase in CO2 absorption of 0.5W/M^2
7) CO2 has then increased to 400ppm, resulting in 189.228W/M^2. A 1.7W/M^2 increase. Temperatures have increased 1.6°C.
8) The current temperatures are below the Holocene Max.
Bottom line, the change in temperatures seem to be unrelated to the changes in W/M^2 absorption by CO2.
That’s some interesting stuff you’ve been doing but I think you posted to the wrong thread. This is about solar and sea level, I think you were commenting on the thread on recent paper about “correcting” bolder dates to give the policially correct, required result.
Willis,
I have run a number of tests on various tide-gauges.
To render the cycle visible in the level series you need to do the following pre-processing
(a) remove the linear trend to stabilise the low frequency results
(b) run a 13 month box filter over the data (or better still a low pass filter)
(c) run a Fourier analysis
Alternatively, to render the cycle visible in the derivative series, you can:-
(a) convert the data to a 12 month difference series
(b) run a 13 month box filter (or better still a low pass filter)
(c) run a Fourier analysis
(d) convert the amplitude estimates from the derivative series (mm/yr) back to estimates for the level series in mm
These two approaches give me consistent estimates of amplitudes for the level series for the wavelengths of interest.
As a reference case, amongst other tests, I duplicated the New York results posted by Scafetta. This reveals an amplitude of about 10mm at a periodicity of 12 years, using data from 1906 forwards.
The fact that the average global amplitude of this cycle may be around 2.5mm does not imply that it is 2.5mm everywhere. If you wish to argue that the cycle is not visible in the tide records then – a la Feynman – you need to go to the tide records where it should be most visible and demonstrate that it is not present.